PDFsharp & MigraDoc Foundation
https://forum.pdfsharp.net/

adding thousands of table rows
https://forum.pdfsharp.net/viewtopic.php?f=2&t=4297
Page 1 of 1

Author:  Jessica [ Mon Nov 15, 2021 5:01 pm ]
Post subject:  adding thousands of table rows

Hi

I made a table in migradoc with 4 columns.
I add 40000 rows, with each 4 columns in the table.

However it takes forever to do this and the creation of the PDF never finished (waiting more then 10 minutes.

Is migradoc not made for those big amounts of data or what could be wrong?

Some example code to fill the table I use is:

Code:
foreach(Machine machine in MAMEParentsAndCLonesList)
            {
                Row newRow;
                newRow = this.tableSoftwareListScan.AddRow();
                newRow.TopPadding = 1.5;                     
             
                newRow.Cells[0].Format.Font.Bold = true;

                newRow.Cells[0].Shading.Color = TableGray;
                newRow.Cells[0].VerticalAlignment = VerticalAlignment.Center;
                newRow.Cells[0].Format.Alignment = ParagraphAlignment.Left;
                newRow.Cells[0].Format.FirstLineIndent = 1;
                newRow.Cells[0].AddParagraph(machine.Name);

                newRow.Cells[1].Shading.Color = TableGray;
                newRow.Cells[1].VerticalAlignment = VerticalAlignment.Center;
                newRow.Cells[1].Format.Alignment = ParagraphAlignment.Left;
                newRow.Cells[1].Format.FirstLineIndent = 1;
                newRow.Cells[1].AddParagraph(machine.Publisher);

                newRow.Cells[2].Shading.Color = TableGray;
                newRow.Cells[2].VerticalAlignment = VerticalAlignment.Center;
                newRow.Cells[2].Format.Alignment = ParagraphAlignment.Left;
                newRow.Cells[2].Format.FirstLineIndent = 1;
                newRow.Cells[2].AddParagraph(machine.Year);

                newRow.Cells[3].Shading.Color = TableGray;
                newRow.Cells[3].VerticalAlignment = VerticalAlignment.Center;
                newRow.Cells[3].Format.Alignment = ParagraphAlignment.Left;
                newRow.Cells[3].Format.FirstLineIndent = 1;
                newRow.Cells[3].AddParagraph(machine.ROM);
            }


thx

Author:  TH-Soft [ Mon Nov 15, 2021 6:06 pm ]
Post subject:  Re: adding thousands of table rows

Jessica wrote:
However it takes forever to do this and the creation of the PDF never finished (waiting more then 10 minutes.
It seems you are a bit impatient.
Maybe try with 100 rows, 1000 rows, 10000 rows and see how long it takes.

Maybe try rendering to RTF instead of PDF, that should be much quicker. Then try Word to create the PDF.

Creating such large tables with PDFsharp should be much quicker than with MigraDoc.

Author:  Jessica [ Tue Nov 16, 2021 2:40 am ]
Post subject:  Re: adding thousands of table rows

Are you the developer of migradoc?

If yes, would it be possible to look in the source code what's wrong with migradoc?

I added 40000 rows in a table and this gave me a 1100 page pdf file.
It took 25 minutes to generate on a fast gaming pc, core i7 4.2 ghz 16 GB ram. So 25 minutes for 1100 pages! (Just this table)
What if somebody is making an encyclopedia? Then you wait for a whole day...

This means migradoc is very slow.

When will this'issue be fixed and when we be able to download updated nuget migradoc package?

I can't use it when there is no faster performance.
I don't need RTF , only PDF and I have no pdfsharp knowledge

Thx

Author:  Jessica [ Tue Nov 16, 2021 2:46 am ]
Post subject:  Re: adding thousands of table rows

Is this patch alreadyin the nuget package?

viewtopic.php?t=679

Author:  TH-Soft [ Tue Nov 16, 2021 10:06 am ]
Post subject:  Re: adding thousands of table rows

Jessica wrote:
Is this patch alreadyin the nuget package?
No.
The NuGet version is faster than the old version with the patch.
See here (same thread):
viewtopic.php?p=9380#p9380

Author:  Jessica [ Tue Nov 16, 2021 9:04 pm ]
Post subject:  Re: adding thousands of table rows

Since you don't give an answer to the performance problem, does this mean migradoc is abandoned and not updated anymore?

thx

Author:  TH-Soft [ Wed Nov 17, 2021 2:00 pm ]
Post subject:  Re: adding thousands of table rows

Jessica wrote:
Since you don't give an answer to the performance problem, does this mean migradoc is abandoned and not updated anymore?
I had missed you earlier post and only replied to your second post. Sorry.
MigraDoc is not abandoned. Some operations require time. The amount of time depends on the computer that is used.
You say there is a performance problem.
MigraDoc works fine for our use cases.

1100 pages in 25 minutes, that is 44 pages per minute or about 1.5 seconds per page. Not good, but it works.
Table rendering is a bit slow for large tables. An encyclopaedia with 1100 pages should render much faster, assuming most of the text is outside of tables and only small tables are used.

It is possible to run the document generation in a profile to see which parts of the code consume most of the time. You have the code, so you can do it.
I don't have your code, so I cannot do it.
That's what I did to make MigraDoc much faster some time ago, using my test code and my test table.
I'm not the original author of this MigraDoc implementation.

For your use case, PDFsharp could be the better choice, assuming that performance is very important. Implementation takes a bit longer, but rendering will be much faster.

Author:  Jessica [ Wed Nov 17, 2021 9:24 pm ]
Post subject:  Re: adding thousands of table rows

Are there things in the table code of migradoc you think you can make faster? I'm happy to test things if you release BETA packages on github!

What should I profile? I add 4 columns in a loop (see my code).
There is no other code. You have ALL code...

Author:  Jessica [ Wed Nov 17, 2021 9:42 pm ]
Post subject:  Re: adding thousands of table rows

How much faster do you think pdfsharp is? 50%, 100%, ... faster for the same table (see my code)?

Is it possible to give me a complete code example (for pdfsharp) on how to make a table with 4 columns and add 1 row to it.
Also how to insert a hyperlink in one cell with pdfsharp? I then can copy paste it. I don't need more then this table :-)

I have no pdfsharp knowledge and there are almost no code examples to be found.
Pdfsharp is much more low level then migradoc, so it would take weeks to learn this.

Author:  TH-Soft [ Thu Nov 18, 2021 8:04 am ]
Post subject:  Re: adding thousands of table rows

Jessica wrote:
What should I profile? I add 4 columns in a loop (see my code).
There is no other code. You have ALL code...
I don't have the data for 40,000 rows. When adding "foo" and "bar" to all cells, the timing could be different and I might miss some bottlenecks that make execution slow on your computer.

Jessica wrote:
How much faster do you think pdfsharp is? 50%, 100%, ... faster for the same table (see my code)?
I guess 10000% or more.

Jessica wrote:
I have no pdfsharp knowledge and there are almost no code examples to be found.
Pdfsharp is much more low level then migradoc, so it would take weeks to learn this.
It takes few hours to learn it.
You just have to draw text and then draw lines around it.
You gain much speed, but lose some flexibility with respect to looks and layout.

The XTextFormatterEx2 class might be useful:
viewtopic.php?p=9372#p9372

PDFsharp samples:
https://github.com/empira/PDFsharp-samples
http://pdfsharp.net/wiki/PDFsharpSamples.ashx

Author:  Jessica [ Thu Nov 18, 2021 9:27 pm ]
Post subject:  Re: adding thousands of table rows

Hi

I have no experience with profiles.
Is it possible to make a table with 4 columns and add the data found in the file data.txt? Here is download link

https://we.tl/t-EOxbDZMwzR

The data for each column is separated by ",". One line is one row in the table. You can use this to split on ",": https://docs.microsoft.com/en-us/dotnet ... ew=net-6.0
If the data between 2 "," is an empty string, this means the column contains just an empty string there. (can occur on several columns, even multiple times in one row)
It are 43908 rows that must be added to the table using migradoc and that took 25 min here.

example, the line "Gunpey (Japan), Bandai / Banpresto, 2000, gunpey" will be 4 columns: "Gunpey (Japan)", "Bandai / Banpresto", "2000", "gunpey".

thank you so much!

Author:  TH-Soft [ Mon Nov 22, 2021 8:47 am ]
Post subject:  Re: adding thousands of table rows

Jessica wrote:
Is it possible to make a table with 4 columns and add the data found in the file data.txt?
It is possible, but I do not have time to do it soon.

Author:  Jessica [ Mon Nov 22, 2021 9:31 pm ]
Post subject:  Re: adding thousands of table rows

Can you download the file and save it somewhere? Then if you have time and want to do it, you have my data available.
The link is not working forever I see.

Page 1 of 1 All times are UTC
Powered by phpBB® Forum Software © phpBB Group
https://www.phpbb.com/