Optimized Pascal compilation time
I was helping a customer to migrate from Delphi 5 to Delphi 2006 for some time ago.
It was almost only to reopen and compile the project except for some minor changes. We had to add a few units (i.e. variants) to the uses clause here and there and fiddle with conditional defines in a few places to get it to compile with both D5 and D2006. A third party component was not a backward compatible as you cold wish but in general the major work was done in less than a day.
But there where one big issue that we had to address first. The size of the project was about a few hundred forms and half a million lines of code. It took 90 minutes to compile in D5 and even longer in D2006. What is going on here? Ninety minutes for half a million lines of code is not the speed of the lightning fast Pascal compiler as I know it!
After analyzing the code I found that there was a general pattern of including a lot of units in the uses-clause in every unit. It appeared that this crisscross of references had a non-linear time complexity on the compiler. The only way to fix it immediately was to remove as many units as you can from the uses-clause in the interface section or move them to the implementation section. I imagined a nightmare of daunting manual work…
In the end the recipe to fix this was rather simple. It involves some free tools and open source tools but it does the job automatic. And the result was a rather successful optimization. We reduced compiled time from 90 minutes to 50 seconds – over 100 times! And the linker could remove 100.000 lines of unused code. I was pretty pleased with myself that evening. 
Ingredients
- ICARUS – a free tool from the maker of Pascal Analyzer. http://www.peganza.com/products.htm. Icarus parses Delphi or Borland Pascal source code and generates a Uses Report. This report will help you remove unneeded units from your uses lists. You will also know which units that can be moved from the interface uses list to the implementation uses list.
- Uses Cleanup Tool – by Oleg Zhukov http://cc.borland.com/Item.aspx?id=23199.
Step-by-step
- Run ICARUS on your delphi project to get a report of which units depends on each other.
- Input this report in the Uses Cleanup tool to automatically update all uses-clauses in your source code.
- Delete all DCU-files and recompile.
It can be nice grabbing instrument for source code. Not need do especial file search on hostile hard disk!
There also used to be an integrated expert called OPxperts that did both steps in one, although you’d (thankfully) remain in manual control of actually deciding which “unused” units to delete. Unfortunately OPxperts has been abandoned and has not resurfaced since D7. So wouldn’t it be great if this kind of functionality was made available from the “inside” in BDS, from DevCo…!!
I’ll second that!
Try CnWizards.
It has an expert inside which does the exact same thing automatically!
Hi,
Icarus just don’t generate the two files (status.txt and uses.txt) that the Oleg’s tools says it should. Then I did a copy and past from the uses report on screen to a file named “uses.txt”. This time the Oleg’s tool loaded and parsed the file correctly. But when I pressed de “Modify Units” button the tools says that the units modification failed.
the output folder for the .txt files is
something like
C:\Program Files\Peganza\Icarus – Uses List Analyzer for Delphi\this is the project name
So, you can copy them by hand from there.
However, I still get the modification failed message.
Fredrik: did your project have any relative search paths, or did it use the global search path?
I’m trying to understand what steps I need to take to get this to work.
Thanks,
Patrick
Hi,
50 seconds is still a lot of time.
To reduce it, you should break down your project into packages. Each package will then compile by itself, which will take more faster.
To break the project into packages, you need to understand the graph of the unit uses. This graph is, in fact, neighter a tree nor a DAG, but a full graph, with possibly circular references. Now, you should identify each strongly-connected component (SCC) of the graph. You can then order the SCCs using topological sort. Each SCC is then both in a package, and it references exactly the SCCs which are its out-links in the SCC graph.
For creating the SCCs decomposition you can either use your understanding of the project, or some automated tool. I created a simple parser for the Icarus output files which turn them into a .viz graph that I can display using the GraphViz library. This allowed me to break down my huge project into several smaller packages. The benefits are faster compilation times and less coupling.
Note if that use Delphi.NET then this entire process requires no changes at all to the .pas files. Just creating of new packages, including files in them, and creating references between them.
Regards,
Arie
This is something the Delphi compiler should do for us. If we have an un-needed Uses clause, it should report it as a warning (with a compiler flag to optionally turn of reporting of that 1 issue).
That way we would have the option to code clean apps from the start instead of wait until they become huge unmanageable frameworks of grief and suffering that require us to use kludge apps.
Just a thought.
Problem with the Uses cleanup tool is (if it’s the one I remember) that it renames your files to the same file name but all-caps, which hoarks my Version control comparison with starteam.
“Non-Linear Time Complexity”??? Is that the name of your band?
Spaceman, indeed it could be the name of my band
But it is still available so you can take it if you want!
I guess my text could have been clearer… I mean that the has super linear or exceeds linear time complexity in relation to the size of the project when you add almost all units available in the project to the uses clause in every unit.
Thanks, that really clear things up.
i had used opxperts on D7 projects in the past, but it is not supported for D2006. after reading your blog item, i tried icarus on some D2006 source but it seemed not to recognise the keyword “final” and so it could not compile my project. then i went to look for cnWizards mentioned above by Delpho, but D2006 seems not supported by them (yet?). i think these kind of experiences add some weight to the words of Kristofer, Roy, and C above – that this functionality should be provided by Borland/DevCo and incorporated into the IDE.
The tool by Oleg Zhukov sounds nice, but I can’t find it at the given URL. There are also slightly different URLS out there. Which one is correct?
Thanks
-Lübbe
The tool from Oleg can also be downloaded here:
http://www.olegzhukov.com/content/Downloads/tabid/73/Default.aspx