Pushing the envelope of build performance
Build profiling with SparkBuild Insight
Curious about where your build time is being spent? Trying to identify which components take the most time to compile? Making guesses about whether it's worth the effort to parallelize the build? SparkBuild Insight has an array of reports which can answer these profiling questions.
SparkBuild Insight reads the annotation files produced by your build when run with emake. Annotation contains detail information on each make target, including duration, dependencies, and command data. Using this information, SparkBuild Insight classifies each job into a primary action category (compile, link, library link, file I/O, etc) and produces a treemap of the build. The information can be viewed in a variety of ways, including sorting by job time (to find, for example, the longest compiles, or links, or I/O steps).
Reporting on the longest jobs in the build is easy, too; the main display lets you find these by work type, but if you just want to know the overall longest jobs, this report will expose that information immediately.
Other ways to sort this data include looking at the build's aggregate profile:
By sorting the data into duration buckets you can see what size of job your build time is dominated by. In some cases, like our illustrations here, it's not the longest jobs which impact build time, it's the sheer number of jobs in the 15s range. This will inform your optimization efforts - don't spend a week breaking up the two or three long jobs when parallelizing the short ones will pay bigger dividends!
And if you're trying to determine whether it's worth running the build with a parallel tool, the ElectricSimulator report will give you a very accurate picture of potential speedup:
The Simulator uses the dependency and duration information from annotation to calculate the duration of the build based on varying numbers of parallel processes (agents if you're using ElectricAccelerator, or make -J threads). Because the durations are known and the dependencies are as well, this report is reliable; the only thing not taken into account is I/O load on the build machine. The Simulator is a great way to get a quick analysis and see if using more CPUs can help your build time.
SparkBuild Insight is a part of the SparkBuild suite of tools. This analysis works for any build which can run with SparkBuild emake (GNU make 3.80, 3.81, NMAKE 7 and NMAKE 8). You can download installers for Windows and Linux from the main page.
SparkBuild Insight reads the annotation files produced by your build when run with emake. Annotation contains detail information on each make target, including duration, dependencies, and command data. Using this information, SparkBuild Insight classifies each job into a primary action category (compile, link, library link, file I/O, etc) and produces a treemap of the build. The information can be viewed in a variety of ways, including sorting by job time (to find, for example, the longest compiles, or links, or I/O steps).
Reporting on the longest jobs in the build is easy, too; the main display lets you find these by work type, but if you just want to know the overall longest jobs, this report will expose that information immediately.
Other ways to sort this data include looking at the build's aggregate profile:
By sorting the data into duration buckets you can see what size of job your build time is dominated by. In some cases, like our illustrations here, it's not the longest jobs which impact build time, it's the sheer number of jobs in the 15s range. This will inform your optimization efforts - don't spend a week breaking up the two or three long jobs when parallelizing the short ones will pay bigger dividends!
And if you're trying to determine whether it's worth running the build with a parallel tool, the ElectricSimulator report will give you a very accurate picture of potential speedup:
The Simulator uses the dependency and duration information from annotation to calculate the duration of the build based on varying numbers of parallel processes (agents if you're using ElectricAccelerator, or make -J threads). Because the durations are known and the dependencies are as well, this report is reliable; the only thing not taken into account is I/O load on the build machine. The Simulator is a great way to get a quick analysis and see if using more CPUs can help your build time.
SparkBuild Insight is a part of the SparkBuild suite of tools. This analysis works for any build which can run with SparkBuild emake (GNU make 3.80, 3.81, NMAKE 7 and NMAKE 8). You can download installers for Windows and Linux from the main page.
Tags: analysis, build, emake, gmake, gnu, make, nmake, profiler, profiling, sparkbuild
Comment
© 2010 Created by Electric Cloud Administrator
You need to be a member of SparkBuild Community to add comments!
Join SparkBuild Community