10-minute 'self-pilot' for SparkBuild

At Electric Cloud, we're always setting up some new build environment - but we're pretty much in build central by the nature of our products. For most developers, this sort of thing happens once in a while, not every day! So if you're interested in trying out SparkBuild, but don't have a project which is easy to try it on right now, let me give you a ten-minute "self-pilot" that you can do right now.

These instructions are for Windows (because it's the more complicated test platform) - Linux is similar but you won't need to install cygwin (lucky you!)

Step 1
Grab the binutils-2.20 sources [tar.gz] from the GNU FTP server.

Step 2
Install Cygwin from the Cygwin site. Be sure to install all the usual development packages, like make and gcc. You'll also need to install the libiconv package (that's one I had missed the first time through).

Step 3
Install SparkBuild.

Step 4
Open a cygwin shell, and do something like this:
cd /cygdrive/c/example
tar -zxvf binutils-2.20.tar.gz
cd binutils-2.20
./configure

Those commands will unpack the binutils sources and configure them for your system. If the configure was successful, you're good to go. If configure failed, you probably haven't installed the right tools in cygwin - go back to step 2 and select the right packages (or just install everything).

Step 5
Keeping that shell open, now run:
export 'EMAKEFLAGS=--emake-emulation=cygwin --emake-subbuild-db=/cygdrive/c/example/binutils-2.20/emake.db'
(On linux, use this string instead: export "EMAKEFLAGS=--emake-subbuild-db=/example/binutils-2.20/emake.db"

This will set up your SparkBuild environment.

Step 6
To generate the SparkBuild database, and to make sure that you've got everything installed properly, run a full build:
emake --emake-gen-subbuild-db=1 all

If this is successful, proceed to step 7. If not, you are probably missing some cygwin packages - correct the issues, run 'emake clean', and re-run step 6.

Step 7
Now that you have a buildable tree, and a complete subbuild database, you're all set to test SparkBuild out. Here's an example, to exercise the subbuild function. We'll simulate a change to an include file while working on the binutils/readelf.exe tool.
touch include/aout/ar.h
cd binutils

Now try running an in-directory incremental:
make readelf.exe

Notice how nothing happened? That's what is wrong with just building the component you're working on - make can't detect the out-of-component change.

Now try this:
emake readelf.exe

SparkBuild, with its subbuild database to inform it of dependencies, notices the header change, and rebuilds the dependencies readelf.exe needs.

Why is this interesting? Figuring out what subcomponents are required for building a target can be difficult and time-consuming, even on a code base as small as this. The only way to guarantee correctness, and to skip that manual task is to do a top-level incremental:
cd ..
touch include/aout/ar.h
make all

But of course, running this top-level incremental takes more time - about 25% more, on this small example.

This short self-pilot should give you a little hands-on experience using subbuilds. Now you can take this technique and apply it to your own projects. (If you'd like more information on the concepts behind subbuilds take a look at Eric Melski's post at the Electric Cloud blog). Be sure to post a comment with your results!