SparkBuild source code and licence terms

Replies to This Discussion

Permalink Reply by Scott Castle on November 3, 2009 at 3:54pm

Hi Semih,

a) We have no plans to publish the SparkBuild source at this time; it uses much of the same code as Electric Cloud's commercial products so it's hard for us to release it. However, we are committed to keeping SparkBuild free for personal *and* commercial use (we won't be making it a for-pay app once everyone starts using it, for example!)

b) SparkBuild is designed around the GNU make model - so it can be extended with things like the GNU Make Standard Library (by John Graham-Cumming) as well as building on the extensive background of makefile 'prior art'. The SparkBuild Insight tool also supports user extensibility - custom reports can be written for sbinsight and dropped right into the UI.

c) Since SparkBuild replaces make, it is really language-agnostic. SparkBuild can be used with anything that can be built with gmake 3.80, gmake 3.81, or NMAKE. Although it's not nearly as common to use makefiles with languages like Scheme (or Java, which we occasionally see) it is certainly possible, and it will work fine.

d) We're not currently planning Solaris support, but may consider it in the future. (Electric Cloud's commercial offerings do support Solaris on SPARC)

• ▶ Reply

Permalink Reply by Semih Cemiloglu on November 3, 2009 at 4:26pm

Hi Scott,

a) Understood. However, as a potential user I should warn you that keeping the source code closed will force you to relinguish some of the benefits of open source code available. See (b) for details. If this is your plan I think you should release an API (maybe C API?) for emake so that users can bend it to their will.

b) I'm following Mr. John Graham-Cumming's work and aware of the GMSL. Please note that GMSL is only an extension to GMake language, not the core application (GMake) that executes that language. Another interesting article fro Mr.Make column is "GMake User Defined functions part #2"
http://www.cmcrossroads.com/ask-mr-make/10194-gnu-make-user-defined...
in which he described how to extend GMake source code to define a new function within the application and make it available to the GNU Make language.
I wish to see same or similar flexibility in emake as well (back to C API argument).

c) These languages, especially Common Lisp in particular has substantial differences to C/C++ compile-link-run model. Lisp blurs the line between in what compilation and run-time is, thus needs to access dependency graph even while the resultant application runs. See
http://www.cliki.net/ASDF%20System
for details. I wish to see emake catering needs of a Lisp system or application.

d) I will not hide that I'm big fan of OpenSolaris (www.opensolaris.org). I contend it's a terrific development environment for software developers. I wish to see SparkBuild accessible to OpenSolaris developers. Since most of the GNU toolchain is available in OpenSolaris (Gcc, Gnome etc), porting to this platform might mean a simple recompilation.

Thanks, regards,
Semih

• ▶ Reply

Permalink Reply by Scott Castle on November 3, 2009 at 5:01pm

I'm interested in learning more about your ideas on a C API for make, especially since makefiles are effectively descriptions of state machines. What would it do?

Regarding extending make: The idea behind Electric Cloud's 'emake' is to maintain perfect compatibility with gnu make while fixing some of make's inherent problems with parallel execution and cross-submake dependencies - we've always been very cautious not to try and 'improve' the syntax and interfaces of make, in order to prevent the perils of lock-in (you can readily switch between gnu make and emake, for example).

I don't know a lot about the day-to-day tasks with ASDF and Lisp development so i'm not sure what emake might be able to do; where do you see gaps in what ASDF provides?

• ▶ Reply

Permalink Reply by Semih Cemiloglu on November 4, 2009 at 3:22pm

Hi Scott,

Regarding extensibility of emake:

a) It should be possible to define a new function through the C API. That is within make scripts I should be able to use a new function
$(user-defined-function arguments)
which is really defined via the API, may be using dynamic libraries.
This simply use existing make syntax.

b) It should be possible to access internal structures that emake keeps while it's running. Most notably the dependancy graph between the source modules and the file system related data might be of concern.

c) As I indicated earlier, Lisp compilation model may require re-compilation of any module even an application is running. In order to get any benefit from emake here, emake should be embeddeble within a host application.


I understand that primary design goal of emake is compatibility with exisiting de-facto make applications. However, the ideas I listed (extensibility and embedibility) are not really obstacles to this goal. If users choose their scripts to completely be compliant with gmake/nmake you can still do so.

Also, since your aim is to keep the source closed, I contend that you *need* these kind of flexibility, to expect any reasonable uptake of SparkBuild.

HTH, Regards,
Semih

• ▶ Reply

Permalink Reply by Scott Castle on November 5, 2009 at 10:27am

These are really interesting possibilities. Some of the engineers on the SparkBuild team have echoed option 'a' - we're considering something like this for a future release. Option 'b' is actually much more interesting within the scope of our commercial product, ElectricAccelerator, because it is able to collect file usage data and from it discover source-level dependencies automatically; SparkBuild still has to rely on the correctness/completeness of the makefiles, just like gnumake.

For 'c' - how much of a problem does this currently represent for the Lisp developer? Does Lisp (please excuse my ignorance) know what sources need to be recompiled, or does it lack for the correct source dependencies? How long are re-compilation cycles generally? Are the dependencies on source modules at a file level, or are they 'component dependencies' which are based on component versioning, a la maven?

Thanks for your feedback,
- Scott

• ▶ Reply