I’ve been thinking a lot about writing build scripts for code recently, and have come to a few conclusions.
So, I’ve never really liked writing code in xml. It’s normally verbose, akward, and counter-intuitive. There, I’ve said it.
It’s a very unpopular point of view, and has caused a number of arguments in the past (techies are very passionate, and entrenched, in certain design choices).
Xml is a great as a way of describing data (and is indeed the reason it was made) and is thus rightly applied to web services, web pages, and some levels of configuration, but not for writing code.
Incidentally, this is why I’d always prefer a code solution if I was applying the dependency injection pattern, say guice or picocontainer.
Build scripts (the active word there being ‘scripts’) are a strange beast; normally procedural with calls to modules, which, actually, you’d like to be more object oriented if possible. You also would want minimal if/else, foreach support, despite what old school build folks tell you.
For example, I recently wanted to generate a bunch of war files that had very similar web.xml’s but had a few values different. I eventually went with a CSV file, fmpp template merging, and using the for task in antcontrib which called out to a macrodef on ANT.
That was fairly involved, and required more modifications to my basic ANT install than I’d like. I’m also kinda wondering how I’d achieve that with something like MVN… This could’ve been done in 5 lines of code if it was easy enough to stitch together in code.
This made me really think about what do I actually want from a build system.
My Wish List For An Ideal Build System
- Convention over configuration is great if the convention is one that I can understand. I’d like the default to be conventional with overrides. When using ANT I always write coarse grain targets in the build file; make, test, save, and my default ‘ant’ master target has these three as dependencies
- Easily configurable for different environments.
- Dependencies dealt with. Something MVN is good at, but not excellent at. I normally have 2 build scripts; one for the module/project only, and another to also build dependencies, so I can explicitly choose.
Ideally, I’d like to run one build, and it would work out if it had to re-build the dependencies or not. - Not to write code in xml. The best solution I know of for a primarily procedural, with some object oriented / pseudo OO thrown in if needed, is JavaScript (sorry python). Its also has a bonus in that its the most popular coding language in the world. It’s also the language that brought me back to coding when I was in my early 20s (so I’ve got a soft spot for it)
- Re-use all the plugins already made for other build solutions
- Consistent through all the languages I choose to write in
- Concise, and understandable.
This is all very fine waffling about this, but here’s perhaps an example of what this would look like. I’m going to use a recent ANT example, only because I haven’t used a MVN example to hand – last time I used it was about a year ago. A MVN example, after dependencies, would run at about the same amount of lines:
<project name="module" default="module">
<property file="../Commons/environments/${env.COMPUTERNAME}.properties" />
<property file="../Commons/build.properties" />
<property file="build.properties" />
<import file="../Commons/common.xml" />
<target name="module" depends="make.module,test.module,save.module"/>
<target name="make.module" depends="clean.module">
<make module="${module.directory}" jarname="${module.jar}" source="src" targetdir="made">
<make.classpath><classpath><path refid="module.classpath"/></classpath></make.classpath>
</make>
<make module="${module.directory}" jarname="${module.test.jar}" source="test" targetdir="made">
<make.classpath><classpath><pathelement location="${module.made.directory}/${module.jar}"/><path refid="module.classpath"/><pathelement location="${junit.jar}" /><pathelement location="${mockito.jar}" /></classpath></make.classpath>
</make>
</target>
<target name="test.module">
<test jarname="${module.made.directory}/${module.test.jar}" basedir="${module.directory}">
<test.classpath>
<classpath><pathelement location="${module.made.directory}/${module.test.jar}"/><pathelement location="${module.made.directory}/${module.jar}"/><path refid="module.classpath"/><pathelement location="${junit.jar}" /><pathelement location="${mockito.jar}" /></classpath>
</test.classpath>
</test>
</target>
<target name="save.module">
<save targetdir="${module.dist.directory}">
<save.fileset><fileset dir="${module.made.directory}" includes="${module.jar}"/></save.fileset>
</save>
</target>
<target name="clean.module">
<init.dir dir="${module.made.directory}"/>
<init.dir dir="${module.dist.directory}"/>
</target>
<path id="module.classpath">
<pathelement location="${website.dist.directory}/${website.jar}"/>
<pathelement location="${website.model.dist.directory}/${website.model.jar}"/>
<pathelement location="${servlet-api.jar}" />
<pathelement location="${neodatis.jar}" />
</path>
</project>
Now, the proposed JavaScript build solution:
flavour=java
name='module'
load('common_properties.js')
depends=[website, website_model, servlet_api, neodatis]
tests_also_depend_on=[mockito]
Nicer? Obviously, you’d have a register of variables for the library files you’d want to use. And if you’d like it a little more explicit as to what the convention would be…
flavour=java
name='module'
load('common_properties.js')
depends=[website, website_model, servlet_api, neodatis]
tests_also_depend_on=[mockito]
master=[make, test, save]
I’m seriously thinking about starting to write something like this. I think it’ll be way more intuitive than the current Java specific build scripting solutions that are out there.
How would I handle that multiple wars example previously?
flavour=java
name='module'
load('common_properties.js')
depends=[website, website_model, servlet_api, neodatis]
tests_also_depend_on=[mockito]
wars = function() {
var csv = csv('wars.csv')
for (var line in csv.lines) {
var template=merge_template('web.xml.ftl', map(csv.headers, line))
war(line['name'], template)
}
}
after_master=[wars]
If you then had a requirement to use this ‘wars’ function in more than one place, you could put it into an auto-loaded global functions, obviously pulling out the specific values to generic value.
The other bits that aren’t really talked about in the example, and that you don’t need to mention in every build:
- There would be a harness that would run the master array of functions
- Why should I always add my testing libraries to the classpath? I’ll normally always use one lib for this (TestNG or JUnit for Java) and typically never mix and match
- There would also be an included file that would declare all the dependency variables. This would be added to for project/other module dependencies
- The war, map, and csv functions would be globally declared
- There would be one convention file which would allow you to say what your own convention would be for, say, file structure (i.e. source folders, where you want to generate the built code to, where the distribution folder is etc)
How this could be implemented? As a proposal, you could use rhino in order to expose all the stuff written for ANT and MVN, and write a thin wrapper function to ‘de-uglyify’ it. I wouldn’t make the same mistake as some other build scripting solutions and would tidy up / map some of the counter-intuitive calls to MVN/ ANT