JavaOne. Open Source tools for optimizing your development process

Well, I made it to JavaOne this year, and here is the first session from the Community Day, delivered by John Smart.

This session was really about how you can be more focussed in software development by utilising tools and process to take out some of the pain.

Main focus of which was:

• Reducing bugs
• Using build scripts
• Managing dependencies
• Automating builds
• Checking code quality
• Introducing tighter integration with issue tracking
• Automating the generation of some technical documents

Build Scripts: The aim is to have something reproducible, that works anywhere (not just on my pc!) and is portable. Once you have this basis, then you can look at automation through Ant or Maven 2.

The two main choices (though there a few hybrid ones now based on some of the dynamic scripting languages) are Ant and Maven 2. Both have advantages and disadvantage, which is why Apache maintain two!

Ant is widely used and extensible, but does need a lot of low level scripting, and doesn’t have a standard lifecycle (there are conventions, but they are not consistent).

Maven is a declarative build scripting framework that prefers convention over configuration. By following the conventions yourself, much of the ‘lifting’ becomes simpler.

Both support lots of plugins/tasks for deployment, reporting and quality measurement, but because Maven has a standard lifecycle, unlike the adhoc nature of tasks in Ant, it is much easier to understand what is going on in the build process without having to wade through the build source scripts.

• Standard directory structure
• Convention over configuration
• Better dependency management
• Good support for multiple concurrent projects and the dependencies between them

Although Maven imposes a standard directory structure, which yes, isn’t as flexible, but it does mean as you go from project to project you don’t have to work out each time what to put where.

Managing Dependencies: Both internal and external dependencies, e.g. libraries like Commons logging. Traditionally, each project would have it’s own JAR files, but it is hard to keep a track of versions of common JARs you are using (and the ones that are in turn used by things you depend on! Which may well differ). So to get around this we can use Ivy for ANT, or and ANT task for Maven. It’s also worth maintaining a local repository of libraries you depend on, versioned in some sensible (even if it’s renaming them with the providers versioning) way so you know what versions they are.

Good testing practices: Remember that Unit tests document your intentions! they also give comfort – not just to you. More readable cases are easier to maintain and regression test.

The two main alternatives are TestNG and JUnit

TestNG is certainly increasing in popularity, and it looks like JUnit 4.4 is making testing much easier over JUnit 3, with:

• Annotation based testing
• Ability to test for timeouts
• Easier Exception Handling
• Parametrized tests

It was certainly a pain before that everything had to be derived from TestCase. Now any class can be used via annotation @Test. Before there was setup and tear-down, now there are @before, @after, @beforeclass and @afterclass. BTW – JUnit 4 still runs JUnit 3 tests.

Now that it is any plain Java class, and the fact that methods can have any name, developers are encouraged to use behaviour driven testing, for example naming the test method ’shouldUpdateEmployee’.

The parametrized testing is a significant addition. Basically, it takes a set of arbitrary data and test cases in a single bundle. You just need a test class with fields that match the grid of your test data, and some data as rows – it then executes once per row, which is a great improvement – you can now test more data with less!

They’ve also added Hamcrest asserts (see the tutorial here) a much more readable way of using assertions. So instead of the rather weird looking assertEquals(10,x) you can have something like this:

assertThat(theBiscuit, is(equalTo(myBiscuit)));
or
assertThat(“chocolate chips”, theBiscuit.getChocolateChipCount(), greaterThan(10));

It also generates much more friendly error messages e.g. ‘expected red, got blue’.

Test Coverage: The aim of which is really to help you write better unit tests! Obviously, there’s no guarantees that what you have is well written, but points you at areas of your app that are poorly covered.

Add it into you build process to give early feedback and visibility to the team (see Cobertura which is great for project level coverage reporting), or build it into your IDE (tools like EclEmma).

Another useful tool I’d not seen before was Crap4J – boy do they need to work on the name! It gives some good additional statistical data like the number of times it was actually executed too. Sounds like a good addition to the toolkit of any build-meister.

One good point that comes across again and again with many of these tools – not just for test coverage is Why use one? Use both! Chances are they’ll pick up different things.
Continuous Integration: Basically, automatic integration testing and compilation on a central server. Obviously better with some sound testing, and good coverage so I’d sort that first!

You need an automated build process (which you should have already with ANT, maven or similar), automated test processes and a source repository.

Common tools – Continuum, Hudson or Bamboo. I’m going to a session on Friday by Kohsuke Kawaguchi, Hudson’s creator, which should be cool – more later.

So what problems do they solve?

• Infrequent Integration and testing
• Infrequent commits – because people are scared of breaking things, which is ironic really
• Fewer release, which results in less user testing
• Reduces the number of bugs over the lifetime

So, no excuses.

Other good stuff you can leverage from your builds – take a look at the build history. What you want to see is an increasing number of tests, and some indication of build stability e.g. 5 out of the last 6 builds were all successful. Go one step further – auto update your wiki to show it, it gets people more involved in the build process, and fixing it if there are problems, but it also shows when your heading in the right direction. See testing doesn’t always have to be negative

User perception: What else could you do in your build process? How about automatically deploying a successful build occasionally, so that users have got something to see, and can see some progress. Try and show them some way what’s new, so they can see the fruits of all your hard work. Continuous Integration and Continuous feedback (hopefully), not bad.

Code Quality: Increasing code quality give you:

• a better product
• a more maintainable product
• a descriptive statement of intents
• enforcement of coding standards
• an opportunity for training and development of new team members. Plus possibly those who by now need a refresher course!

Usual suspects CheckStyle, PMD, FIndBugs and Crap4J.

CheckStyle gives good pointers on coding style and formatting. FindBugs is good at tracking down potential bugs i.e. those that may bite you in the future. PMD, a bit like CheckStyle, but more technical. Crap4J also gives good indicators of poorly tested classes and overly complex ones.

Automated Documentation: What??? I hear you cry. No, not fully automated, but how about some basics that you should be doing that you can at least part generate automatically, and will always be up to date? Typically wjhat happens is some of the basics are written once and then forgotten as everyone moves on to the next important thing. They are still basics though.

Obviously they will be a little dry, but two good ones to try : UMLGraph and SchemaSpy

UMLGraph will be clickable and navigable class diagrams right into your Javadoc, which is a great start.

SchemaSpy is similar, but for Databases. Given an arbitrary schema will generate a nice ER model for you. This can often be really useful, especially if someone has given you a large legacy DB, and no documentation, just reverse engineer it.

As ever with useful documentation like Javadoc – generated or not, note that you may need to use thumbscrews to get people to actually read them!

Final tips:

Fail early and often in the build process, and give feedback ASAP. Failure at any stage (unit, integration or quality) halts the build. Tests getting too big? work out which ones give the best indicators and run them always. Then move longer running indicators to a less frequent schedule e.g. at night.