Posts Tagged ‘development’

The Downside of Modularity

Saturday, March 13th, 2010

I’m a big fan of modularity. I’ve even got a modular system in my living room. It consists of the following modules:

  • One module that converts a digital signal to a two-dimensional picture.
  • One module that reads a rotating plastic disk with a laser and produces a digital signal.
  • One module that gets a digital signal from a socket in the wall, stores it temporarily on magnetic disk, and sends it out again upon request.
  • One module that generates a digital signal based on a simulation of a virtual world, with which I can interact in real time using motion and pressure sensitive input devices.

In case you hadn’t guessed already, I was talking about my TV, DVD player, Hard-disk recorder, and Game Console.

Imagine if all of this came in one device. A TV+DVD+HDR+Console-in-one. Imagine what it would cost. If only one part broke, I would have to get everything anew. I would never be able to move it abroad, because the HDR is tied to my cable provider. I would never be able to get the games that do not involve Italian plumbers.

But to be fair, there are also disadvantages to modular systems. Just take a look at the remote control that comes with it:

How to develop Modular Software

Saturday, March 6th, 2010

It’s always good to make software modular. Modular software is strong and healthy, monolithic software is sickly and bedridden. I’ve touched before on how modularity increases adaptability. But modularity also helps to keep software small, nimble and unbloated. I’ll illustrate how we’re applying modular design in BridgeDb.

Modularity is the only known antidote against bloatware. The more features a piece of software has, the larger it has to be. When you don’t use 90% of those features, it’s perceived as a problem. Bloated software takes a long time to start, fills up your hard drive, clogs your tubes. We want bioinformatics developers to use BridgeDb as much as possible, and we don’t want them to complain that BridgeDb is bloated.

For example, BridgeDb supports identifier mapping through several different web services. Some of those webservices are based on SOAP, others on XML-RPC or on REST. For each type of webservice, you need additional libraries. If it was only one monolithic chunk, you’d always need several megabytes of library dependencies.

You may say: “A few megabyte, so what?”. When I was at mediamarkt the other day, I couldn’t even find memory sticks smaller than 2 Gb anymore. But size still matters when you expect fast download times. For example WikiPathways uses BridgeDb on each pathway page. Bigger libraries means longer load times, which means annoyed users.

We want many features, but we don’t want bloat. The solution is to cut BridgeDb up into many small pieces, where you can choose the ones you need, and ignore the rest. You also don’t need the dependencies of the parts you ignore.

So how do you decide which pieces of BridgeDb you need? I’ve compiled this handy graph. On the right side, you see all the different “features” (i.e. identifier mapping services) that you can choose. Follow the arrows to the left, and note the modules that you encounter. Those are the modules you need for that mapping service.

If you’re getting started with modular software development, I can give you a few tips. You really don’t need any of those terribly complicated frameworks like Maven or OSGi. All you need is a good IDE like Eclipse and a bit of determination.

You have to be careful to manage the boundaries between modules. Eclipse can help you a great deal with this. Put each module in its own directory. In your Eclipse workspace, set up a separate project for each module, and add dependent projects in each project build path. This way you can never introduce cyclic dependencies or go across module boundaries. Eclipse will simply refuse to find the class and flag it as a compiler error.

For example, here is how I’ve set up BridgeDb in eclipse. In the package explorer you see that I’ve defined a separate project for each module in BridgeDb.

And to complete the example, here is how I configured the build path for the module. As you can see, the org.bridgedb project is listed as its sole dependency.

Martijn’s Continuous Build System part 1

Thursday, June 25th, 2009

Joel said it best – Daily Builds are your friend.

A continuous build system is quality control for a program that’s being developed. It’s a computer that tests the state of the program every day. Or preferably every ten minutes. Under my desk I have a computer whose sole job it is to continuously monitor the state of PathVisio. Does it compile correctly? Are there any style problems in the source code? Are the automated tests positive? Fresh documentation is automatically generated and uploaded. The webstart version is refreshed. A fresh zip archive is created and placed in a convenient place. If there are any problems, an email is sent to our mailing list. There is an online report where you can find out everything about the current health of PathVisio.

bigstockphoto_fire_alarm_1536674The real effect of a preventive measure is always hard to tell. It’s like fire prevention measures. They are costly, yet most houses won’t need them because they’ll never burn to the ground. Setting up a good continuous build system is quite a bit of work. Is it just red tape, a lot of effort but YAGNI?

Here is an example of how it saved us. After I hooked up Helen’s summer-of-code project to the continuous build system, we soon got a ton of errors by email. It turns out that she started using GroupLayout, a java class that is only available in Java 6 and higher. However, unbeknownst to Helen, both Cytoscape and PathVisio aim to be compatible with Java 5, and I configured the build system to check for Java 5 compatibility. So we found out about this problem right away, and Helen could fix the problem immediately before going too far down a dead-end road. It would probably have taken a lot more work to fix this if we had found out later.

For PathVisio we’ve used a continuous build system since day one. But recently I’ve taken the time to make quite a few improvements. I plan to write about that in more detail in the coming days.

Edit:here is part 2