Agile – the way forward for projects?

One of the biggest flaws with the traditional “waterfall” model of software development is that the customer requirements are only validated at the end.

Of course this reveals two types of flaw in the documented requirements:

  1. They were simply wrong/inadequate in the first place
  2. They have been overtaken by events

This is particularly obvious in long defence projects like the F35 replacement for the Harrier, or public vanity projects like the Millennium Dome and Scottish Parliament Building.

Much of the cost-overrun of big projects results from the delays and rework costs of enhancing the finished product to meet the newly-clarified/revealed requirements.

So – if you get the requirements right and build to meet them right first time, the project will stand out as being cheaper than normal. The Rion-Antirrion Bridge in Greece is a fabulous example of getting it right first time, and Cross-rail is looking good too.

The huge strength of the Agile approach is that it validates requirements as it goes along, by integrating end users into the development team, and for software development projects this is great (with a couple of caveats). This means that Agile projects have a tendency to be more successful and better value for money.

So is it a no-brainer? Should we do all projects as Agile?

Agile isn’t necessarily cheaper – the Rion-Antirrion Bridge wasn’t Agile, and there are Agile projects that have gone badly wrong.

In my experience, the Achilles heel of Agile is that if the fundamental requirements are not adequately understood at the start, the wrong “architecture” of the solution can be selected and only when a lot of time and effort has been invested does it become clear that you’re on the wrong horse. Swapping horses in mid-stream is difficult and risky, and usually hugely expensive, so the pressure is on to struggle on and make do with what you’ve got.

I prefer a multi-stage approach with rigorous feasibility assessments in managing major strategic projects, where the architecture is really sorted out first. The Rion bridge needed to be tackled as a single entity, because that is the way it functions. Picking the wrong ERP system crippled the business and IT strategy of a FTSE 100 company, leading to a complete U-turn in its IT plans.

Agile approaches have some real benefits, but need using with care, and must not be used as an alternative to thinking the problem through first.

Rule 10 – Never rush the planning!

Fail to plan – plan to fail!

A bit of a truism, I know, but valuable none the less. I don’t mean “take forever over the planning”, just don’t skimp it.

There is usually pressure to do things as quickly as possible, and to many executives, this means “start work now!”. This creates a dynamic tension between the need to report progress and the need to work out how to achieve the right progress.

Everyday English is full of sayings offering advice on this:

  • Look before you leap
  • More haste, less speed
  • Measure twice, cut once
  • A stitch in time saves 9
  • Nine women can’t make a baby in one month (Brooks)

and so on.

So why, in business, is this ignored? I wish I knew! I can only attribute it to said executives being out of touch with the reality of the situation at the coal face – what I call the “Excel Effect”.

As part of my teaching, I read many assignments about projects that have failed because the basic thinking-through of the problem was skipped at the beginning, and fundamental (and easily avoidable) errors were made that cost millions to fix. The build of Dungeness B nuclear power station was started well before the design was complete – unlike the other AGRs, it was 17 years in construction (10 years late) and way over budget. In a similar but smaller-scale project, a building was erected that proved too small to contain the plant intended to go inside!

How do I deal with this?

  • Insist on the irreducible minimum of planning time at the beginning
  • Get the design substantially complete before committing to build, then building the right thing quickly
  • Identify long-lead critical path items that we must start – this shows progress –  but continue planning in parallel
  • Log all planning assumptions as risks, and manage the hell out of them!

I’ve found this works well (unless the sponsor is unwilling to listen, and has committed to impossible deadlines!)

Case studies are NHS Direct website and THUS billing migration

When isn’t Agile the answer?

I’m strongly against the pure waterfall models of projects, especially for systems development, as the last thing to happen is requirements validation!  I also very much like the close liaison between end users and developer that Rapid Applications development (using methodologies like DSDM and Agile)  builds – such relationships really ensure that requirements are validated asap.

Incremental design is a real risk, though, for major architectural projects such as merging a number of legacy products into a common framework.

I’m currently part of the INCOSE/APM Joint Working Group on integrating Systems Engineering and project management. The reason this energizes me (hence me leading the communications and exploitation workstream) is that systems thinking and looking at the big picture has been a core element of my project management approach for as long as I can remember. I’m convinced that looking at the big picture in the beginning really helps avoid going down “rabbit holes” in solution design, avoiding all the cost and delay of redesign and rework.

The overall design of major engineering projects must be done up front – incremental delivery thereafter is fine.