Why big IT projects crash

FT.com

September 18, 2013 8:13pm

By Henry Mance

There are several ways the US Air Force could have wasted $1.1bn. It could have poured tomato ketchup into 250m gallons of jet fuel or bought a sizeable stake in Bear Stearns.

Instead it upgraded its IT systems. Work began in 2007 to reconfigure how the force managed its logistics, with the aim of replacing 200 dated networks with a single piece of Oracle software. By the time the project was abandoned last November, it was at least four years behind schedule and would have required an additional $1.1bn to become usable.

Yet in making such mistakes, the Air Force is not flying solo.

This week a UK parliamentary watchdog described a failed National Health Service patient IT programme – the cost of which has spiralled to £9.8bn – as “one of the worst and most expensive contracting fiascos in the history of the public sector”. Earlier this month the Department for Work and Pensions admitted that it had written off £34m of IT costs, incurred in an attempt to overhaul how social security benefits are paid. A week earlier Co-operative Bank said it had written off the £148m cost of a new IT system that would no longer be implemented.

“It is quite scary,” says Ralf Dreischmeier, the global head of Boston Consulting Group’s IT practice. “From my experience, 20 per cent of projects fail, and 40-50 per cent have a cost overrun, time overrun or don’t meet requirements. Only a third could be described as good projects.”

Why are companies and governments still suffering such embarrassing failures?

In a 2011 study, Bent Flyvbjerg and Alexander Budzier at Oxford university’s Saïd Business School examined 1,471 IT projects against their forecast costs and overruns. They found that the projects exceeded their budgets by an average of one-quarter.

“Over the past decade there have been no improvements even though a lot of things have been tried,” says Mr Budzier. The researchers posited that planners consistently underestimated the costs and overestimated the benefits of IT projects. They also failed to appreciate the “black swan” scenario – that is, the chance that something will go really wrong.

Citing Nicholas Nassim Taleb, author of the book The Black Swan, the researchers argue that “the high over-incidence of black swans underlines that ICT projects are a very important source of uncertainty in an organisation”. In one in six projects examined by Prof Flyvbjerg and Mr Budzier, the cost was at least triple what had been estimated. When Hershey’s, the chocolate maker, implemented a new ordering system, it ended up missing out on a whole Halloween of sweet sales, worth $100m.

What have we learnt from past mistakes?

Make realistic estimates
“People always underestimate the cost of software development. Suppliers always push their prices,” says John Fotheringham, a partner at Deloitte.

Keep it short
Longer projects tend to see a higher turnover of personnel and a greater likelihood of a change in objectives. “A reasonable recommendation would be to try to complete a project within 18 months,” says Alexander Budzier of Oxford university’s Saïd Business School. The state of South Australia recently proposed only commissioning IT projects lasting less than 90 days. Yet Mr Budzier adds that there is no statistical correlation between the value of an IT project and its success.

Everyone loves ‘agile’ 
Up to four-fifths of new IT projects are now implemented using agile methods, whereby pieces of the project are implemented quickly then improved if necessary. This can, however, require more active management.

Your project is not different
“If the project manager thinks this is a unique project, then it’s going to explode,” says Mr Budzier. “And there’s a clear reason – they don’t benchmark themselves.”

Although much criticism has been directed at civil servants, IT overruns are present in both the private and public sector.

“The private sector is just much better at hiding these things,” says Mr Budzier. He points out that large blue-chip companies continue to operate failing IT systems because they are unwilling to write down the expense.

Drawing on the work of Daniel Kahneman, the Nobel Prize-winning behavioural economist, Prof Flyvbjerg and Mr Budzier call for companies to forecast their costs better by comparing their projects to previous experience.

The problems do not end with poor forecasting. Organisations are often very ambitious in what they set out to achieve – designing long projects that require customised software. That is likely to be another mistake.

In a report published this year, the Texas state auditor’s office examined 13 IT projects, nine of which had overrun. It concluded, admittedly on a small sample, that agencies using commercial off-the-shelf technology “exceeded their budgets by a smaller amount and took less time to complete their projects” than those that did not.

There is also an unwillingness to admit to failings when they occur. In the US Air Force case, the can-do attitude of some military officials meant concerns were not raised. “Program managers fear that an honest delivery of program status will result in cancellation,” noted an independent progress report into the project.

Poor communication – particularly between business and technical experts – is a constant problem. “Corporations are organised in hierarchies, in line roles,” says Mr Dreischmeier. “Doing projects is different. Projects go across silos.”

Boston Consulting Group argues that a project must have clear objectives, an understood business case, good governance, clear deliverables and an agreement planning structure.

In recent years companies have laid the blame for IT failures at the door of their contractors. The IT industry has sought to borrow methods from the construction industry, where suppliers incur penalties for late delivery. However, IT has special challenges.

“You’ve got lots of potential excuses for things being not your problem,” says John Fotheringham, a partner at Deloitte, the professional services firm.  Suppliers can blame difficulties on integration with existing IT systems. “They are very good at extracting extra money,” he says. Clients can heighten this risk by changing a project’s specifications after it has started.

Some suggest that more time should be spent on the design of IT projects. “A transport project is often planned for five years,” says Mr Budzier. “With IT there is very little upfront planning.”

Yet getting that right can also be problematic. Unlike road technology, software changes very fast.

Boston Consulting Group’s Mr Dreischmeier also warns that too much time can be spent on design. “I’ve seen a project implementing a global finance system, and you’re just doing design work for six months,” he says. “Then, when you put it together, it doesn’t work.”

An emphasis is now being placed on “agile” methods, whereby work is broken up into small chunks and the design is not considered complete before the work is started.

Those methods, which were used in the Department for Work and Pensions’ benefits overhaul, have the advantage of trial and error. Yet they also can require greater oversight.

Nonetheless, not everyone believes that such soul-searching is helpful. “If reports keep focusing on failures, we risk IT being seen as purely a cost and something that needs to be cut,” says Georgina O’Toole, an analyst at TechMarketView.

The focus on budget and cost overruns may obscure the wider benefits of a project. “You need a way to evaluate whether your overall objectives have been met,” says Larry Maccherone of Rally Software.

Some observers suggest that cloud computing, which requires less on-site hardware, will simplify future projects, rendering failure less common. Executives are also more technologically literate now, and increasingly aware of the history of failure. “There are certain changes that make me fairly optimistic that some things will be better,” says Mr Dreischmeier.

For the moment, however, that is hope over experience.