What Makes a Good Project Fail?
Object Magazine; September, 1996

Introduction  Project Background  Contractor Selection Project Scoping  Project Approach  Tools and Methodologies Rules To Follow

Introduction

The purpose of this article is to explore the series of events leading up to the failure of an object technology project. Following such failures the sound bite often heard is that "object technology and its tools failed us." It is unlikely that technology is the major factor in most cases. More likely, it is a series of failure points that compound each other to cause the project to fail.

The only way that we in the object technology industry can improve our chances of success is to analyze not only the successful projects, but also the failures. This is the spirit in which the article is written. The client company involved will be referred to as "The Utility". Hindsight being 20/20, some of the observations made here were not so obvious during the project process.

The project specifics can be summarized as follows:

• large customer service and billing applications;
• more than 1.5 million customers;
• more than 3000 users (only about 500 were high volume call center users);
• relational database to be used for data storage;
• object technology (specifically Smalltalk) to be used for GUI and as much business modeling as possible;
• OBA methodology to be used; and

re-engineering of the relevant business processes concurrently with the system development.

Project Background

Having attempted to replace the existing customer information system several years prior, The Utility was convinced that it lacked both the technical and project management skills necessary to be successful with this project. This caused it to prepare a request for proposal (RFP, a basic high-level specification) and distribute it to several large systems integrators. Naturally, the requirements stated in the RFP were not completely stable since re-engineering was about to commence. Internally, The Utility wanted to outsource not only the project execution, but also the responsibility for its success. At the time, there were only a small handful (less than five) of The Utility's personnel with any experience with Smalltalk. Therefore, the staffs' contribution was to be strictly limited to domain knowledge. A strong secondary goal of the project was the creation of a system support team.

Contractor Selection

The Utility received proposals from the large systems integrators. The prices quoted were understandably high considering the size of the system and the uncertainty of the requirements, given the concurrent re-engineering. The estimated project schedules were all 36 months and higher.

The major decision criteria for contractor selection included financial depth in combination with proven experience with object technology.  The Utility's management required this to reduce the risk in the event of non-performance of the contractor. After several months of deliberation, a contractor was selected. We will refer to the company as "Old Reliable", because of its long history of delivering this type of system in the industry.

Old Reliable’s price was very high, but it was financially strong and had experience with Smalltalk. Its estimated time to completion was 36 months. It also had a reputation for complete client control on its projects. Some of The Utility's management reasoned that this would lead to unnatural suppression of changes streaming forth from the re-engineering team. After awarding an unrelated contract to Old Reliable to assist with the re-engineering team, client management decided that it could not work with Old Reliable for 36 months on the customer information system. Nor did it help that Old Reliable’s attorneys had bloated the proposed contract up to more than several hundred pages!

While the decision to terminate Old Reliable did not cause a problem, it did set the stage for a major point of failure. With Old Reliable disqualified, The Utility was left only with the also-rans from the previous bid. In an effort to expand the number of bidders, The Utility sent the RFP out to several new companies. It would be fair to say that these companies were not generally thought of in terms of delivering systems of this size with object technology. To give these companies a fair chance, the requirement for object technology experience was dropped for the prime contractor. This was the first failure point for the project.

During the deliberations, it became apparent that the experience with Old Reliable had affected the state of mind of some of the client staff. Specifically, the second place finisher in the previous bidding process (who also had financial strength and experience with Smalltalk systems) was derided based on some client staffs' negative experience with them. The fact that The Utility had no experience at all with the ultimate winner (refereed to here as "RDB Consulting") did not enter into the equation. It was decided that in order to strengthen RDB Consulting’s bid, they would be required to take on a subcontractor who would provide the object technology and Smalltalk experience (referred to here as "OT Inc.").

So the hands were shaken and the teaming agreements negotiated between RDB Consulting and OT Inc.. The nature of the agreement was also to be a major factor in the future of the project. RDB Consulting, having the financial strength, retained responsibility for the delivery of the system. OT Inc. was a time and material subcontractor to RDB Consulting, having no direct responsibility for the project in the contract with The Utility. This was the second failure point for the project. The general attitude of RDB Consulting’s project managers was to ignore the advice given to them by OT Inc.’s staff, even though RDB Consulting’s staff had no object technology experience. This advice generally concerned project approach and appropriate use of object technology. Since OT Inc. had no direct contractual relationship with The Utility, RDB Consulting was able to prevent OT Inc.’s staff from communicating their concerns to The Utility.

Project Scoping

The project schedule estimates were prepared exclusively by RDB Consulting’s staff using traditional estimating techniques. They allowed The Utility to negotiate the overall project schedule from the original length to 24 months. This was done and committed contractually prior to the inclusion of project scoping/estimates by object technology staff of OT Inc. RDB Consulting’s staff assumed that the traditional estimates would be conservative since object technology was supposed to be "so much more productive". These estimates assumed the use of a 4GL and forms based application development tools. The design of an object oriented business model that is extensible for the future often requires more time than "traditional" application design using these types of tools. The superior productivity of Smalltalk offsets this somewhat but not completely. The reward for additional time spent is a system that is much more flexible. This reduction in the length of the project timeline, without a corresponding reduction in the deliverables, reduced the margin of safety in the project plan and was the third failure point for the project.

Owing largely to the failed prior attempt to replace the customer information system, The Utility's management was feeling immense pressure from the customer service staff and the company executives to deliver a system. It correctly recognized that many of the existing business processes could not be allowed to be used as requirements for the new system. This led to the fourth failure point for the project: re-engineering of the business processes concurrently with system development. The re-engineering team could not stabilize the requirements in time for the system developers to use them.

Project Approach

The 24 month schedule led RDB Consulting’s project managers to estimate more than 50 Smalltalk developers for the project, along with numerous other support staff plus a dozen of The Utility's staff. This was based on the infamous assumption that "nine women can have a baby in one month." In order to utilize the large staff, a massively parallel project process was to be used with seven teams of developers working concurrently. This was intended to mimic the approach used in symmetrical multiprocessing systems, where performance can be increased simply by adding processors.

The performance increase gained from a processor addition in SMP systems depends heavily on partitioning the workload into independent tasks. In a software project, you must factor in significant inter-team communications overhead to allow for the creation and negotiation of contracts between the teams’ subsystems. This reduces the independence of each team's task.

Additionally, insufficient time was spent on architecture and high level analysis/design prior to the multi-team split. There was an independent architecture/frameworks team, but they were given very little head start on the remainder of the project. This re-enforces the rule of thumb on object technology projects: start small and build up, rather than starting large and burning down. This large staff size coupled with the parallel development process was the fifth failure point for the project.

With more than 50 Smalltalk programmers to hire in such a short amount of time, RDB Consulting’s project managers had a daunting task before them. They could take half a dozen from one supplier here, three from another supplier there; or they could find a few large suppliers that could provide the required numbers, but with less well known experience levels. The project accounting nightmare of paying a dozen or more contract firms throughout the project led RDB Consulting’s project managers to select the latter choice.

Little, if any Smalltalk experience was required of incoming developers. Smalltalk and other tool training was scheduled for nearly everyone included in the project. What was not adequately taken into account was the unproductive time following training where the developers were getting comfortable with the new tools. Additionally, this would be the first "real OO" project for many of the developers, so additional time for the paradigm shift was also required. Compounding these problems was the fact that many of the developers had difficulty speaking English clearly. OT Inc. provided half a dozen experienced Smalltalk developers, but they were spread very thin between establishing project development standards, assisting in training sessions, mentoring other staff, and developing system architecture and frameworks. The ratio of experienced to inexperienced Smalltalk developers was around 1:10; clearly unacceptable for a productive project. This was the sixth failure point for the project.

From the beginning of the project, it was clear that RDB Consulting intended to run things using a traditional waterfall approach. There were no project managers assigned who had any experience with object technology. Internally, a rigid, hierarchical management approach was instituted within the project while communications to The Utility oozed an image of teamwork and consensus. Experienced object technology personnel from OT Inc. were unable to change the project direction in any meaningful way. Adjectives like "insubordinate" began to flow forth from the mouths of RDB Consulting’s project managers when describing OT Inc.’s staff. This situation was exacerbated by the tremendous size of the project staff...all patiently waiting for a work assignment. This was the seventh failure point for the project.

Tools and Methodologies

The OBA methodology, and especially its prototype toolset, proved to be insufficient for such a large number of concurrent developers. After all, it's not your ordinary object technology project that has 50 simultaneous users for the analysis tool!  Due to RDB Consulting’s inexperience with object technology, the selection process for a new methodology and tool was delegated to a subgroup of the project staff (more than 20 people). The result was a large discontinuity in the project that could have been avoided if a decision was made quickly and new tooling acquired. Once again, the sheer size of the project staff transformed this situation from a mole-hill to a mountain.

The project was starting just as a new release of the Smalltalk to be used was announced. The new release contained a number of enhancements, not the least of which was an improved interface to relational databases. An over-emphasis on the database connectivity -- and an under-emphasis on the object oriented business model -- led RDB Consulting to try to use several beta versions of the new release. Precious time was wasted trying to get the beta releases operational (24 month total project schedule) with the database. If RDB Consulting had more experience with object technology and Smalltalk in particular, it would have realized that major parts of the system could have been developed and delivered using the previous Smalltalk release.

The reliance of the project on prototype or beta level tools was the eighth failure point for the project. In the case of OBA, RDB Consulting did not have enough experience to refuse The Utility's requirement to use OBA and its prototype tool, given the size of the project staff. For Smalltalk, a combination of RDB Consulting’s lack of experience and OT Inc.’s extremely tight coupling with the Smalltalk in question led to the decision to use the beta level releases.

After the project had been executing for over a year, it was evident to The Utility that progress was not being made by RDB Consulting. Efforts to correct the problems eventually failed and the project was canceled. The official reason given for cancellation was, predictably, the failure of object technology. Based on this conclusion, The Utility abandoned its strategic efforts to utilize objects and is in the process of fixing and extending the old customer information system.

This postmortem examination of the entire project lifespan has shown that there were many failure points that contributed to its ultimate demise. By following the suggestions listed below, you can help to guarantee that the same fate does not await your next project.

 

Rules To Follow A number of decisions made during the execution of this failed object technology project contributed significantly to the failure. These decisions were made by both The Utility and RDB Consulting. As stated in the introduction, the purpose of identifying these important decisions is not to assign blame but to allow others to learn from this experience. Some rules of thumb that you can use to avoid the pitfalls discussed are shown in bold type.  Object technology experience was not required of the prime contractor. Ensure that any prime contractor with delivery responsibility for your project has relevant experience with the tools to be used. The subcontractor did not have contractual responsibilities with The Utility. Ensure that third parties involved in your project have a delivery responsibility in the contract. Avoid three tier arrangements that prevent the third party from exposing project problems to you. RDB Consulting allowed The Utility to reduce the project schedule to 24 months without a significant reduction in deliverables. When working with a consulting company with object technology experience, do not try to negotiate a reduction in the estimated project schedule timeline without reducing deliverables. If the consulting company allows you to do this, BEWARE. The Utility planned to carry out re-engineering concurrently with system development. Do not try to re-engineer business processes and develop a system for the same business area concurrently. If a consulting company accepts a project where this is the situation without a significant schedule increase, BEWARE. RDB Consulting estimated a very large staff size and therefore required a massively parallel project process designed to utilize them. If a project plan is proposed to you with more than 10 Smalltalk developers on board at the beginning of the project, BEWARE. Remember: start small and build up, rather than starting large and burning down. The ratio of experienced Smalltalk staff to inexperienced staff was too low. Ensure that you get the experience level for which you are paying. If you are paying premium prices, insist on developers with multiple years of experience with the tools to be used. If some inexperienced staff is acceptable to you, ensure that the ratio of experienced to inexperienced staff does not fall below 1:4. RDB Consulting utilized a rigid, hierarchical management style, preventing experienced object technologists on the project from affecting the project course.. Ensure that contracts are written such that the most experienced object technology staff are responsible for important project decisions. Do not allow individuals without object technology experience to control the ultimate destiny of the project. The project relied too heavily on prototype/beta tools. Do not bet the project on any tools that are not generally released. If the consulting company that you are working with suggests or agrees to use one or more prototype/beta tool releases, BEWARE.

Introduction  Project Background  Contractor Selection Project Scoping  Project Approach  Tools and Methodologies Rules To Follow

Send mail to webmaster@dsbconsulting.com with questions or comments about this web site.
Copyright © 1997-2003 DSB Consulting Inc.