The issue of quality is becoming a centerpiece in discussions surrounding the software development industry. There are good reasons for it, too. Software development in the U.S. has blossomed into a $250 billion industry, and many experts expect that figure to double in the next 10 years. As the digital age continues to embrace the nation (and the globe), there is an ever-greater reliance on newer, better, and bigger software systems. The more we integrate software operations into the fabric of daily life—business and personal—the greater the need for a dependable, reliable product from the software industry. But, stretched at the seams for resources and breathing room, the IT industry has been taking lumps on the head for its reputation when it comes to its ability to deliver consistent quality.
We’ve all heard the statistics. A recent study by the Standish Group (to rub it in they named it the CHAOS study) looked at quality in the IT industry. Among the 175,000 or so projects undertaken, here is what they found: 31 percent were canceled before completion, at a loss of more than $81 billion. Fifty-three percent of all projects came in at an average of 189 percent over budget and about 222 percent over schedule. And of all the projects surveyed, only 16 percent were completed on time and on budget (Source: www.standishgroup.com).
This and other studies like it have helped focus a beam of scrutiny on software development, especially in corporate America where the bottom line is continually evaluated. In response, many IT organizations have begun to adopt quality programs with the hopes of turning these performance trends around. Many such programs have been adopted. The best known is probably the ISO 9000 series, but there is also Deming, Baldridge, and others. However, the one program that has been growing steadily in its acceptance within the IT community is the Software Engineering Institute’s Capability Maturity Model (CMM).
The Capability Maturity Model
A lot of the popularity of CMM has to do with the fact that it was specially designed for IT organizations. ISO, Deming and the others are general quality improvement programs. They can apply to most any industry. CMM is also (in my opinion at least) easier to implement. For all its robustness, it is a somewhat smaller model than the others, and so is easier to manage.
What CMM is, is a process improvement framework. It is built on the assumption that the more “mature” a software development shop is, the higher the quality of its end products. The term “maturity” refers to the effectiveness of the shop’s development processes. If you don’t use any processes at all you are considered immature. If you are process-refined you are considered mature. Mature organizations, according to CMM, work in environments where predictability is high and risk is low. Immature organizations work in environments where risk is high and predictability is low.
The Five Levels of CMM
CMM rates an organization’s maturity on a five-tier level:
- Level 1 – Initial
- Level 2 – Repeatable
- Level 3 – Defined
- Level 4 – Managed
- Level 5 – Optimizing
- Level 1 organizations are termed “Initial” by CMM. These organizations (at the low end of the scale) are not process-oriented shops. Their successes or failures are usually dependent upon the talents of their staff, good fortune, or luck. Because they work in an ad-hoc manner they can’t take what worked well on one project and repeat it on the next. They don’t learn from their environment. As a result it’s hard for a Level 1 shop to improve. It’s a place for heroes and headaches. Amazingly, it is estimated that about 70 percent of IT shops worldwide would assess at Level 1.
- Level 2 is the first real entry point into CMM. This Level is termed “Repeatable.” When you reach Level 2 you are managing projects using repeatable processes and you have in place the infrastructure needed to support a process-driven environment. At this point you begin to measure the success of your processes so that you can begin to refine them, discarding what doesn’t work and repeating what does.
- At Level 3, the organization is able to take its processes and, now that they have had the time to become proven, institutionalize them across the organization. This level is termed “Defined.” Here the entire organization becomes integrated into the process improvement process. The focus is not just on projects but also on the full spectrum of development activities in an IT shop.
- Level 4 organizations are in the “Managed” stage, a high level of process maturity. Here quantitative measures become an important tool in assessing areas of improvement, not just in view of development processes but in views of overall product quality, too.
- Level 5 – “Optimizing” is the highest level of maturity. Here the entire organization is focused on continuous process improvement.
Activities in Each Level
At each level on the CMM scale you will have to support certain policies and perform certain activities in order to be compliant with what the model expects. To help you with this the CMM breaks down each level into a series of Key Process Areas (KPA). These are broad areas of focus, areas you must develop in order to foster quality. For example, at Level 2 you need to develop processes for Requirements Management, Project Planning, Project Tracking, Configuration Management, and Quality Assurance.
Under each of these KPAs you will need to adopt certain key practices that are presented as general guidelines that describe (among other things) the goals, activities, and abilities you will need in place to make each KPA effective.
Most of these guidelines are intentionally expressed in high level terms, giving you the latitude to implement them in a way that best suits the structure and culture of your IT shop.
The Keys to Success
CMM is not a magic wand you wave over your projects and presto, everything’s better. Nor is it a cookie cutter checklist that you can run off blindly. For your CMM program to succeed you must have both knowledge and commitment. By this I mean knowledge of the structure and spirit of CMM (see sidebar), and a sincere commitment to integrate it into your way of doing business.
For the knowledge part, it often helps to participate in some form of SEI-endorsed CMM training or bring in a consultant, especially at the start of your CMM efforts. But the more important quality is commitment. You must stick with your CMM program over time; you can’t shelve it whenever things get hot.
The best way to manage your CMM program, especially if you’re moving from Level 1 to Level 2, is to start small, evaluate what works in this arena, improve on that, and then expand. It takes time but you’ll find that this approach “sticks” better in your environment than a hasty, all-at-once approach.
Benefits of CMM
Like any conscientiously implemented process improvement program, the benefits of operating under CMM come with time. But a dedicated commitment will show results. The “soft” benefits you can expect include reduced re-work times, fewer faults, enhanced predictability, increased customer satisfaction, and an enhanced performance reputation. There have also been studies that point out the “hard” benefits. One by Samuel Keene looked at the number of design faults in shops of varying CMM levels. He found that shops that would assess at Level 1 had on average twice as many faults as those that would assess at Level 3 (Keene, Samuel. Modeling Software Characteristics, 1999). Another study by Karl Williams looked at defect density. The results were that Level 3 shops tended to have less than half as many defects as Level 1 shops (Williams, Karl. “The Value of Software Improvement,” Spire97, June 4, 1997). And an SEI-sponsored study showed that the ability to predict development schedules accurately rose from 40 percent for Level 1 shops to 80 percent for Level 3 shops. And the ability to predict budgets costs rose from 41 percent to about 73 percent (The Sea’s Process Maturity Profile Update, 2000, SEMA 3.0).
In the long run these benefits provide you with greater control and planning acumen, and allow your IT organization to operate in a more efficient, accountable, and responsive manner.
Where to Go for More Information
There are many places you can go to find out more about the SEI and CMM. One place is the Institute’s Web site. Go to www.sei.cmu.edu and you’ll find a wealth of information on the structure and implementation of CMM, as well as the other products and services offered by the SEI.
You might also want to visit the Matrix Resources web site, www.MatrixResources.com.
Of course many of the concepts contained in this article are discussed in-depth in the book, “Achieving Level 2: A Practical Guide for Implementing the SEI’s Capability Maturity Model.” This book is available from MATRIX, and is specially geared to people just entering the CMM framework.
Ready to Get Started?
If you think your organization is ready to adopt CMM there are a few directions you can take.
Most obvious, and probably easiest, you can go to your local bookstore, pick up a copy of “The Capability Maturity Model” (Addison-Wesley), read it, and start to design your program. This can be done relatively quickly and cheaply, but you need to really know what you are doing if you hope to get productive results.
Going in the other direction, you can hire one of the qualified CMM consulting companies to come and design a program for you. You’re probably assured of getting good results, but expect to spend a lot of time and more than a little money.
A better way—one that I often recommend—is to take the stop-and-go approach. This works especially well for organizations just getting into CMM (moving from Level 1 to Level 2), but it also works well for companies wanting to step up a level in the model. Here’s the approach:
The idea here is to first stop and evaluate where your organization currently stands in relation to CMM, and then develop a process program plan that tells you where you need to go from this point forward in order to meet your CMM goals.
Moving forward, you should begin with a Pre-assessment Review. You will need someone with a deep familiarity of CMM to work with you, but not necessarily an SEI-certified appraiser or assessor, or even a high-end consultant. Here you conduct an informal assessment of where your organization likely falls on the CMM scale. You document what policies, processes, and structures you have in place that meet CMM guidelines, and then you document at a high level the policies, processes, and structures you’ll need to get where you want to be on the scale.
It should also include an Assessment Report for management. This report outlines what steps your organization will need to undertake to shape your business along CMM lines. This report helps management plan its CMM efforts accurately and with a firm understanding of where they are in relation to CMM, where they need to go, and what it’ll take to get there.
The Spirit of Capability Maturity Model
The most common beginning mistake I have seen managers make when they adopt CMM is that they treat the model as a fixed process, as a set of rigid instructions that must be followed to the letter when being implemented in the workplace. They see the key processes and key practices outlined in the model as a series of laws, and they work to bend the way they do business to fold it under the banner of the model.
That approach rarely works, and—more importantly—it’s not the way CMM was designed to be used. The Software Engineering Institute understands that software shops are different. They come in different sizes. They work in different industries. They use different sets of tools. What works well in one place might not work so well in another. CMM, then, was designed as a process improvement framework, as a general guide that you tailor to the needs of your organization. To do this effectively and within the boundaries of the program, you need to approach CMM with two managerial skills at the ready:
Professional Judgment
To effectively use CMM in your shop you need to design and shape your CMM program using professional judgment. Based on your knowledge of your business and on the basic requirements of CMM, you interpret the guidelines for each Key Process Area into a program that fits your needs. Your judgment becomes important here because you must custom tailor the program so that it addresses the unique needs of your software shop, but doesn’t violate the spirit of CMM. The good news is that CMM provides plenty of latitude in implementing its recommendations. As long as your decisions are in harmony with the model, your program should end up CMM compliant.
Consistency of Application
It may sound funny to note this, but CMM does not really care too much if your processes start out being any good or not. What CMM cares about is process improvement—getting better at what you’re doing. In order to improve, you must know two things: 1) What it is you are doing now, and 2) How well it’s working. When you know that, you can tweak your performance and then observe it again. But, to do this you need to conduct your program in a consistent manner. It must be a dedicated effort, one you adhere to as your way of doing business. That’s not to say that consistency is the same as rigidity. You can tailor your processes to fit the larger or smaller demands of new projects, but you should strive to keep the foundation of the program intact. Consistent application will bring you two major benefits. It will help train your employees to work along process quality lines, and it will give you the ability to measure your performance in order to improve it.
If you can approach CMM with these two qualities in mind—that it is a flexible program designed for consistent application—you should find that CMM will fit well within most any software development environment.
About the Author
James Persse is a process and technology consultant working in the Atlanta area. His company, Light Touch Systems, Inc., provides process improvement and design services for the telecommunications, healthcare, and aerospace industries. He holds doctoral degrees in business and technology management from both Canella University and Columbia Pacific University, and he has written numerous papers in the field of technology and systems management. He is the author of “Achieving Level 2: A Practical Guide for Implementing the SEI’s Capability Maturity Model.”
You can contact Dr. Persse at 404-927-2161 or via email at james_persse@msn.com |