When we originally put together the whole maturity model framework, we were doing it for the acquisition community. We knew that we had to give guidance on the question, “What do you look for?” So we focused on artifacts.
What is the evidence of an organization’s performance? Say we want an organization that is producing plans and that uses configuration management and requirements management. What are the things that you’d have to have if you did that? You could say, “Well okay, if you do planning then you ought to have plans.” You can now look around and say, “Do you have plans?”, “Do you have review meetings?”, and “Do you have review meeting minutes?”
It shouldn’t be expensive to produce. If you’re actually using that process those are things you ought to naturally have; you look and make sure they’re there.
When we put together the original maturity models, this is what we did. While the acquisition people didn’t really understand the details, they could tell that somebody had a development plan, it was for this project, it was signed off, and it had what appeared to be the right stuff in it. It was fairly easy to do. The original intent was that these artifacts were the natural consequence of the process being used, so there shouldn’t be a lot of cost involved in preparing for such a review.
Now notice what happened with CMMI: Appraisals became important so organizations were in a great hurry to reach a high maturity level. Increasingly, organizations discovered that it is extremely hard to change what the development teams actually do. It’s a heck of a lot quicker to have task groups generate documents that meet the needs of the appraisal. So you’ve got groups that put together configuration plans, development plans, and all of this stuff. And it’s not developed by the developers—but there is nothing in CMMI says that it’s wrong to do this—so you’ve got all of these artifacts. Now you have these independent groups bureaucratically producing stuff that has no relationship to the work that is being done.
And so you don’t improve organization performance at all. Unfortunately CMMI, as currently built, doesn’t protect against that.
See also what Tom DeMarco had to say about “big M methodology” as the CMMI in his book Peopleware.