|
|
|
|
Published on Thursday, August 21, 2003 - 06:34 PM
|
CMMI® Level 5 Changes the Dynamics and Culture of an Entire Organization
A Case Study of RCG Information Technology’s
Road to Integrated Software Process Management
The Challenge
The lackluster growth of the economy has motivated many corporations to reevaluate their IT initiatives and seek the advantage of lower labor costs provided by offshore development centers (ODC). The fear, and sometimes reality, which these companies experience is that quality and accountability may be sacrificed as larger, more complex projects are sent overseas. Sometimes faster and cheaper means just that. How can clients be assured that, in their desire to do things faster and more economically, they are not being short-changed on quality? What safeguards exist to provide a predictably successful project outcome?
Faced with a growing demand for its offshore services, RCG Information Technology, Inc. committed to quality by making process improvement priority #1 of its market strategy -- a mission named delivery excellence. One of the most important pillars of this initiative was to achieve the highest-level evaluation of the Software Engineering Institute’s Capability Maturity ModelÒ Integration (SW-CMMIÒ Level 5) at its Philippines ODC. As explained by Manny Rodriguez, RCG IT national practice-offshore delivery, “Considering the amount of services the ODC provides our regions in the execution of their projects, having a strong process-centric offshore organization is key.” This goal was achieved on July 7, 2003, with the successful assessment of RCG IT Philippines as operating at CMMI-5 for software processes. This article details the road taken to attain this level of software process maturity, and how this achievement is fulfilling the promise of delivery excellence enterprise-wide.
What is CMMI 5?
The original Capability Maturity Model for Software (SW-CMM) is the creation of the Software Engineering Institute (SEI). SEI is a federally funded research and development center operated by Carnegie Mellon University created in 1984 through the sponsorship of the Department of Defense (DOD). Its charter was to provide government agencies with a means by which to assess the capability of government contractors in providing capable and repeatable software processes. SEI established the SW-CMM as a framework for organizational process improvement and a model for measuring the process maturity of an organization. It quickly became the de facto, internationally recognized standard for assessing and improving software development processes and was successfully adopted by more than 5,000 organizations worldwide.
Adaptations of the SW-CMM framework evolved for other organizational areas such as People (P-CMM), Software Acquisition (SA-CMM), Systems Engineering (SE-CMM), and Integrated Product Development (IPD-CMM). Acknowledging that organizations would need a more comprehensive framework for their enterprise, SEI, integrated these diverse frameworks into Capability Maturity Model Integration (CMMI).
The framework was designed to be flexible enough to support four disciplines (Figure 1): software engineering, systems engineering, integrated product and process development and supplier sourcing. Additional process areas and practices are added to the model as disciplines are accumulated.
Figure 1: CMMI Model Maturity Disciplines
CMMI has five levels of maturity (Figure 2). Level 1 characterizes organizations operating at an unpredictable, poorly controlled and reactive process. Organizations operating at Level 2 have a process characterized for individual projects and often reactive. Organizations operating at Level 3 have defined a proactive organizational process. Organizations operating at Level 4 apply greater control and measurement to their processes. Finally, organizations operating at Level 5 have optimized their process with an organizational focus on process improvement.
Figure 2: CMMI Model Maturity Levels
Process Improvement, One Bite at a Time
RCG IT has been involved in process improvement since 1998 when it standardized its proprietary project management methodology. Early in February 2000, the firm took its software development processes to task. It retained Process Plus, Inc., an SEI authorized assessor, to have its processes evaluated against the SW-CMM model. In October 2000, the firm received its first certificate of assessment indicating that it was operating enterprise-wide at Level 2 of SW-CMM. Remarkably, the company was not assessed at this level only for one location or project – which often occurs in these assessments - but for all of the firm’s business units.
Typically, companies graduate from one level of CMM to the next, and originally, this was RCG IT’s intent, since during the assessment for CMM Level 2 it had missed only nine key practices of the 100 that would have qualified it for a Level 3 maturity rating. However, the situation changed when the firm began to implement Level 3 process improvements. Moving from Level 2 to Level 3 required the implementation of a product life cycle. This life cycle development generally defines system development. For example, installing a data warehousing system for a client involves phases of development from gathering requirements, designing the system, coding testing, to finally deploying the finished product. Software is such an important component of systems being built today that it is virtually impossible to separate systems engineering from software engineering. For that reason, RCG IT “integrated” the software and system development processes into a Solution Development Life Cycle (SDLC).
Concomitantly, SEI announced the release of their new CMMI created for projects or organizations that wish to use common, integrated processes for both systems and software. So, it was natural for the firm to transition to the new CMMI model and to assess its wholly US-owned Philippine ODC at the newest CMMI Level 5. This made sense both from a practical and first-mover advantage standpoint, since very few companies worldwide are assessed at any CMMI level, and only eight are assessed at Level 5.
Getting Started
The first step taken to adapt and implement a new model was training. Training diminishes suspicion and resistance often encountered with sweeping organizational change. The firm contracted the services of Environmental Compliance Consultants International (ECCI), a leading standards-based consulting and training company located in the Philippines. ECCI was asked to conduct a series of related training sessions for the Manila personnel and to take the lead in the formulation, documentation and implementation of CMMI policies and procedures.
ECCI performed an initial assessment and needs-gap analysis to assess the current position of the firm in relation to the CMMI Level 5 requirements. Established policies, processes, procedures and artifacts were analyzed to ensure that they supported the practices and process areas of the CMMI model. The findings of this initial assessment formed the basis of a detailed activity and training plan. Training responsibilities were divided into two major areas. ECCI provided high-level CMMI conceptual training. Regional subject matter experts provided process training in areas of concentration such as configuration management, peer reviews, and function point analysis and tool sets.
General awareness training, an SEI requirement, was also provided for the entire ODC staff to orient them to the CMMI model and give them greater understanding of the scope of the process improvement efforts. Seven core CMMI implementation team members were selected from various existing ODC projects. The team participated in intensive training to enhance the skills needed to formulate, document and implement policies and procedures. Training continues as new employees are added to the RCG IT ODC team.
The needs-gap analysis revealed that the organization’s prior process improvement activities furnished a strong foundation for achieving CMMI Level 5, albeit there were areas of weaknesses which needed to be resolved when measured against the new “tougher” integrated model. The CMMI implementation team worked directly with ECCI in enhancing their existing project methodology artifacts to better align with the practices of the CMMI model. Specific skills of Functional Area Representatives (FARs) from within the ODC region were also enlisted. A dedicated Manila team addressed existing issues of requirements traceability, defect tracking and risk management. They also worked on new process requirements, such as measurement and analysis and decision analysis and resolution. As the artifacts were modified or added, they were peer reviewed by the core team members.
Spreading the Knowledge
As new and enhanced materials were being developed in Manila, they were posted and maintained in Livelink™, a knowledge management repository implemented by the firm in March of 2002 as part of their process improvement initiative. Livelink is a collaborative intranet, extranet and e-business application that delivers a fully integrated set of enterprise services including document management, virtual team collaboration, information retrieval, and business process automation. All project and solution development materials are maintained in Livelink and controlled through the ownership and permission schema of the application.
Automatic notifications of new items and versions of documents were communicated to the corporate Implementation and Quality Assurance (IQA) group in Edison, New Jersey, who were acting as the organization’s Solution Engineering Process Group. The materials were inspected, reviewed and implemented organizationally to preserve a consistency of process across the corporation. Real-time collaborative feedback from IQA ensured a steadfast focus on the business objectives while pursuing process efficiency.
As the work products were created during the analysis and development phases, they were implemented organizationally. At the same time, activities began to prepare the Manila implementation team for a full-scale CMMI appraisal. Early in February, ECCI introduced the team to the assessment and evaluation method. The Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) is designed to provide benchmark quality ratings relative to the CMMI models.
Demonstrating Compliance
The SCAMPI method relies upon an aggregation of evidence collected to demonstrate compliance with the practices and process areas of the CMMI model. Four sources of evidence are accepted: instruments, presentations, documents, and interviews. Instruments include such assets as questionnaires or an organizational mapping of CMMI model practices to its corresponding processes. Presentations consist of information prepared by the organization and visually or verbally delivered to the appraisal team to describe organizational processes and implementation of the CMMI model practices. Documents are artifacts, which reflect the implementation of one or more model practices. These typically include organizational policies, procedures and implementation-level artifacts. Interviews are verbally conducted with project team members, leaders and managers in both formal and informal settings using scripts or exploratory questions developed to elicit evidence of model compliance.
A CMMI Readiness Check (CRC) or pre-appraisal review, required under the SCAMPI method, was conducted on-site to determine readiness to proceed with a full verification-based appraisal. RCG IT had to provide objective evidence of sufficient quality and completeness to advance to the full appraisal. ECCI assisted in selecting lead appraiser and senior consultant Mick Spiers of the Quality Assurance Institute (QAI) India Limited. QAI has assisted companies such as Lucent Bell Labs, Oracle IDC, HCL Infosystems and ***** Software with implementing SEI-CMM frameworks.
Ready, Set, Go
The dates for the pre-appraisal review were mutually agreed to and set for the last week of February. Mr. Spiers organized a meeting with the firm’s senior management in Manila and sponsors of the CMMI implementation. This group helped establish business goals for the assessment to be followed during the appraisal process.
In order to evaluate the impact of process improvement activities throughout the capability maturity levels, four projects were chosen for the appraisal representing various phases of the SDLC. A SCAMPI appraisal team of 12 was created using the core implementation team and FARs. The team was selected because of their proficiency with the CMMI model and their understanding of direct association of artifacts to specific CMMI practices.
The team members were given three full days of compulsory training in the SCAMPI appraisal method by the lead appraiser. Over the following five days, the appraisal team evaluated each project against the SCAMPI model and reviewed project workbooks looking for consistent evidence of practice compliance. They augmented ‘paper’ evidence of artifacts with affirmations of process adherence by interviewing project team members to ensure that the practices were institutionalized. Finally, they identified opportunities for further improvement in nine of the 22 process areas needed to qualify at CMMI Level 5. The assessors produced an informal list (the full appraisal included a formal written report) detailing the areas in need of improvement, with specific recommendations on how to address each of the requirements. The appraisal team met with the regional sponsors to discuss the strategies, activities and timetable to bring each of the areas up to the CMMI Level 5 criteria.
The team immediately began to attack a new process improvement plan based on the findings of the CRC. The majority of the finding involved issues relatively easy to correct. Tailoring guidelines for the project management and SDLC processes were completed and implemented. Risk occurrence probabilities criteria were reworked as was the criteria used to develop alternate solutions, when necessary. The decision was made to standardize requirements management with a Rational® RequisitePro® tool.
The Appraisal
The improved processes and procedures were implemented organizationally and specifically retrofitted into the four projects slated for the appraisal by the SCAMPI team. This process took several months. Metrics were gathered and maintained to monitor the impact these improvements had on overall project quality and performance. Progress against the process improvement plan was analyzed and fine-tuned by the team. Anticipation for the impending appraisal gripped the entire team.
Mick Spiers returned to Manila in late June to conduct a full SCAMPI. No assumptions regarding to the maturity or capability of an organization are allowed to be carried over from the CMMI Readiness Check. Everything that was evaluated during the CRC had to be re-evaluated in the SCAMPI. Due to the commitment of the organization and the hard work and dedication of the SCAMPI team, the Philippines ODC achieved their goal of CMMI Level 5 on July 7, 2003.
The Benefits – Quantitatively Speaking
Weighing the effects of process improvement activities takes years of gathering and analyzing metrics. Quantitative measurements of increased productivity, shorter cycle times, increased quality and improved customer and employee satisfaction will not be available for some time. The achievement of a CMMI Level 5 rating through SEI has furnished strong, unimpeachable evidence of the firm’s process capability. The achievement of this level of maturity provides assurances to the IT industry of the seriousness of the commitment to delivery excellence and predictability of services. The fact that it was achieved in the ODC removes the last obstacle of concern – will quality and accountability be sacrificed by sending the work overseas?
The firm’s experience with the prior SW-CMM assessment serves as positive evidence of the benefits of achieving a SW-CMM Level maturity rating. To best illustrate this it is fitting to compare project results against reported industry statistics. Standish Group International, Inc. has been gathering data for the past nine years on why projects fail. Their research is among the most widely quoted statistics on project performance in the IT industry. They have surveyed thousands of software development projects from hundreds of companies, compiling and releasing those statistics as The CHAOS Chronicles.
The Standish Group, a research and analysis organization, divided project classifications into three resolution types: successful, challenged or failed. Successful projects are completed on-time, on-budget and with all the features and functions as originally specified. Challenged projects are completed and operational, but are either over-budget, over the time estimate or with fewer features and functions than initially specified. Failed projects are cancelled before completion. According to their research, only 72 percent of all projects undertaken by IT organizations are completed. Twenty-six percent of all projects can be classified successful while 46 percent are challenged. Sixty percent of the challenged projects involve schedule overruns, and 40 percent involve cost overruns. Twenty-eight percent of all projects fail outright.
RCG IT’s performance in fiscal year 2001 following their Level 2 assessment and iterative process improvement activities far outpaced published industry performance indicators. The organization had a 99 percent completion rate of all projects undertaken. Ninety-four percent of their projects can be classified successful including four percent of their projects that completed under-budget. Five percent of their projects were challenged incurring a cost overrun average of five percent over budget. One project failed due to a client loosing their funding and canceling it before completion. Successful project completion rates rose six percent over the prior fiscal-year figures, and there is consensus that the success can be directly attributed to the continued focus on processes necessary for achieving the repeatable maturity level.
The expectations of achieving shorter cycle times and greater productivity by adopting the new CMMI model are being realized. Initial evidence since the recent SCAMPI assessment indicates improvements in estimate accuracy, decreased post release defects, decreased cycle times and increased productivity rates. These project measurements are being maintained in a metrics database and analyzed monthly by the original CMMI implementation team to determine future process improvement activities.
The Benefits – Qualitatively Speaking
There are also qualitative benefits that avoid measurement but are just as easily realized. One benefit which defies measurement is the impact on the organizational culture and advances in the firm’s delivery excellence commitment to clients. Employees, as stakeholders in the growth and development of their organization, become more focused and interested in maintaining process improvement, as they are exposed to the appraisal activities. The CMMI framework ensures employees understand their roles and responsibilities, commit to owning those responsibilities and rely on a disciplined mechanism by which to monitor their execution. Appraisal team members specifically gain process improvement insight. They carry that knowledge and insight into the new projects impacting the way projects are conducted and extending that insight to the project team members. An organization undertaking a SCAMPI appraisal reaps the qualitative benefit of having inculcated process improvement into their culture. Organizational predictability, accountability and delivery excellence just gets easier as it is ingrained into the culture.
References
[CMU/SEI 01] Members of the Assessment Method Integrated Team Standard CMMISM Appraisal Method for Process Improvement (SCAMPISM), Version 1.1: Method Definition Document (CMU/SEI-2001-HB-001). Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2001. http://www.sei.cmu.edu/publications/documents/01.reports/
01hb001.html.
[CMU/SEI 02] CMMI Product Team Capability Maturity Model® Integration (CMMISM), Version 1.1, CMMISM for Software Engineering (CMMI-SW, V1.1), Continuous Representation (CMU/SEI-2002-TR-028, ESC-TR-2002-028). Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2002. http://www.sei.cmu.edu/publications/documents/
02.reports/02tr028.html.
[Phillips 02] Phillips, Mike Presentation by Mike Phillips, CMMI Program Manager
at the European Software Engineering Process Group Conference
April 2002. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, December 2002. http://www.sei.cmu.edu/cmmi/
presentations/euro-sepg-tutorial/sld001.html
[Shrum 99] Shrum, Sandy Continuous and Staged, a Choice of CMMISM Representations Spotlight. Volume 2. Issue 4. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, December 1999. http://interactive.sei.cmu.edu/Features/1999/December/Spotlight/
Spotlight.dec99.html
|
|
|
|
Tip of the day:
Establish an environment where reporting bad news in a timely manner is encouraged rather than an environment where fear prevents the flow of critical information.
2009-10 allPM.com Editorial Calendar
Register for allPM
December/January Poll Question
Get Involved With allPM.COM
Submit your...
PM Glossary
Latest Forum Posts
|
