Achieving CMMI Levels 2 and 3 with LabVIEW

As a result of the PC revolution, a flurry of standards and processes emerged to address software and system development. Nearly every standards body, industry, and government organization developed their own guidelines to tackle a range of issues; many of which had significant overlap. This became a burden to organizations due to the costs associated with training, certification, and assessments. Further complications arose due to the differences in terminology between guidelines. To address this problem, the Software Engineering Institute (SEI) at Carnegie Mellon University, with the help of the DoD (Department of Defense) and industry leaders, created the Capability Maturity Model Integration (CMMI) initiative. While the goals and concepts behind CMMI are independent of programming languages, the implementation and tools used for process adherence will need to vary between National Instruments LabVIEW graphical development environment and text-based languages in order to account for differences in the code created. This paper highlights these differences and provides tips and techniques for meeting the CMMI requirements with NI LabVIEW.

This document is supplied by Select National Instrumens Alliance Partner V I Engineering

What is CMMI?

The purpose of CMMI is to provide guidance for improving an organization’s processes and its ability to manage the development, acquisition, and maintenance of products or services. CMMI places proven approaches into a structure that helps companies appraise organizational maturity or process area capability, establish priorities for improvement, and implement these improvements. There are multiple CMMI models available. Consequently, you need to be prepared to decide which CMMI model best fits your organization’s process improvement needs. The place to start is to select which disciplines you want to include in your process improvement program and select a model representation. The two representations are staged and continuous. ¹

Since LabVIEW as a language is focused on the test and measurement community, the most applicable disciplines are in the CMMI-SE/SW model which stands for Systems Engineering and Software Engineering. Choosing a representation for the CMMI-SE/SW model largely depends on the organization’s requirements. The continuous representation enables organizations to select which process areas are best suited for continuous improvement and pursue them in any order. The staged representation is designed to provide maximum benefit through an implementation sequence of stages or Maturity Levels (ML). It also requires an organization to satisfy all process areas in a given ML before advancing to the next level. It is expected that all DoD divisions and contractors select the staged representation due to DoD requirements. Both representations are comprised of the same process areas.

This paper will assume the adoption of the staged representation. The staged representation consists of the following five levels.

Level 1 – Initial.

The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics.

Level 2 – Repeatable.

Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.

Level 3 – Defined.

The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization's standard software process for developing and maintaining software.

Level 4 – Managed.

Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled.

Level 5 – Optimizing.

Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies.²

Each level, with the exception of 1, is comprised of multiple process areas. In a staged representation all process areas in a given level must be successfully met in order to achieve that level. Each process area is comprised of specific goals and a single generic goal; all of which must be met in order to successfully meet that process area. The generic goal is the same for all process areas within a given level, whereas specific goals are unique to the associated process area. Specific goals and generic goals are then comprised of specific practices and generic practices, respectively, and are considered expected model components. Expected components describe what an organization will typically implement to achieve a required goal. Either the practices as described, or acceptable alternatives to them, are expected to be present in the planned and implemented processes of the organization before goals can be considered satisfied.

Figure1. CMMI Model Components

Benefit or Burden

While CMMI at first glance may appear to be a burden, there are a number of documented cases to prove otherwise. Below are a few of the actual results realized after implementing CMMI.³

1. Reduced Cost
   • 20% reduction in unit software costs (Lockheed Martin M&DS)
2. Decreased Development Time
   • Decreased the average number of days late from approximately 50 to fewer than 10 (General Motors)
3. Increased Quality
   • Reduction in defects found from 6.6 per KLOC (1000 Lines of Code) to 2.1 over 5 causal analysis⁴ cycles (Northrop Grumman)
4. Improved Customer Satisfaction

While results will vary between organizations and maturity level, nearly all organizations analyzed achieved a positive return.

Achieving Level 2 per CMMI SE/SW - Staged

To achieve CMMI Level 2 an organization must meet the Generic Goals (GG) and Specific Goals (SG) for 7 process areas. Those areas are:

• Requirements Management
• Project Planning
• Project Monitoring and Control
• Supplier Agreement Management
• Measurement and Analysis
• Process and Product Quality Assurance
• Configuration Management

Generic Goals

The GG for all process areas in level 2 is to Institutionalize a Managed Process, which consists of 10 Generic Practices (GP).

GG 2 Institutionalize a Managed Process

GP 2.1 Establish an Organizational Policy

GP 2.2 Plan the Process

GP 2.3 Provide Resources

GP 2.4 Assign Responsibility

GP 2.5 Train People

GP 2.6 Manage Configurations

GP 2.7 Identify and Involve Relevant Stakeholders

GP 2.8 Monitor and Control the Process

GP 2.9 Objectively Evaluate Adherence

GP 2.10 Review Status with Higher Level Management

For some organizations, meeting this GG simply requires mapping their current processes to the generic practices. As noted earlier, many test and measurement organizations fail to have detailed processes and thus require more assistance.

For those organizations, the VISTA consulting services are available to assist in a three phased approach. The first phase (Analysis) consists of a process assessment to understand the organizational goals, current development processes, development and process tools, and the guidelines which may be mapped to CMMI with minimal alteration. The result of this phase is a structured improvement plan to achieve ML 2 and the creation of a vision team. The second phase (Development) consists of mapping available guidelines to the generic and specific practices within each process area. Where there aren’t any guidelines in place, the VISTA consulting services will provide documentation and tailor it to meet both CMMI and customer requirements. During this phase, the development lifecycle (Modified Waterfall, Staged, etc) is defined to match organizational goals and project development. In this phase, tools are also selected as needed to assist with GP 2.6, 2.8, and 2.9. The final phase (Implementation) consists of implementation of the process, documentation, guidelines, and tools. This phase includes training for the organization on the process, process related tools, and development guidelines. It is typical at the conclusion of this phase to provide an analysis of what is needed to achieve the next set of goals.

Requirements Management

Requirements Management (RM) is the ability to manage requirements and the change impact throughout the development lifecycle. This is extremely important in a lifecycle process based on iterative development since changes are ongoing. The single goal within the RM process area is to manage requirements. The goal is further defined as “Requirements are managed and inconsistencies with project plans and work products are identified.” To assist in implementing the SG 1, CMMI lists five specific practices:

SG 1 Manage Requirements

SP 1.1 Obtain an Understanding of Requirements

SP 1.2 Obtain Commitment to Requirements

SP 1.3 Manage Requirements Changes

SP 1.4 Maintain Bidirectional Traceability of Requirements

SP 1.5 Identify Inconsistencies between Project Work and Requirements

SP 1.1 and 1.2 require a defined process to capture requirements and ensure their feasibility. Once the requirements are accepted, they need to be managed for changes. SP 1.3, 1.4, and 1.5 address this process. Once a requirement is changed, there must be a way to understand the impact that change will have on other requirements and work products. This link is often referred to as traceability. With bidirectional traceability it is then easy to identify discrepancies between work products and the requirements. It is possible to meet SP 1.3, 1.4, and 1.5 through intensive manual tracking and involvement. Unfortunately, this approach is prone to human error and can be costly for a large group. There are a variety of commercial off-the-shelf (COTS) packages available to assist in meeting these three areas with prices ranging from a few hundred dollars to many thousands per user. High-end users may be interested in tools such as IBM Requisite Pro or Telelogic Doors. These tools are further enhanced with integration to the LabVIEW development environment through VISTA to ensure bidirectional traceability between requirements, test plans, and VIs. For those users that aren’t interested in the high-end tools, an easy solution utilizes a Source Code Control (SCC) provider, such as Microsoft Visual SourceSafe or IBM ClearCase, with the VISTA Traceability Tracker. Both solutions may be tailored to meet CMMI requirements; therefore, the VISTA Consulting Services are available to help devise a solution to fit each customer’s needs.

Project Planning

The Project Planning (PP) process area consists of three specific goals. These goals are focused on creating reliable estimates, developing a project plan based on those estimates, and gaining commitment to the plan by the stakeholders. Since changes are likely to occur throughout the project, a plan needs to be devised to address changes. Each goal consists of a number of specific practices designed to assist with implementation.

SG 1 Establish Estimates

SP 1.1 Estimate the Scope of the Project

SP 1.2 Establish Estimates of Work Product and Task Attributes

SP 1.3 Define Project Life Cycle

SP 1.4 Determine Estimates of Effort and Cost

SG 2 Develop a Project Plan

SP 2.1 Establish the Budget and Schedule

SP 2.2 Identify Project Risks

SP 2.3 Plan for Data Management

SP 2.4 Plan for Project Resources

SP 2.5 Plan for Needed Knowledge and Skills

SP 2.6 Plan Stakeholder Involvement

SP 2.7 Establish the Project Plan

SG 3 Obtain Commitment to the Plan

SP 3.1 Review Plans that Affect the Project

SP 3.2 Reconcile Work and Resource Levels

SP 3.3 Obtain Plan Commitment

There are a variety of methods and tools available to meet SG 1 for software projects, such as COCOMO, Delphi, SEER, and others. Each method requires a variety of inputs to assess risk, size, and other factors. Many of these inputs are dependent on historical data from similar projects. Without reliable historical data the chances of creating an accurate estimate diminish quickly. Therefore, it is important to have access to past projects’ metrics. For LabVIEW, this is easily accomplished through the VISTA Metrics Tracker which keeps track of nodes, GOBS, and man-time spent by project, user, and lifecycle phase.

Once a reliable estimate is established, a project plan is needed to monitor progress and ensure successful completion. Developing this plan requires a defined process to account for potential issues such as project risks, data management, resources, skills, and stakeholder involvement. COTS packages such as MS Project and Primavera are useful in developing the initial plan to a corresponding budget and schedule requirements.

The last goal within the PP process area is Obtain Commitment to the Plan. This includes getting commitment to the initial plan and any and all future modifications. Due to this requirement, SG 3 needs to be closely linked to the Requirements Management and Configuration Management process areas to ensure traceability and commitment to the correct plan.

Project Monitoring and Control

After a project plan is established and the project begins, the Project Monitoring and Control (PMC) process area accounts for quantifying progress and providing corrective actions based on actual versus plan deviations. The process area consists of two specific goals and specific practices to assist with implementation of each:

SG 1 Monitor Project Against Plan

SP 1.1 Monitor Project Planning Parameters

SP 1.2 Monitor Commitments

SP 1.3 Monitor Project Risks

SP 1.4 Monitor Data Management

SP 1.5 Monitor Stakeholder Involvement

SP 1.6 Conduct Progress Reviews

SP 1.7 Conduct Milestone Reviews

SG 2 Manage Corrective Action to Closure

SP 2.1 Analyze Issues

SP 2.2 Take Corrective Action

SP 2.3 Manage Corrective Action

In order to meet SG 1, an organization must have metrics to monitor the project against. Therefore, this process area is closely linked to the Measurement and Analysis process area which defines the metrics to be used for project plans and project monitoring. Examples of metrics for project monitoring include SLOC, GOBS, nodes, status, hours, issues, and earned value. It is often useful to track multiple metrics to gain more insight into the actual performance. For LabVIEW the best way of tracking actual results is through the VISTA Project Management Tool and Metrics Tracker which records time, GOBS, nodes, status, and issues and compares it by user, file, and life cycle phase. These results may then be displayed through built-in actual versus planned reports or exported to MS Project for progress and milestone reviews.

In order to meet SG 2, an organization must have a process to address issues and deviations from the plan once they occur. There are a number of COTS-based tools to help automate issue tracking from inception to resolution. The benefits of these tools are heavily based on the quantity of issues and the size of the organization. Many smaller organizations are successful by implementing issue tracking spreadsheets controlled with configuration management (CM). For LabVIEW developers the VISTA Issue Tracker is integrated within the development environment, enabling developers to easily assign and review issues for specific VIs or the project.

Supplier Agreement Management

One area that is often neglected in a process improvement plan is Supplier Agreement Management (SAM). While an organization may be highly efficient in developing and managing their internal tasks, the overall project could be at risk if the suppliers are not able to deliver an acceptable solution. Within the SAM process area are two goals:

SG 1 Establish Supplier Agreements

SP 1.1 Determine Acquisition Type

SP 1.2 Select Suppliers

SP 1.3 Establish Supplier Agreements

SG 2 Satisfy Supplier Agreements

SP 2.1 Review COTS Products

SP 2.2 Execute the Supplier Agreement

SP 2.3 Accept the Acquired Product

SP 2.4 Transition Products

To meet SG 1, an organization must have a process to determine the source of products (COTS, internal, outsource), then evaluate the available suppliers on predetermined criteria and finally establish a formal agreement to set expectations. With supplier agreements in place, the organization must ensure that the desired supplier’s products or services meet all of the requirements of the project. Once determined, the organization may proceed in executing the agreement. Prior to final acceptance, the organization must once again ensure that the delivered product or service meets the requirements outlined in the agreement. After acceptance, the organization must integrate the product or service into the project. To assist in meeting the Supplier Agreement Management process area, the VISTA Consulting Services offer custom plans and templates to meet SG 1 and 2.

Measurement and Analysis

The purpose of the Measurement and Analysis (MA) process area is to create quantifiable results for all aspects of the engineering, management, and business areas. These results are the basis for many other process areas, including: Project Planning, Project Monitoring and Control, and Quantitative Project Management (ML 3). The initial focus of the MA process area is at the project level; however, the results may prove useful in addressing organizational and/or enterprise-wide needs.⁵ The MA process area includes two specific goals and many specific practices to guide implementation.

SG 1 Align Measurement and Analysis Activities

SP 1.1 Establish Measurement Objectives

SP 1.2 Specify Measures

SP 1.3 Specify Data Collection and Storage Procedures

SP 1.4 Specify Analysis Procedures

SG 2 Provide Measurement Results

SP 2.1 Collect Measurement Data

SP 2.2 Analyze Measurement Data

SP 2.3 Store Data and Results

SP 2.4 Communicate Results

To successfully implement SG 1, an organization needs to understand improvement objectives and devise a method of measuring those objectives. For LabVIEW, there are a variety of measures available to meet specific improvement objectives. The following table outlines a sample list of project objectives and corresponding measures.

Table 1. Measurement Objectives and Measures

Once the measures are defined, SG 2 requires a process to collect, analyze and report results. For LabVIEW projects this is easily accomplished through integration of the VISTA Metrics Tracker with the VISTA Project Management Tool and MS Project.

Process and Product Quality Assurance

The purpose of Process and Product Quality Assurance (PPQA)is to provide staff and management with objective insight into processes and associated work products. The practices in the PPQA process area ensure that planned processes are implemented, while the practices in the Verification (ML 3) process area ensure that the specified requirements are satisfied. These two process areas may on occasion address the same work product but from different perspectives. Projects should take care to minimize duplication of effort.⁶

SG 1 Objectively Evaluate Processes and Work Products

SP 1.1 Objectively Evaluate Processes

SP 1.2 Objectively Evaluate Work Products and Services

SG 2 Provide Objective Insight

SP 2.1 Communicate and Ensure Resolution of Noncompliance Issues

SP 2.2 Establish Records

To successfully meet SG 1, an organization needs to develop a plan to objectively evaluate the processes and products. An organization may evaluate the processes and products by using predefined criteria to measure adherence to process descriptions, standards and procedures. For products, it is important to complete the evaluations at incremental milestones and prior to customer delivery. After completing the evaluation an organization must communicate the results to relevant stakeholders, address all noncompliance issues, and establish records to meet SG 2.

Configuration Management

SG 1 Establish Baselines

SP 1.1 Identify Configuration Items

SP 1.2 Establish a Configuration Management System

SP 1.3 Create or Release Baselines

SG 2 Track and Control Changes

SP 2.1 Track Change Requests

SP 2.2 Control Configuration Items

SG 3 Establish Integrity

SP 3.1 Establish Configuration Management Records

SP 3.2 Perform Configuration Audits

The first step in meeting the CM process area is the establishment of baselines⁷. The CM process must first outline which items are required parts of a baseline. Typical configuration items may consist of code, support files (.mnu, .ini, .dll), documentation, requirements, and process guidelines. Once the configuration items are identified, organizations need to store the configuration items in a CM system. The CM system is responsible for tracking and controlling changes to configuration items in a central repository. There are a number of COTS CM repositories such as Microsoft Visual SourceSafe and Rational ClearCase. When combined with the COTS CM repositories, the VISTA Configuration Management and Project Management Tools offer the easiest solution to create, restore, and deploy baselines for LabVIEW applications.

To meet SG 2, an organization requires a defined process to track and control changes. Once a change request occurs, the impact of the change should be assessed in order to make corrective action to the project plan. Once a change is approved and made, the baseline must be updated.

The final step in implementing the CM process area is establishing integrity of the baseline. The first part to establish integrity is to maintain records about the configuration items. One way to accomplish this is to assign a status to each configuration item and version. The second part to establish integrity is through configuration audits. The role of the audits is to ensure the integrity of the items in the CM system and the baselines deployed. When deploying LabVIEW source code instead of executables, the best way to meet SG 3 is with the VISTA Configuration Management Tools and Integrity Tracker, which include an integrity check algorithm to ensure files match the correct baseline.

Level 2 Summary

In Figure 2, CMMI Implementation with VISTA shows an example of how VISTA can help an already structured development team improve to meet CMMI Level 2 and begin a foundation to meet Level 3.

[+] Enlarge Image

Planning for Level 3

Achieving Level 2 is a large accomplishment for most organizations resulting in a vast array of benefits. For other organizations, Level 2 is just the beginning for continuous process improvement. While Level 2 focuses on creating a process to manage projects in order to gain repeatable results on similar projects, Level 3’s goal is to create a process for the entire organization that not only defines how to manage the project but how to develop and test it as well. Since the goal is to have a single process for the entire organization, it is important to define and tailor guidelines to accommodate all types of projects.

To achieve Level 3, an organization must meet 11 new process areas. In addition, the organization must implement a new GG for all Level 2 process areas. The goal is to institutionalize a defined process; the process used for each process area must be part of the organizational standard process or a tailored version thereof, and the used process must contribute to building the organizational assets. Meeting this GG does not require the addition of new tools, but does require the creation of a more tightly coupled process for the entire organization and appropriate tailoring guidelines. The 11 new process areas that comprise Level 3 are:

Requirements Development

• Technical Solution
• Product Integration
• Verification
• Validation
• Organizational Process Focus
• Organizational Process Definition
• Organizational Training
• Integrated Project Management
• Risk Management
• Decision Analysis and Resolution

Since Level 3 is focused on creating a process for the entire organization, a Level 3 process should not be affected by the differences between developing a LabVIEW program or building an instrument. Instead, these differences need to be addressed through tailoring guidelines specific to each type of project. Instead of going through each process area, the remaining portion of this paper will highlight four specific process areas that may require tailoring for LabVIEW projects or where tools are available to aid in adherence. These process areas are Technical Solution, Verification, Validation, and Risk Management.

Technical Solution

Achieving Level 2 is a large accomplishment for most organizations resulting in a vast array of benefits. For other organizations, Level 2 is just the beginning for continuous process improvement. While Level 2 focuses on creating a process to manage projects in order to gain repeatable results on similar projects, Level 3’s goal is to create a process for the entire organization that not only defines how to manage the project but how to develop and test it as well. Since the goal is to have a single process for the entire organization, it is important to define and tailor guidelines to accommodate all types of projects.

To achieve Level 3, an organization must meet 11 new process areas. In addition, the organization must implement a new GG for all Level 2 process areas. The goal is to institutionalize a defined process; the process used for each process area must be part of the organizational standard process or a tailored version thereof, and the used process must contribute to building the organizational assets. Meeting this GG does not require the addition of new tools, but does require the creation of a more tightly coupled process for the entire organization and appropriate tailoring guidelines. The 11 new process areas that comprise Level 3 are:

Verification

Verification and validation are often confused with each other due to the similarities. CMMI defines Verification as “you built it right,” whereas validation is “you built the right thing.” Verification is also closely linked to the TS process area. After creating a component, such as a LabVIEW VI, it must be verified.

The verification process area consists of three goals. The first is to prepare for verification, which may be accomplished through a verification plan that details the components and procedures. The VISTA Documentation Template Set features sample plans for both verification and validation to assist organizations. The next goal is to perform peer reviews. To increase the effectiveness of peer reviews, it is important to have well documented and easy to follow code. The LabVIEW VI Analyzer Toolkit from National Instruments also complements peer reviews to look for specific issues. The last goal is to verify selected components, which requires a comparison of the current components to the requirements. The VISTA Configuration Management Tools make linking the CM process area to verification easy by establishing appropriate labels to versions of each component and establishing new baselines.

Validation

After the individual components are developed and verified, the components need to be integrated together into a final product. This process is defined in the Product Integration process area. Following the integration process, the resulting product must be validated according to the process defined in the validation process area. It is important to note that both verification and validation may occur at different stages of completion to ensure the components and products “were built correctly” and “are the correct items.”

The validation process area consists of two goals. The first is to prepare for validation, which may be accomplished through a validation plan that details the products, environment, and procedures. The VISTA Documentation Template Set features sample plans for both verification and validation to assist organizations. The second goal is to validate selected products and components, which requires a comparison of the expected performance to actual. The VISTA Configuration Management tools make linking the CM process area to validation easy by establishing appropriate labels to versions of each component of the product and establishing a new validated baseline.

Risk Management

The Risk Management process area defines the process for identifying, analyzing, and addressing risks. Risk management lasts the entire duration of the development life cycle and may include risks ranging from technical to organizational. An example of a technical risk is the complexity of the code, as more complex code will result in the risk of greater maintenance, verification, and validation time. Therefore the risk management strategy may state that a piece of software must be rewritten if it fails to meet a defined maximum complexity. For LabVIEW applications, the VISTA Path Tracker and Metrics Tracker may be used together to calculate the complexity as well as assess the risk associated with modifying code.

Summary

Achieving CMMI Level 2, 3, or higher is an intensive process that requires the dedication of all involved. For LabVIEW groups, it may be difficult to understand the benefits, as most do not currently measure productivity, quality, and delivery. For these groups, the first step is to understand the true cost of development. Only by understanding the cost of development are organizations able to set realistic goals and track improvement.

By understanding process improvement and the LabVIEW community, VISTA is able to provide organizations with the resources needed to improve. Since organizations vary in their needs and maturity, VISTA offers solutions to address each customer’s unique situation. The solutions often include a mix of consulting, training, and tools.

Resources

¹ From the "Getting Started with CMMI Adoption" at http://www.sei.cmu.edu/CMMI/adoption/CMMI-start.html

² From the "Capability Maturity Model Integration (CMMI), Version 1.1 (CMMI-SE/SW, V1.1)" at http://www.sei.cmu.edu/pub/documents/02.reports/pdf/02tr002.pdf

⁵ From the “Capability Maturity Model Integration (CMMI), Version 1.1 (CMMI-SE/SW, V1.1)” at http://www.sei.cmu.edu/pub/documents/02.reports/pdf/02tr002.pdf

⁶ From the “Capability Maturity Model Integration (CMMI), Version 1.1 (CMMI-SE/SW, V1.1)” at http://www.sei.cmu.edu/pub/documents/02.reports/pdf/02tr002.pdf

This document is supplied by Select National Instruments Alliance Partner V I Engineering

V I Engineering is an engineering solutions provider specializing in test strategy consulting, engineering information management systems, test and control systems, and staffing support. With sustained growth since 1992, V I Engineering has delivered the results Military/Aerospace, Medical Device, Consumer Electronics, and Automotive customers require. If you are looking to drive business improvement through test, V I Engineering should be your value added partner. V I Engineering is a Select Alliance Partner of National Instruments.

3 ratings | 3.33 out of 5

Print

Reader Comments | Submit a comment »