3.2. Quality Improvement Paradigm -cycle
The Quality Improvement Paradigm (QIP) evolved from the co-operative effort between NASA Goddard Space Flight Center / Flight Dynamics Division and the University of Maryland Department of Computer Science. The effort, called the Software Engineering Laboratory (henceforth SEL), was formed in 1976, with the goal of reducing the defect rate, cost and development cycle time of software. (Basili et al. 1994, 83.) The experiences of doing this work have been packaged into QIP-model designed to support process improvement and technology infusion. The chief author of the model is Victor Basili and it was first published in 1984 (Basili & Weiss 1984), but has been evolving even since and is currently actively applied and developed further by the Fraunhofer Institute, Germany. The QIP model is a part of a larger system model, called the Experience Factory. This system model also includes an infrastructure model, which is described in section 5.1.
The purpose of the QIP model is to support continuous process improvement and engineering of the development processes (Rombach 1998, 13), and to help in technology infusion (Basili 1994, 2, 65). One way to look at the model is also to see it as a model for the learning organisation, where the organisation establishes a way to develop practices through experimenting with them, and then capture and package them into a form that can be reused elsewhere, within certain boundaries. (Basili 1994, 4, Basili & McGarry 1998, 1.9.)
QIP is based on the principles that software discipline is, by its nature, evolutionary and experimental (Basili 1994, 3). The work for developing software is human-based and it is design work, not manufacturing. This means that there is very little repetition in the software development, which makes the use of statistical control as used in manufacturing sciences extremely hard and dubious. (Basili 1994, lecture notes.) Here the developers of QIP clearly take a different approach than e.g. the authors of the CMM and a number of other models that are based on the very idea of statistical control of processes. (Humphrey 1989, 3.)
The QIP takes the premise that all project environments and products are different – flight control software is entirely different thing from game software – and this means that there are certain prerequisites to experience reuse. These prerequisites include capturing and packaging of experiences, explanations on what kind of project and product types they have been applied successfully (and unsuccessfully) to, and how to tailor them to different environments and products. (Rombach 1994, 6, Basili 1994, 3.) The development of reusable experiences is analogous to developing reusable code. It takes more time and effort to produce it (Basili & McGarry 1998, 3.24, but see also Reifer 1997), and this is realised in the QIP approach in the form of repeated experimentation and need for dedicated resources to do the packaging. (Basili 1994, 3.)
The underlying philosophy is actually the making of scientific study through experimentation. (Basili & McGarry 1998, tutorial notes.) It is a bottom-up, closed-loop approach and inherently iterative process. (Rombach 1998, 15.)
The QIP does not – strictly speaking – require any specific method or approach to be used. However it very strongly recommends the use of GQM and is closely integrated to the Experience Factory infrastructure (Basili 1994, 21).
3.2.1. The QIP cycle
The QIP cycle (Fig. 1.) is broken into two closed loop cycles – the organisational (larger) and the project (smaller) cycle. The project specific feedback cycle is to provide feedback to the project during the execution phase in order to prevent and solve problems, monitor and support the project and to realign chosen processes with defined goals. The Organisational feedback cycle provides feedback to the organisation after the completion of the project. The purpose of the Organisational feedback is to analyse the concordance and discrepancy of the collected data against previous experiences and models. This helps to increase the understanding of the concluded experience and to capture some of that experience, and to accumulate reusable experience in the form that can be used by other projects. (Rombach 1994, 15.)
The organisational cycle represents how organisation learns. It is divided into following six phases:
Characterise and Understand
Set Goals
Choose Processes, methods, techniques and tools
Execute the Processes (run the project cycle)
Analyse Results
Package and store experience
The QIP cycle can be used both as a tool to learn more of already existing packaged experiences, or to create completely new, packaged experiences. The QIP cycle itself does not change, but if the goal is to produce a new experience and package it for future reuse, the fourth phase (Execute the Processes) requires several iterations. The reason for this is that the experience should not be packaged based on one single case, but requires several experimentations until there is sufficient knowledge of where it works and what it requires to work. (Rombach 1994, lecture notes, Basili 1994, 64.)
The Characterise and Understand is the starting phase for the cycle. The aim is to describe and comprehend the current project and its environment with respect to available process/product/quality models, data, intuition, etc. The phase also establishes quantifiable baselines based on past experiences and characterises their criticality. (Basili 1994, 4, Rombach 1994, 12, lecture notes.) The characterisation builds models of various factors and studies the interactions between these to increase understanding of the context where the improvement should be carried out. Factors taken into account include people, problems, processes, products, resources, and so forth. (Basili 1994, 19-20.)
The second phase is to Set Goals for successful project performance (covering both processes and products) and improvement results. The aim is to be able to get reliable, measurable data of the improvement and for this reason the goals need to be quantifiable and defined from a variety of perspectives, including customer, project and organisation viewpoints. (Basili 1994, 4, 21.) To be applicable to the case, they are based on the initial characterisation and strategic goals of the improvement. (Rombach 1994, 12.) The measurement plays multiple roles in the process engineering. When the measurements are baselined, they create an organisational memory. Across projects, they provide insight to the strengths and weaknesses of the current processes and products. They are a decision making tool, as they can be used to decide which techniques need to be adopted and/or refined. They also give objective evidence of the impact of changes and in general can be used to evaluate the quality of the process and product.
The objective of the third phase is to Choose Processes, Methods, Techniques and Tools that are appropriate for this project. The decision is based on the characterisation of the environment and product and on the goals that have been set. It is important to make sure that the selection is consistent with the goals set for products and processes, since otherwise there is little point in doing the measurements derived from the goals. (Rombach 1994, 12, Basili 1994, 39.) In addition the selected processes and the supporting methods, techniques and tools – which are all likely to be generic – need to be tailored to form an integrated set, that is applicable for the project and context (Basili 1994, 39).
The fourth phase of the organisational cycle is where the selected project(s) Execute the Processes. From organisation point of view, this phase is where the project cycle runs. The project cycle, which represents how project learns and guides itself, is divided into three activities:
Process Execution
Analyse Results
Provide Process with Feedback
The projects Execute the Processes to construct the products. At the same time data of resources, processes and products is being collected, validated and Results Analysed to measure the achievement of the goals. This information is then fed back to the project for corrective action. (Basili 1994, 4, 43.) The data collection needs to be integrated to the process to make it feasible. Data needs to be validated, because it is error prone and education and training in data collection is necessary, as everyone needs to understand the model. (Basili 1994, 42.)
The fifth phase is to Analyse Results to evaluate the practices, determine problems, record findings and make recommendations for future project improvements. The data is analysed against the goals and used to achieve better characterisation and understanding of the context, evaluate and analyse the experiments (improvements), determine problems, gain more information to be used for better prediction and control and to motivate future improvements. (Basili 1994, 4, 95.)
The sixth and final phase on the organisational cycle is to Package Experiences and to store them in the experience base for future reuse. It should be noted that if the QIP cycle is used for improving processes through experimenting with new procedures, methods or tools, it may require several cycle iterations and projects before there is sufficient information for packaging the experiences. New experiences should not be packaged based on single cases and it can sometimes take quite long to achieve improvements (Basili & McGarry 1998, 3.24), depending on project cycle duration and availability of projects for experimenting. In this phase, the experience gained is consolidated in the form of new or updated and refined models, baselines and other forms of structured knowledge gained from this and prior projects. (Rombach 1994, 13.) The types of experience packaged covers models for prediction (such as effort, error, and cycle time models), product models, process definitions, method and technique evaluations, quality models, products themselves, lessons learned and tailoring guidelines (Basili 1994, 45, Basili & McGarry 1998, 1.14).
3.2.2. PIA-variant of the QIP cycle
A variant of the QIP has been developed at Fraunhofer Institute, Germany. It is called PERFECT Improvement Approach, or PIA for short. The model has a modified project cycle and the steps have been refined to a more detailed level. The PIA model is depicted in Fig. 2., and the detailed phases are explained below. (Fraunhofer Institute 1998).
The organisational cycle, called Strategic Cycle in PIA, is much the same as in the QIP, except more detailed in descriptions. The fourth phase has been revised more than other phases.
In the first phase, Characterise, the function produces (or updates) a characterisation of the organisation and identifies the organisational improvement goals and major problems. The characterisation includes references to available process models, but if these don't exist it is recommended that one be developed during this phase, for further use in the measurement program. The improvement goals need to be derived from high-level business goals. The problems are used to find a starting point for improvement goals.
In the second phase, Set Goals, the strategic improvement goals are defined, corresponding hypotheses are developed, and an improvement program plan is produced for how to achieve the goals. The strategic goal is an instance of organisational goals and hypothesis further refines that goal. The formulation of the hypothesis is based on possible causes of the observed problems. The scope of the goals and hypotheses on this phase is generic (problem-specific), not project-specific.
In the third phase, Develop Improvement Plan, the projects, pilots, or experiments are identified for investigating the hypotheses. The resource usage and schedule for the experimentation is planned. The project characterisation provides a tool to select candidate projects, and documentation of the context within which the experiences apply. Suitable project(s) are evaluated and selected based on the goals in the improvement program plan. The resource and schedule planning is updated based on the information of the number of selected pilots within the program. It should be kept in mind that introducing new technology, methods or processes are likely to require more resources than deploying a packaged, reusable experience.
The fourth phase is where the selected pilot projects are running. At the strategic level, this is where the organisation Performs its Improvement Plan by conducting experiments. The strategic actions are to initiate the project and monitor and give guidance to it in order to ensure the success of the measurement program (data collection) and to motivate the project team. When initiating the project, the strategic level needs to prepare the project according to improvement plan and motivate and train the people participating in the project. Monitoring the project is to ensure that the improvement program is performed according to the defaults and guidance is given to coach the project through the improvement program part of the project. After the first experiment has been concluded, the improvement program plan is refined / modified (PIA phases 2 and 3 on the strategic level) by evaluating the results and experiences.
The fifth phase, Analyse Results, is entered once the improvement program and selected pilots have been concluded. The collected data is prepared for presentation. Data is then presented in a feedback session and feedback is analysed with reference to corresponding hypotheses of the goals of the improvement program. Results of other improvement programs can be used as a base for comparisons in this activity.
The sixth phase, Package Experience, the experiences are identified for future reuse needs and stored as experience package to the experience base.
The project cycle has been changed to include same six phases as the organisational cycle has. These are:
First Characterise the project and identify relevant models to be reused. This includes the characterisation of project, organisational environment of the project, and project goals. Existing reusable models are retrieved from experience base.
Next Set the project Goals in measurable terms and derive metrics from them. This includes goal identification, making the measurement plan for those goals, and validating the plans.
In the third phase the project Chooses appropriate Models for processes and develops the project plan. This phase includes tailoring the selected reusable models, developing new models, creating an instance of the agreed models and integrating the models and the measurement plans.
The fourth phase – Execute – is where the project performs according to its plans, collects data and both provides and gets feedback for project guidance.
The fifth phase is to Analyse the project and collected data and to suggest improvements. The analysis is done with reference to hypotheses.
The sixth phase is to Package the analysis results into improved reusable models. How this compares to the sixth phase of the strategic cycle is not entirely clear from the material. Either this is a small-scale for-this-project-use-only -kind of packaging, or perhaps refers to packaging the results of one project experiment in a multiple parallel experiment set-up.