PV260 - SOFTWARE QUALITY
SOFTWARE QUALITY MANAGEMENT PROCESS
Bruno Rossi
brossi@mail.muni.cz
LAB OF SOFTWARE ARCHITECTURES
AND INFORMATION SYSTEMS
FACULTY OF INFORMATICS
MASARYK UNIVERSITY, BRNO
"Essentially, all models are wrong,"Essentially, all models are wrong,
but some are useful"but some are useful"
George BoxGeorge Box
3/60
●
Kitchenham gave five perspectives of software quality:
1. Transcendental view
→  can be recognized but difficult to define exactly
2. User view
→  fitness for purpose
3. Manufacturing view
→  conformance to specification
4. Product view
→  from inherent product characteristics
5. Value-based view
→  depends on customer’s willingness to pay
Introduction
B. Kitchenham and S. L. Pfleeger, “Software quality: the elusive target,” IEEE Software, vol. 13, no.
1, pp. 12–21, Jan. 1996.
4/60
© 2012 Steve Easterbrook creative commons license.
We deal with process
quality in this lecture
Introduction
5/60
●
Quality is a concept that starts with the development
process, goes on with the software product and finally to
the user with the results from software usage
Introduction
6/60
●
“Software quality management (SQM) is the collection of all processes
that ensure that software products, services, and life cycle process
implementations meet organizational software quality objectives and
achieve stakeholder satisfaction” (SWEBOK 3.0)
●
SQM defines processes, process owners, requirements for the processes,
measurements of the processes and their outputs, and feedback
channels throughout the whole software life cycle.
What is Software Quality Management (SQM)
7/60
The Impact of Quality
8/60
How much “Quality” is needed? If we could quantify it, should/could we
have 100% quality in our software products?
Thinking about some of the systems that are/have been market leaders,
how was the quality of the proposed solution?
It was shown that even by starting with the same level
of customer satisfaction what really matters is the
competition with the other companies
Babich, P. (1992) “Customer Satisfaction: How Good is Good Enough?” Quality Progress. https://docuri.com/download/customer-satisfaction_59c1d667f581710b2866ba80_pdf
A
B C
dissatisfied
A
dissatisfied
C
dissatisfied C
dissatisfied B
dissatisfied
A
dissatisfied
B
satisfied A
satisfied B satisfied C
This implies that you need to do better than your
competitors in terms of quality
The Impact of Quality
9/60
●
What makes software different
– Really big economies of scale
→  additional physical inputs for production result in a non-proportional
increase in output with a decrease in average costs
– Increasing returns
→  “tendency for that which is ahead to get further ahead, for that
which loses advantage to lose further advantage” (W.B. Arthur)
– Network Externalities
→  the value of goods to users increases as more people adopt them
– High initial costs
→  software is complex to design and to deliver to the market
– Switching costs
→  switching to other software might be costly (e.g. training to
re-do, change of infrastructure)
The Impact of Quality
10/60
Network Externalities: for some types of goods, the value of a
good increases with the number other people adopting the same
good
●
“These effects arise both because the ability to communicate and share data with
others will be greater, and because it is more likely that complementary hardware,
software, and wetware (i.e., brain cells) will be available, when there is a large
base of users of the software”, Katz & Shapiro
Network Externalities
Path Dependence
Lock-in
Network Externalities
11/60
●
Path Dependence: in common knowledge means 'history matters'
●
But there is more.. If we consider historical paths, not only the path
depends on previous events but also produces a self-reinforcing
mechanism that leads to the reinforcement of the path selected. In this
way, switching to another path will become at every step more costly
●
In general, due to increasing returns, a phenomenon can assume
contagious effects
Path Dependence
12/60
●
So are we destined to keep the current status in the future
(e.g. using Facebook, Google, etc..)?
●
One nice answer comes from Christensen (2013) that
identified two types of technologies:
– Sustaining – an innovation that improves a product in either
expected or unexpected way, but does not lead to a paradigm
shift
– Disruptive – an innovation that can potentially create a new
market and constitutes a paradigm shift (e.g. facebook
creating the whole social networking idea, or touch screens for
the whole mobile phones industry)
●
Disruptive innovations are a way to change from the status quo
→  can you name some companies that were market leaders before a
disruptive technology appeared?
Christensen, C. (2013). The innovator's dilemma: when new technologies cause great firms to fail.
Harvard Business Review Press.
Sustaining & Disruptive Innovations (1/3)
13/60
●
Disruptive technologies: initially the quality might be lower
than current technologies, but will catch-up quickly
Christensen, C. (2013). The innovator's dilemma: when new technologies cause great firms to fail. Harvard
Business Review Press. (img source: https://commons.wikimedia.org/wiki/File:Disruptivetechnology.png)
Sustaining & Disruptive Innovations (2/3)
14/60
●
Gartner's Hype Cycle (GHC) for Emerging Technologies → 
maturity of technologies in a domain
←  2013 GHC
Do you see any
technology that
maintained the
“hype”?
Sustaining & Disruptive Innovations (3/3)
15/60
Q&A
16/60
a) Can you name one disruptive innovation either in
Software or in the IT world that impressed you particularly in
the last few years?
b) Thinking in terms of 5-10 years from now, what do you
think will be a disruptive technology in the software/IT
world?
General Questions
17/60
SQM Categories
18/60
●
According to ISO/IEC 12207/15288:2007
Processes require a purpose and outcome. All processes
have at least one activity
Activities are constructs for grouping together related tasks
A task is a detailed arrangement for the implementation of
a process. It can be a requirement (“shall”), a
recommendation (“should”) or a permission (“may”)
Notes are used to explain better the intent or mechanism
of a process
What is a Process (according to IEEE)
19/60
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
A → All Planning for SWQ
B → Assess the adequacy of
software processes
C → Compliance to standards
established for the project for
products
D → Activities to improve
software processes
SQM comprises four subcategories
20/60
Source: http://softwaretestingfundamentals.com/sqa-vs-sqc/
SQA vs SQC – What’s the difference?
21/60
●
A quality plan defines how an organization will reach the quality
objectives
●
Usually covers
– Quality objectives and goals
– Quality management scope
– Organisation & responsibilities
– Resource requirements
– Cost benefit analysis
– Activities and deliverables
– Schedule
– Risk analysis
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
A. Software Quality Planning (1/2)
22/60
●
Quality at the organization and project Levels
Source: http://www.chambers.com.au/glossary/quality_planning.php
Project /
Product
development
Project /
Product
development
Quality
management
Group
Process Group
SQMS
Development
Process
Improvement
Plan
Project Manager /
Quality
Representative
Project /
Product
development
Project
Quality Plan
Quality Management System
standards
procedures
process
description
QA objectives
& metrics
Goals
Resources
SQMS tailoring
SQMS = Software Quality Management System
CompanyLevelProjectLevel
A. Software Quality Planning (2/2)
23/60
●
SQA means monitoring constantly the software engineering
process to ensure that the approaches/methods/processes applied
lead to quality within the project
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
B. Software Quality Assurance
24/60
●
IEEE Std 730-2014 – Outline for software quality planning:
Heimann, D. I. (2014). An Introduction to the New IEEE 730 Standard on Software Quality Assurance, SQP Vol 16, N.3
1. Purpose & Scope
2. Definitions & acronyms
3. Reference documents
4. SQA Plan Overview:
4.1 Organization & independence
4.2 Software Product Risk
4.3 Tools
4.4 Standards, practices and conventions
4.5 effort, resource, schedules
5. Activities, Outcomes and tasks:
5.1 Product Assurance:
5.1.1 Evaluate plans for conformance
5.1.2 Evaluate product for conformance
5.1.3 Evaluate product for acceptability
5.1.4 Evaluate product life cycle support for
conformance
5.1.5 Measure products
5.2 Process assurance:
5.2.1 Evaluate life cycle support for conformance
5.2.2 Evaluate environments for conformance
5.2.3 Evaluate subcontractor processes for
conformance
5.2.4 Measure processes
5.2.5 Assess staff skills & knowledge
6. Additional Considerations
6.1 Contract review
6.2 Quality Measurement
6.3 Waiver and deviations
6.4 Task repetition
6.5 Risks in performing SQA
6.6 Communication strategy
6.7 Conformance process
4. SQA Records:
7.1 Analyze, identify, collect, file, maintain,
dispose
7.2 Availability of records
B. Software Quality Assurance Planning (1/2)
25/60
IEEE Std 730-2014 – how is Agile considered?
●
Agile →  the product backlog is the contract, the standards help in
defining the role of the backlog as contract
●
The SQA product part in IEEE730 can be used to defined the “done”
criteria
●
Non-conformances to standards are inserted in the backlog and
addressed in sprints in which are scheduled
●
Acceptance is a continuous process in Agile
●
IEEE730 contains an appendix with details about Agile adoption of the
standard
Heimann, D. I. (2014). An Introduction to the New IEEE 730 Standard on Software Quality Assurance, SQP Vol 16, N.3
B. Software Quality Assurance Planning (2/2)
26/60
●
SQC means to constantly monitor the software engineering
process/product to check for conformance to applied standards
(e.g. CMMI) or produced artifacts
●
Some examples of methods
→  The Goal Question Metrics Approach (seen on the measurement lecture)
→  The Plan-Do-Check-Act method
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
C. Software Quality Control
27/60
●
Improve process effectiveness, efficiency and other characteristics
with the aim to improve software quality
●
Very often software process improvement practices are
embedded within the process (e.g. capability models)
●
Some methods:
→  Capability Maturity Model (CMM) and Capability Maturity
Model Integration (CMMI)
→  ISO/IEC 15504 (SPICE)
→  ISO 9001 Specification (seen during PA017 SEII)
→  Personal Software Process (PSP)
and Team Software Process (TSP)
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
D. Software Process Improvement
28/60
Some Historical
Models for Software
Quality Management
29/60
●
What is the name of the simplest quality process
management practice in your opinion?
→  Actually, it involves no process
Simplest Quality Management Model
30/60
●
Cowboy Coders write code according to their rules
●
Some sentences you might have heard:
“If possible, the customer should only see the final versions of the
product. It is important to minimize the contact with the customer so
time is not wasted”
“The code is mine and none is allowed to touch it!”
“I do not need any analysis, design nor documentation”
“Even if it is broken, do not touch it! Try to hide it!”
“People who need comments in order to
understand my code are too dumb to be
working with me “
Image: https://www.cs.utexas.edu/blog/cowboy-rides-away-now
→  See http://c2.com/cgi/wiki?CowboyCoder
Cowboy Coding
31/60
●
The Personal Software Process (PSP) is a disciplined software development process
that works at the individual level
PSP0
Defining &
using
processes
PSP1
Planning &
Tracking
PSP2
Quality
Management
TSP
Team
Development
Estabilish a measured
performance baseline
Practice size and effort
estimation
Practice defect
management and improve
design practices
Defined around 1995: Humphrey, Watts
S. A discipline for software engineering.
Addison-Wesley Longman Publishing
Co., Inc., 1995.
Personal Software Process (PSP)
32/60
Process
scripts
PSP0 Process
Planning
Development
Postmortem
Design
Code
Compile
Test
Requirements
Time & Defect
Logs
Plan Summary
Finished Product
Scripts provide
the steps for
all processes
Templates
helpful when
following the
process
PSP0 – First Level
33/60
●
Emphasizes making accurate and precise size measurements
●
Incorporates measuring the size of the programs produced
●
Accounts for various types of LOC in the programs produced
●
Begins to look at process improvement
●
The following elements are added
– Estimating and reporting software size
– Use of a coding standard
– Recording process problems and improvement ideas
PSP0.1 - Improvement
34/60
●
All is based on forms, scripts and logs
W. S. Humphrey, “A Discipline for Software Engineering”, 1995
Table C3 PSP0.1 Project Plan Summary Example
Student Student 11 Date 2/1/94
Program Object LOC Counter Program # 3A
Instructor Humphrey Language C
Program Size (LOC) Plan Actual To Date
Base(B) 87
(Measured)
Deleted (D) 0
(Counted)
Modified (M) 6
(Counted)
Added (A) 113
(T-B+D-R)
Reused (R) 0 0
(Counted)
Total New & Changed (N) 90 119 315
(A+M)
Total LOC (T) 200 396
(Measured)
Total New Reuse 0 0
Time in Phase (min.) Plan Actual To Date To Date %
Planning 10 11 36 6.4
Design 25 21 63 11.2
Code 75 97 249 44.2
Compile 20 4 35 6.2
Test 45 39 105 18.7
Postmortem 20 33 75 13.3
Total 195 205 563 100.0
Defects Injected Actual To Date To Date %
Planning 0 0 0
Design 1 3 11.5
Code 8 23 88.5
Compile 0 0 0
Test 0 0 0
Total Development 9 26 100.0
Defects Removed Actual To Date To Date
%
Planning 0 0 0
Design 0 0 0
Code 0 0 0
Compile 2 13 50.0
Test 7 13 50.0
Total Development 9 26 100.0
After Development 0 0
PSP0.1 - Improvement
35/60
●
PSP1 introduces the concept of
software effort estimation and
the usage of historical data
●
Using the PROBE (PROxy-Based
Estimating) size estimating
method
●
Using linear regression, PROBE
size estimation, regression
analysis is based on historical
estimated object LOC (the x
data) and actual new and
changed LOC (the y data)
Estimate Objects (proxies)
Nr. of
methods
Start with the conceptual design
Object
type
Relative
size
Reuse
categories
Estimate new and changed LOCs
Estimate development time
Calculate prediction interval
Planning estimates
W. S. Humphrey, “A Discipline for Software Engineering”, 1995
PSP1 – Second Level
36/60
●
PSP2 introduces design and code reviews methods for
evaluating and improving the quality of your reviews
●
There are three new process elements
– PSP2 project plan summary
– PSP2 design review checklist
– PSP2 code review checklist
PSP2 – Third Level
37/60
●
The idea behind PSP is that it should lead to more team-aware
processes once developers have tried a self-disciplined approach
PSP Skill-building
Personal Plans
Planning Methods
Earned Value
Process data
Quality Measures
Defined Processes
TSP Team-building TSP Team-working
Engineering
Disciplines
Team
Disciplines
Management
Disciplines
Committment
Aggressive plans
Quality ownership
Project goals
Plan ownership
Plan detail
Team roles
Team resources
Quality priority
Cost of quality
Follow the process
Review status
Review quality
Communication
Change management
Integrated
Product
Teams
→ see hackystat
From PSP to TSP
38/60
●
A measure of Software Quality developed at Motorola
●
The focus of Six Sigma is on eliminating defects, that is everything that is
outside from the customers specifications
●
No more than +/- six times the standard deviation from the process
mean → 3,4 Defects Per Million datapoints
●
Data is a key to understand the underlying processes and take decisions
Sigma Level Defects x 1M
2 308.537
3 66.807
4 6.210
5 233
6 3,4
Defined in the ‘80s in industry but only in
90’s adopted widely, later on adopted by
software engineering community.
σ=
√∑(xi−x)
n−1
Six Sigma
39/60
Define Measure Analyze Improve Control
Who are the
customers and the
needs
How is the process
defined and how
are the defects
measured
What are the most
important causes of
the defects
How can the
causes of the
defects be
eliminated
What actions are
needed to sustain
improvement
In Six Sigma, a defect is defined as “Any product, service, or process
variation which prevents meeting the needs of the customer and/or
which adds cost, whether or not it is detected”
Key aspect: understand the relation between depend variables (Y,
defects) and independent variables (X, causes)
Y =f (x1. x2, x3, ..., xn)
Six Sigma – Process (1/2)
40/60
Define Measure Analyze Improve Control
●Define Project
scope
●Estailish formal
project
●Identify needed
data
●Obtain data set
●Evaluate data
quality
●Summarize &
baseline data
●Explore data
●Characterize
process & problem
●Update
improvement
project scope &
scale
●Identify possible
solutions
●Select solution
●Implement (pilot as
needed)
●Evaluate
●Define control
method
●Implement
●Document
Six Sigma – Process (2/2)
41/60
Define Measure Analyze Improve Control
●
Benchmark
●
Contract/charter
●
Kano Model
●
Voice of the
customer
●
Voice of the
Business
●
Quality
Function
Deployment
(QFD)
●
GQ(I)M and
indicator
templates
●
Data collection
methods
●
Measurement
System
Evaluation
●
Cause & Effect
Diagram/Matrix
●
Failure models
& effects
analysis
●
Statistical
inference
●
Reliability
Analysis
●
Root Cause
Analysis
●
Hypothesis Test
●
Design of
experiments
●
Modeling
●
ANOVA
●
Tolerancing
●
Robust Design
●
System
Thinking
●
Decision & Risk
Analysis
●
Performance
Analysis Model
●
Statistical
Controls:
●
Control charts
●
Time series
methods
●
Non-Statistical
controls:
●
Procedural
adherence
●
Performance
Management
●
Preventive
measures
Basic Tools (Histogram, scatter plots, run charts, pareto charts, cause & effect diagram, Control chart,
descriptive statistics), baseline process flow map, project management, management by fact, sampling
techniques, survey methods, defect metrics
Six Sigma - Tools
42/60
Software Process Maturity Models
43/60
●
As defined in ISO/IEC 15504-2 (SPICE)
→ The process is not implemented or fails to achieve
the purpose
→ The process is achieving the purpose
→ The process is now running in a managed way
(planned, monitored, adjusted) – work products are
established, controlled and maintained
→ The managed process is now implemented using a
defined process capable of achieving process
outcomes
→ The established process now operates within
defined limits to achieve its process outcomes
→ The predictable process is continuously improved
to meet relevant current and projected business goals
Capability
Level
Process Capability
0 Incomplete Process
1 Performed Process
2 Managed Process
3 Established Process
4 Predictable Process
5 Optimizing Process
Process Maturity Levels
44/60
ENG.1
Engineering
Customer-
Relationship
Support
ManagementManagement Organisation Support
ENG.1
ENG.1
CUS.1
ENG.1
MAN.1
ENG.1
ORG.1
ENG.1
SUP.1
ENG.1
ENG.1.BP1
ENG.1
CUS.1.BP1
ENG.1
MAN.1.BP1
ENG.1
ORG.1.BP1
ENG.1
SUP.1.BP1
ENG.1
ENG.1.WP1
ENG.1
CUS.1.WP1
ENG.1
MAN.1.WP1
ENG.1
ORG.1.WP1
ENG.1
SUP.1.WP1
5 Categories
24 Processes
201 Base
Practices
109 Work
Products
ISO/IEC 15504 (SPICE)
45/60
5 Process
Categories
(SUP=Support)
Example Process Definition (1/2)
46/60
Overall 24 processes
are specified
Example Process Definition (2/2)
47/60
●
SPICE is a two-dimensional level model
– Processes and categories on one side (Process Dimension)
includes Base practices, work products, characteristics→ 
→  Does the process reach its goals?
– Capability of processes on the other side (Capability
Dimension) includes levels, process attributes,→ 
management practices →  How well is a specific goal met?
Process Maturity Levels & Attributes (1/2)
48/60
●
As defined in ISO/IEC 15504-2 (SPICE)
Process Maturity Levels & Attributes (2/2)
49/60
Assessment Model
Process Dimension Capability Dimension
Process categories (5)
Processes (24)
Capability Levels (6)
Process Attributes (9)
Base Practices (201)
Work Products (109)
Management Practices (33)
Resources & Infrastructure
characteristics
Indicators of process
performance
Indicators of process
capability
SPICE Assessment Model (1/2)
50/60
Assessment Model
Process Dimension Capability Dimension
Process categories (5)
Processes (24)
Capability Levels (6)
Process Attributes (9)
Base Practices (201)
Work Products (109)
Management Practices (33)
Resources & Infrastructure
characteristics
Indicators of process
performance
Indicators of process
capability
Assessment Scale
●
up to 15% – N (Not performed/achieved)
●
> 15% to 50% – P (Partial)
●
> 50 to 85% - L (Large)
●
> 85% - F (Full performance / achievement)
PA1.1 PA2.1 PA2.2 PA3.1 PA3.2 PA4.1 PA4.2 PA5.1 PA5.2
ENG.1
ENG.2
MAN.2
ORG.1
N
P
L
F
→ who performs the assessment?
Process Attributes
1.1 Process performance
2.1 Performance management
2.2 Work product management
3.1 Process definition
3.2 Process deployment
4.1 Process measurement
4.2 Process control
5.1 Process innovation
5.2 Process optimization
SPICE Assessment Model (2/2)
51/60
Capability Maturity Model Integrated (CMMI)
Upgrade from CMM appearing around year 2000
CMMI (1/2)
52/60
Staged assessment of the maturity of the→  entire software
development process
Continuous assessment of the→  capabilities of different Process
Areas (PA)
Sample Pas:
– Requirements Development (RD)
– Requirements Management (REQM)
– Project Monitoring and Control (PMC)
– Project Planning (PP)
– Process and Product Quality Assurance (PPQA)
– Quantitative Project Management (QPM)
– Risk Management (RSKM)
– Supplier Agreement Management (SAM)
– Technical Solution (TS)
– Validation (VAL)
– ...
CMMI (2/2)
53/60
●
Defining the measurement construct ISO/IEC 15939:2007
defines the
measurement process
for software systems
engineering
→  was discussed in the
lecture about software
metrics & measurement
ISO/IEC 15939 Example
54/60
There are around 40 Agile Maturity models that sometimes adapt SPICE/CMMI
levels/processes – each one uses different naming for levels:
T. Schweigert, D. Vohwinkel, M. Korsaa, R. Nevalainen, and M. Biro, “Agile Maturity Model: A Synopsis as a First Step to Synthesis,” in Systems, Software and Services
Process Improvement, F. McCaffery, R. V. O’Connor, and R. Messnarz, Eds. Springer Berlin Heidelberg, 2013, pp. 214–227.
Level1 Level2 Level3 Level4 Level5
●
Rhetorical stage
●
Team level
maturity
●
Neutral or Chaotic
●
Emergent
●
Engineering Best
Practices
●
Introductory
●
Collaborative
●
Dormant
●
No Agile
●
Waterfall
●
Non-Agile
●
Core Agile
Development
●
Adherence to Agile
Principles
●
Getting Started
●
Improvising
●
Certified stage
●
Department Level
Maturity
●
Collaborative
●
Continuous
Practices at
Component Level
●
Learn
●
Novice
●
Evolutionary
●
Speed
●
Early Adoption
●
Forming
●
Minimum
●
Discipline Agile
Delivery
●
Repeteable
Process across the
organization
●
Scrum at project
level
●
Practicing
●
Plausible stage
●
Business Level
Maturity
●
Operating
(consistent
exhibition of
competence)
●
Cross component
continuous
integration
●
Leverage
●
Intermediate
●
Effective
●
Reactive
●
Self Service
●
Agile
●
Consolidated
●
Agility at scale
●
Scalability –
SCRUM of
SCRUMS
●
Respectable stage
●
Project
Management Level
Maturity
●
Adaptive
●
Cross Journey
Continuous
integration
●
Advanced
●
Adaptive
●
Responsive
●
The Lake effect
●
Performing
●
Items on the right
●
SCRUM at
Enterprise Level
●
Governed
●
Measured stage
●
Management Level
Maturity
●
Innovating
●
On Demand Just in
Time Release
●
Optimise
●
Insane
●
Ambient
●
Scaling
●
Coexistence with
non-agile
●
Enterprise
transformation
●
Matured
Agile Process Maturity (1/2)
55/60
Some approaches suggest even to reuse SPICE Process definitions, example:
T. Schweigert, M. Ekssir-Monfared, and M. Ofner, “An Agile Management Process Group for TestSPICE®,” in Systems, Software and Services Process Improvement, F.
McCaffery, R. V. O’Connor, and R. Messnarz, Eds. Springer Berlin Heidelberg, 2013, pp. 228–236.
Agile Process Maturity (2/2)
56/60
Source: http://www.agigante.it/different-levels-of-agile-in-a-company/
Level1 – ad hoc
Agile
Level2 – doing
agile
Level3 – being
agile
Level4 – thinking
agile
Level5 – culturally
agile
Agile is not yet used
or agile practices are
used sporadically
Teams start to exhibit
some agile habits
Lean portfolio
management
Communities of
practice support agile
habits
Lean and agile are
part of organizational
culture
Variable quality Consistency across
teams is still variable
Mature embodiment
of essential
characteristics and
behaviour of agile
Successful use of
agile at scale
Perfecting waste
reduction, smooth
flow of delivery
Predominantly manual
testing
Some knowledge
sharing activities
under way
Disciplined Agile
delivery processes
and practices with
continual
improvement and
repeatable results
Success even with
teams in multiple
geographies
Sustainable pace of
innovation
Very little crossproject
knowledge
and collaboration
Use of agile tools and
practices become
commonplace
Respect for people
and continuous
improvement
Measurement
systems in place keep
track of business
value delivered
Continuous
organizational
learning and
optimization of the
work process and the
work products
Success achieved
primarily through
heroic individual
efforts
Solution quality
improves
Appropriate agile
governance
Autonomation:
automation with a
human touch
Standard work is
defined
Agile Maturity Matrix
57/60
●
One of the existing models is a simplified version of CMMI taking into
account the distributed nature of Open Source development
●
Trustworthy elements (TWE) are micro-characteristics that allow to define
maturity levels
Source: http://qualipso.icmc.usp.br/OMM/
PDOC – Product Documentation
STD – Use of Established and Widespread Standards
QTP – Quality of Test Plan
LCS – Licenses
ENV – Technical Environment
DFCT – Number of Commits and Bug Reports
MST – Maintainability and Stability
CM – Configuration Management
PP1 – Project Planning Part 1
REQM – Requirements Management
RDMP1 – Availability and Use of a (product) roadmap
RDMP2 – Availability and Use of a (product) roadmap
STK – Relationship between Stakeholders
PP2 – Project Planning Part 2
PMC – Project Monitoring and Control
TST1 – Test Part 1
DSN1 – Design Part 1
PPQA – Process and Product Quality Assurance
PI – Product Integration
RSKM – Risk Management
TST2 – Test Part 2
DSN2 – Design 2
RASM – Results of third party assessment
REP – Reputation
CONT – Contribution to FLOSS Product from SW Companies
Open Source Maturity Model (OMM)
58/60
●
Each TWE is divided in goals (G) practices (P) checklists (C) metrics (M)→  →  → 
●
Example: PDOC (Product Documentation) G1.Provide HQ documentation P1.Create Development→  → 
Documentation C1. is detailed architectural design documentation available? (yes/no) C2…...→  → 
●
The Goal Question Metric approach is used to define the elements of the current
OMM based on the TWEs
●
All levels of the GQM are aggregated according to following rating calculation:
R(Pi)=
∑ Mi
count (M )
R(Gi)=
∑ Pi
count (P)
R(TWEi)=
∑ Gi
count (G)
No weighting of metrics, Maturity Level calculated
by using practices (not TWEs)
R(ML)=
∑ Pi
max∑ Pi
P=practice, G=goal, ML=Maturity Level, R=rating
Open Source Maturity Model (OMM)
59/60
For the part about increasing returns, path dependency, if you are interested :)
[1] Arthur, W. Brian (1989). Competing Technologies, Increasing Returns, and Lock-In by Historical Events, 97
Economic Journal 642-65.
[2] Farrell, Joseph and Garth Saloner (1985). Standardization, Compatibility, And Innovation, 16 Rand Journal
70-83.
[3] Katz, M. L., & Shapiro,C. (1985). Network Externalities, Competition, and Compatibility. The American
Economic Review, 75(3), 424-440.
[4] Katz, M. L., & Shapiro, C. (1986). Technology Adoption in the Presence of Network Externalities. Journal of
Political Economy 822-841.
[5] Liebowitz, S. J. and Stephen E. Margolis (1990). The Fable of the Keys, Journal of Law and Economics, 33:1,
1-26.
[6] Liebowitz, S. J. and Stephen E. Margolis (1994). Path Dependency, Lock-In, and History, working paper,
1994b.
[7] Liebowitz, S. J. and Stephen E. Margolis (1994). Network Externality: An Uncommon Tragedy, 8 Journal of
Economic Perspectives 133-50.
[8] Economides, N. (1996). The Economics of Networks, International Journal of Industrial Organization, vol. 14,
no. 6, pp. 673-699
[9] Paul A. David (2000), Path dependence, its critics and the quest for ‘historical economics, in P. Garrouste and
S. Ioannides (eds), Evolution and Path Dependence in Economic Ideas: Past and Present, Edward Elgar Publishing,
Cheltenham, England.
[10] Shapiro, C. e Varian, H.R. (1999). Information Rules: A Strategic Guide to the Network Economy, Harvard
Business School Press.
[11] Windrum, P., (2003). Unlocking a lock-in: towards a model of technological succession, in Applied
Evolutionary Economics: New Empirical Methods and Simulation Techniques, P.P. Saviotti (ed.), Cheltenham:
Edward Elgar.
References
60/60
Other references mentioned in the slides, plus:
●
Bourque, P., & Fairley, R. E. (2014). Guide to the Software Engineering
Body of Knowledge (SWEBOK (R)): Version 3.0. IEEE Computer Society
Press.
References