LAB OF SOFTWARE ARCHITECTURES
AND INFORMATION SYSTEMS
FACULTY OF INFORMATICS
MASARYK UNIVERSITY, BRNO
PV260 - SOFTWARE QUALITY
SOFTWARE QUALITY MANAGEMENT PROCESS
Bruno Rossi
brossi@mail.muni.cz
"Essentially, all models are wrong,"Essentially, all models are wrong,
but some are useful"but some are useful"
George BoxGeorge Box
3-63
●
Kitchenham gave five perspectives of software quality:
1. Transcendental view
→ can be recognized but difficult to define exactly
2. User view
→ fitness for purpose
3. Manufacturing view
→ conformance to specification
4. Product view
→ from inherent product characteristics
5. Value-based view
→ depends on customer’s willingness to pay
Introduction
B. Kitchenham and S. L. Pfleeger, “Software quality: the elusive target,” IEEE Software, vol. 13, no.
1, pp. 12–21, Jan. 1996.
4-63
© 2012 Steve Easterbrook creative commons license.
Introduction
We deal with process
quality in this lecture
5-63
Introduction
●
Quality is a concept that starts with the development
process, goes on with the software product and finally to
the user with the results from software usage
6-63
●
“Software quality management (SQM) is the collection of all processes
that ensure that software products, services, and life cycle process
implementations meet organizational software quality objectives and
achieve stakeholder satisfaction” (SWEBOK 3.0)
●
SQM defines processes, process owners, requirements for the processes,
measurements of the processes and their outputs, and feedback
channels throughout the whole software life cycle.
What is Software Quality Management (SQM)
7-63
The Impact of Quality
8-63
How much “Quality” is needed? If we could quantify it, should/could we
have 100% quality in our software products?
Thinking about some of the systems that are/have been market leaders,
how was the quality of the proposed solution?
The impact of Quality
It was shown that even by starting with the same level
of customer satisfaction what really matters is the
competition with the other companies
Babich, P. (1992) “Customer Satisfaction: How Good is Good Enough?” Quality Progress. https://docuri.com/download/customer-satisfaction_59c1d667f581710b2866ba80_pdf
A
B C
dissatisfied
A
dissatisfied
C
dissatisfied C
dissatisfied B
dissatisfied
A
dissatisfied
B
satisfied A
satisfied B satisfied C
This implies that you need to do better than your
competitors in terms of quality
9-63
●
What makes software different
– Really big economies of scale
→ additional physical inputs for production result in a non-proportional
increase in output with a decrease in average costs
– Increasing returns
→ “tendency for that which is ahead to get further ahead, for that
which loses advantage to lose further advantage” (W.B. Arthur)
– Network Externalities
→ the value of goods to users increases as more people adopt them
– High initial costs
→ software is complex to design and to deliver to the market
– Switching costs
→ switching to other software might be costly (e.g. training to
re-do, change of infrastructure)
The impact of Quality
10-63
Network Externalities: for some types of goods, the value of a
good increases with the number other people adopting the same
good
●
“These effects arise both because the ability to communicate and share data with
others will be greater, and because it is more likely that complementary hardware,
software, and wetware (i.e., brain cells) will be available, when there is a large
base of users of the software”, Katz & Shapiro
Network Externalities
Network Externalities
Path Dependence
Lock-in
11-63
●
Path Dependence: in common knowledge means 'history matters'
●
But there is more.. If we consider historical paths, not only the path
depends on previous events but also produces a self-reinforcing
mechanism that leads to the reinforcement of the path selected. In this
way, switching to another path will become at every step more costly
●
In general, due to increasing returns, a phenomenon can assume
contagious effects
Path Dependence
12-63
●
So are we destined to keep the current status in the future
(e.g. using Facebook, Google, etc..)?
●
One nice answer comes from Christensen (2013) that identified
two types of technologies:
– Sustaining – an innovation that improves a product in either
expected or unexpected way, but does not lead to a paradigm
shift
– Disruptive – an innovation that can potentially create a new
market and constitutes a paradigm shift (e.g. facebook creating
the whole social networking idea, or touch screens for the whole
mobile phones industry)
●
Disruptive innovations are a way to change from the status quo
→ can you name some companies that were market leaders before a
disruptive technology appeared?
Sustaining and Disruptive Innovations (1/3)
Christensen, C. (2013). The innovator's dilemma: when new technologies cause great firms to fail.
Harvard Business Review Press.
13-63
●
Disruptive technologies: initially the quality might be lower
than current technologies, but will catch-up quickly
Sustaining and Disruptive Innovations (2/3)
Christensen, C. (2013). The innovator's dilemma: when new technologies cause great firms to fail. Harvard
Business Review Press. (img source: https://commons.wikimedia.org/wiki/File:Disruptivetechnology.png)
14-63
●
Gartner's Hype Cycle (GHC) for Emerging Technologies →
maturity of technologies in a domain
Sustaining and Disruptive Innovations (3/3)
← 2013 GHC
Do you see any
technology that
maintained the
“hype”?
15-63
Q&A
16-63
a) Can you name one disruptive innovation either in
Software or in the IT world that impressed you particularly in
the last few years?
b) Thinking in terms of 5-10 years from now, what do you
think will be a disruptive technology in the software/IT
world?
General Questions
17-63
SQM Categories
18-63
●
According to ISO/IEC 12207/15288:2007
What is a process?
Processes require a purpose and outcome. All processes
have at least one activity
Activities are constructs for grouping together related tasks
A task is a detailed arrangement for the implementation of
a process. It can be a requirement (“shall”), a
recommendation (“should”) or a permission (“may”)
Notes are used to explain better the intent or mechanism
of a process
19-63
SQM comprises four subcategories
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
A → All Planning for SWQ
B → Assess the adequacy of
software processes
C → Compliance to standards
established for the project for
products
D → Activities to improve
software processes
20-63
SQA vs SQC – What's the difference?
Source: http://softwaretestingfundamentals.com/sqa-vs-sqc/
21-63
●
A quality plan defines how an organization will reach the quality
objectives
●
Usually covers
– Quality objectives and goals
– Quality management scope
– Organisation & responsibilities
– Resource requirements
– Cost benefit analysis
– Activities and deliverables
– Schedule
– Risk analysis
A. Software Quality Planning
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
22-63
●
Quality at the organization and project Levels
A. Software Quality Planning
Source: http://www.chambers.com.au/glossary/quality_planning.php
Project /
Product
development
Project /
Product
development
Quality
management
Group
Process Group
SQMS
Development
Process
Improvement
Plan
Project Manager /
Quality
Representative
Project /
Product
development
Project
Quality Plan
Quality Management System
standards
procedures
process
description
QA objectives
& metrics
Goals
Resources
SQMS tailoring
SQMS = Software Quality Management System
CompanyLevelProjectLevel
23-63
●
SQA means monitoring constantly the software engineering
process to ensure that the approaches/methods/processes applied
lead to quality within the project
B. Software Quality Assurance
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
24-63
●
IEEE Std 730-2014 – Outline for software quality planning:
B. Software Quality Assurance Planning
Heimann, D. I. (2014). An Introduction to the New IEEE 730 Standard on Software Quality Assurance, SQP Vol 16, N.3
1. Purpose & Scope
2. Definitions & acronyms
3. Reference documents
4. SQA Plan Overview:
4.1 Organization & independence
4.2 Software Product Risk
4.3 Tools
4.4 Standards, practices and conventions
4.5 effort, resource, schedules
5. Activities, Outcomes and tasks:
5.1 Product Assurance:
5.1.1 Evaluate plans for conformance
5.1.2 Evaluate product for conformance
5.1.3 Evaluate product for acceptability
5.1.4 Evaluate product life cycle support for
conformance
5.1.5 Measure products
5.2 Process assurance:
5.2.1 Evaluate life cycle support for conformance
5.2.2 Evaluate environments for conformance
5.2.3 Evaluate subcontractor processes for
conformance
5.2.4 Measure processes
5.2.5 Assess staff skills & knowledge
6. Additional Considerations
6.1 Contract review
6.2 Quality Measurement
6.3 Waiver and deviations
6.4 Task repetition
6.5 Risks in performing SQA
6.6 Communication strategy
6.7 Conformance process
4. SQA Records:
7.1 Analyze, identify, collect, file, maintain,
dispose
7.2 Availability of records
25-63
IEEE Std 730-2014 – how is Agile considered?
●
Agile → the product backlog is the contract, the standards help in
defining the role of the backlog as contract
●
The SQA product part in IEEE730 can be used to defined the “done”
criteria
●
Non-conformances to standards are inserted in the backlog and
addressed in sprints in which are scheduled
●
Acceptance is a continuous process in Agile
●
IEEE730 contains an appendix with details about Agile adoption of the
standard
B. Software Quality Assurance Planning
Heimann, D. I. (2014). An Introduction to the New IEEE 730 Standard on Software Quality Assurance, SQP Vol 16, N.3
26-63
●
IEEE Std 730 format and content of a→ software quality assurance plan
●
IEEE Std 1061 describes a methodology—spanning the life cycle—for→
establishing quality requirements and for identifying, implementing, and
validating the corresponding measures.
●
IEEE Std 1465 (withdrawn standard) describes quality requirements→
specifically suitable for software "packages". It is expected to be replaced
by an IEEE adoption of ISO/IEC 25051
B. Summary of IEEE Stds related to SWQA
27-63
●
SQC means to constantly monitor the software engineering
process/product to check for conformance to applied standards
(e.g. CMMI) or produced artifacts
●
Some examples of methods
→ The Goal Question Metrics Approach (seen on the measurement lecture)
→ The Plan-Do-Check-Act method
C. Software Quality Control
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
28-63
●
Improve process effectiveness, efficiency and other characteristics
with the aim to improve software quality
●
Very often software process improvement practices are
embedded within the process (e.g. capability models)
●
Some methods:
→ Capability Maturity Model (CMM) and Capability Maturity
Model Integration (CMMI)
→ ISO/IEC 15504 (SPICE)
→ ISO 9001 Specification (seen during PA017 SEII)
→ Personal Software Process (PSP)
and Team Software Process (TSP)
D. Software Process Improvement
A. Software quality planning
(SQP)
which quality standards are to be used,
defining specific quality goals, and
estimating the effort and schedule of
software quality activities
B. Software quality
assurance(SQA)
define and assess the adequacy of software
processes to provide evidence that
establishes confidence that the software
processes are appropriate for and produce
software products of suitable quality for
their intended purposes
D. Software process
improvement (SPI)
The activities in this category seek to
improve process effectiveness,
efficiency, and other characteristics
with the ultimate goal of improving
software quality
C. Software quality control
(SQC)
activities examine specific project artifacts
(documents and executables) to determine
whether they comply with standards
established for the project (including
requirements, constraints, designs,
contracts, and plans)
29-63
Some Historical
Models for Software
Quality Management
30-63
●
What is the name of the simplest quality process
management practice in your opinion?
→ Actually, it involves no process
Simplest Quality Management Form
31-63
●
Cowboy Coders write code according to their rules
●
Some sentences you might have heard:
“If possible, the customer should only see the final versions of the
product. It is important to minimize the contact with the customer so
time is not wasted”
“The code is mine and none is allowed to touch it!”
“I do not need any analysis, design nor documentation”
“Even if it is broken, do not touch it! Try to hide it!”
“People who need comments in order to
understand my code are too dumb to be
working with me “
Cowboy Coding
Image: https://www.cs.utexas.edu/blog/cowboy-rides-away-now
→ See http://c2.com/cgi/wiki?CowboyCoder
32-63
●
The Personal Software Process (PSP) is a disciplined software development process
that works at the individual level
Personal Software Process (PSP)
PSP0
Defining &
using
processes
PSP1
Planning &
Tracking
PSP2
Quality
Management
TSP
Team
Development
Estabilish a measured
performance baseline
Practice size and effort
estimation
Practice defect
management and improve
design practices
Defined around 1995: Humphrey, Watts
S. A discipline for software engineering.
Addison-Wesley Longman Publishing
Co., Inc., 1995.
33-63
PSP0 – First Level
Process
scripts
PSP0 Process
Planning
Development
Postmortem
Design
Code
Compile
Test
Requirements
Time & Defect
Logs
Plan Summary
Finished Product
Scripts provide
the steps for
all processes
Templates
helpful when
following the
process
34-63
●
Emphasizes making accurate and precise size measurements
●
Incorporates measuring the size of the programs produced
●
Accounts for various types of LOC in the programs produced
●
Begins to look at process improvement
●
The following elements are added
– Estimating and reporting software size
– Use of a coding standard
– Recording process problems and improvement ideas
PSP0.1 – Improvement
35-63
●
All is based on forms, scripts and logs
PSP0.1 – Improvement
W. S. Humphrey, “A Discipline for Software Engineering”, 1995
Table C3 PSP0.1 Project Plan Summary Example
Student Student 11 Date 2/1/94
Program Object LOC Counter Program # 3A
Instructor Humphrey Language C
Program Size (LOC) Plan Actual To Date
Base(B) 87
(Measured)
Deleted (D) 0
(Counted)
Modified (M) 6
(Counted)
Added (A) 113
(T-B+D-R)
Reused (R) 0 0
(Counted)
Total New & Changed (N) 90 119 315
(A+M)
Total LOC (T) 200 396
(Measured)
Total New Reuse 0 0
Time in Phase (min.) Plan Actual To Date To Date %
Planning 10 11 36 6.4
Design 25 21 63 11.2
Code 75 97 249 44.2
Compile 20 4 35 6.2
Test 45 39 105 18.7
Postmortem 20 33 75 13.3
Total 195 205 563 100.0
Defects Injected Actual To Date To Date %
Planning 0 0 0
Design 1 3 11.5
Code 8 23 88.5
Compile 0 0 0
Test 0 0 0
Total Development 9 26 100.0
Defects Removed Actual To Date To Date
%
Planning 0 0 0
Design 0 0 0
Code 0 0 0
Compile 2 13 50.0
Test 7 13 50.0
Total Development 9 26 100.0
After Development 0 0
36-63
●
PSP1 introduces the concept of
software effort estimation and
the usage of historical data
●
Using the PROBE (PROxy-Based
Estimating) size estimating
method
●
Using linear regression, PROBE
size estimation, regression
analysis is based on historical
estimated object LOC (the x
data) and actual new and
changed LOC (the y data)
PSP1 – Second Level
Estimate Objects (proxies)
Nr. of
methods
Start with the conceptual design
Object
type
Relative
size
Reuse
categories
Estimate new and changed LOCs
Estimate development time
Calculate prediction interval
Planning estimates
W. S. Humphrey, “A Discipline for Software Engineering”, 1995
37-63
●
PSP2 introduces design and code reviews methods for
evaluating and improving the quality of your reviews
●
There are three new process elements
– PSP2 project plan summary
– PSP2 design review checklist
– PSP2 code review checklist
PSP2 – Third Level
38-63
●
The idea behind PSP is that it should lead to more team-aware
processes once developers have tried a self-disciplined approach
Personal Software Process (PSP)
PSP Skill-building
Personal Plans
Planning Methods
Earned Value
Process data
Quality Measures
Defined Processes
TSP Team-building TSP Team-working
Engineering
Disciplines
Team
Disciplines
Management
Disciplines
Committment
Aggressive plans
Quality ownership
Project goals
Plan ownership
Plan detail
Team roles
Team resources
Quality priority
Cost of quality
Follow the process
Review status
Review quality
Communication
Change management
Integrated
Product
Teams
→ see hackystat
39-63
●
A measure of Software Quality developed at Motorola
●
The focus of Six Sigma is on eliminating defects, that is everything that is
outside from the customers specifications
●
No more than +/- six times the standard deviation from the process
mean → 3,4 Defects Per Million datapoints
●
Data is a key to understand the underlying processes and take decisions
Six Sigma
Sigma Level Defects x 1M
2 308.537
3 66.807
4 6.210
5 233
6 3,4
Defined in the ‘80s in industry but only in
90’s adopted widely, later on adopted by
software engineering community.
σ=
√∑(xi−x)
n−1
40-63
Six Sigma - Process
Define Measure Analyze Improve Control
Who are the
customers and the
needs
How is the process
defined and how
are the defects
measured
What are the most
important causes of
the defects
How can the
causes of the
defects be
eliminated
What actions are
needed to sustain
improvement
In Six Sigma, a defect is defined as “Any product, service, or process
variation which prevents meeting the needs of the customer and/or
which adds cost, whether or not it is detected”
Key aspect: understand the relation between depend variables (Y,
defects) and independent variables (X, causes)
Y =f ()
Y =f (x1. x2, x3, ..., xn)
41-63
Six Sigma - Process
Define Measure Analyze Improve Control
●Define Project
scope
●Estailish formal
project
●Identify needed
data
●Obtain data set
●Evaluate data
quality
●Summarize &
baseline data
●Explore data
●Characterize
process & problem
●Update
improvement
project scope &
scale
●Identify possible
solutions
●Select solution
●Implement (pilot as
needed)
●Evaluate
●Define control
method
●Implement
●Document
42-63
Six Sigma - Tools
Define Measure Analyze Improve Control
● Benchmark
● Contract/charter
● Kano Model
● Voice of the
customer
● Voice of the
Business
● Quality
Function
Deployment
(QFD)
● GQ(I)M and
indicator
templates
● Data collection
methods
● Measurement
System
Evaluation
● Cause & Effect
Diagram/Matrix
● Failure models
& effects
analysis
● Statistical
inference
● Reliability
Analysis
● Root Cause
Analysis
● Hypothesis Test
● Design of
experiments
● Modeling
● ANOVA
● Tolerancing
● Robust Design
● System
Thinking
● Decision & Risk
Analysis
● Performance
Analysis Model
● Statistical
Controls:
● Control charts
● Time series
methods
● Non-Statistical
controls:
● Procedural
adherence
● Performance
Management
● Preventive
measures
Basic Tools (Histogram, scatter plots, run charts, pareto charts, cause & effect diagram, Control chart,
descriptive statistics), baseline process flow map, project management, management by fact, sampling
techniques, survey methods, defect metrics
43-63
Software Process Maturity Models
44-63
●
As defined in ISO/IEC 15504-2 (SPICE)
Process Maturity Levels
→ The process is not implemented or fails to achieve
the purpose
→ The process is achieving the purpose
→ The process is now running in a managed way
(planned, monitored, adjusted) – work products are
established, controlled and maintained
→ The managed process is now implemented using a
defined process capable of achieving process
outcomes
→ The established process now operates within
defined limits to achieve its process outcomes
→ The predictable process is continuously improved
to meet relevant current and projected business goals
Capability
Level
Process Capability
0 Incomplete Process
1 Performed Process
2 Managed Process
3 Established Process
4 Predictable Process
5 Optimizing Process
45-63
●
As defined in ISO/IEC 15504-2 (SPICE)
Process Maturity Levels & Attributes
46-63
SPICE - Overall
ENG.1
Engineering
Customer-
Relationship
Support
ManagementManagement Organisation Support
ENG.1
ENG.1
CUS.1
ENG.1
MAN.1
ENG.1
ORG.1
ENG.1
SUP.1
ENG.1
ENG.1.BP1
ENG.1
CUS.1.BP1
ENG.1
MAN.1.BP1
ENG.1
ORG.1.BP1
ENG.1
SUP.1.BP1
ENG.1
ENG.1.WP1
ENG.1
CUS.1.WP1
ENG.1
MAN.1.WP1
ENG.1
ORG.1.WP1
ENG.1
SUP.1.WP1
5 Categories
24 Processes
201 Base
Practices
109 Work
Products
47-63
Example Process Definition (1/2)
5 Process
Categories
(SUP=Support)
48-63
Example Process Definition (2/2)
Overall 24 processes
are specified
49-63
●
SPICE is a two-dimensional level model
– Processes and categories on one side (Process Dimension)
includes Base practices, work products, characteristics→
→ Does the process reach its goals?
– Capability of processes on the other side (Capability
Dimension) includes levels, process attributes,→
management practices → How well is a specific goal met?
Process Maturity Levels & Attributes
50-63
Process Maturity Levels & Attributes
Assessment Model
Process Dimension Capability Dimension
Process categories (5)
Processes (24)
Capability Levels (6)
Process Attributes (9)
Base Practices (201)
Work Products (109)
Management Practices (33)
Resources & Infrastructure
characteristics
Indicators of process
performance
Indicators of process
capability
51-63
Process Maturity Levels & Attributes
Assessment Model
Process Dimension Capability Dimension
Process categories (5)
Processes (24)
Capability Levels (6)
Process Attributes (9)
Base Practices (201)
Work Products (109)
Management Practices (33)
Resources & Infrastructure
characteristics
Indicators of process
performance
Indicators of process
capability
Assessment Scale
● up to 15% – N (Not performed/achieved)
● > 15% to 50% – P (Partial)
● > 50 to 85% - L (Large)
● > 85% - F (Full performance / achievement)
PA1.1 PA2.1 PA2.2 PA3.1 PA3.2 PA4.1 PA4.2 PA5.1 PA5.2
ENG.1
ENG.2
MAN.2
ORG.1
N
P
L
F
→ who performs the assessment?
Process Attributes
1.1 Process performance
2.1 Performance management
2.2 Work product management
3.1 Process definition
3.2 Process deployment
4.1 Process measurement
4.2 Process control
5.1 Process innovation
5.2 Process optimization
52-63
Capability Maturity Model Integrated
Upgrade from CMM appearing around year 2000
CMMI
53-63
Staged assessment of the maturity of the→ entire software process
Continuous assessment of the→ capabilities of different Process
Areas (PA)
Sample Pas:
– Requirements Development (RD)
– Requirements Management (REQM)
– Project Monitoring and Control (PMC)
– Project Planning (PP)
– Process and Product Quality Assurance (PPQA)
– Quantitative Project Management (QPM)
– Risk Management (RSKM)
– Supplier Agreement Management (SAM)
– Technical Solution (TS)
– Validation (VAL)
– ...
CMMI
54-63
Ragaisis, S., Peldzius, S., & Simenas, J. (2010). Mapping CMMI-DEV Maturity Levels toISO/IEC 15504 Capability Profiles. Assessment, 23, 24.
Mapping CMMI to ISO/IEC 15504 (SPICE)
55-63
Ragaisis, S., Peldzius, S., & Simenas, J. (2010). Mapping CMMI-DEV Maturity Levels to ISO/IEC 15504 Capability Profiles. Assessment, 23, 24.
Mapping CMMI to ISO/IEC 15504 (SPICE)
…........................................................................................................
This was the
sample process
we saw some
slides ago
Legenda: up to 15 % – N (Not performed/achieved), > 15 % to 50 % – P (Partial), > 50 to 85
% – L (Large), and F (Full performance / achievement) > 85 % - “ML2”- “ML5” maturity
levels in CMM-i - “CL1”-“CL5” are capability levels in ISO/IEC 15504
56-63
●
Defining the measurement construct
ISO/IEC 15939 - Example
ISO/IEC 15939:2007
defines the
measurement process
for software systems
engineering
→ was discussed in the
lecture about software
metrics & measurement
57-63
There are around 40 Agile Maturity models that sometimes adapt SPICE/CMMI
levels/processes – each one uses different naming for levels:
Agile Process Maturity
T. Schweigert, D. Vohwinkel, M. Korsaa, R. Nevalainen, and M. Biro, “Agile Maturity Model: A Synopsis as a First Step to Synthesis,” in Systems, Software and Services
Process Improvement, F. McCaffery, R. V. O’Connor, and R. Messnarz, Eds. Springer Berlin Heidelberg, 2013, pp. 214–227.
Level1 Level2 Level3 Level4 Level5
● Rhetorical stage
● Team level
maturity
● Neutral or Chaotic
● Emergent
● Engineering Best
Practices
● Introductory
● Collaborative
● Dormant
● No Agile
● Waterfall
● Non-Agile
● Core Agile
Development
● Adherence to Agile
Principles
● Getting Started
● Improvising
● Certified stage
● Department Level
Maturity
● Collaborative
● Continuous
Practices at
Component Level
● Learn
● Novice
● Evolutionary
● Speed
● Early Adoption
● Forming
● Minimum
● Discipline Agile
Delivery
● Repeteable
Process across the
organization
● Scrum at project
level
● Practicing
● Plausible stage
● Business Level
Maturity
● Operating
(consistent
exhibition of
competence)
● Cross component
continuous
integration
● Leverage
● Intermediate
● Effective
● Reactive
● Self Service
● Agile
● Consolidated
● Agility at scale
● Scalability –
SCRUM of
SCRUMS
● Respectable stage
● Project
Management Level
Maturity
● Adaptive
● Cross Journey
Continuous
integration
● Advanced
● Adaptive
● Responsive
● The Lake effect
● Performing
● Items on the right
● SCRUM at
Enterprise Level
● Governed
● Measured stage
● Management Level
Maturity
● Innovating
● On Demand Just in
Time Release
● Optimise
● Insane
● Ambient
● Scaling
● Coexistence with
non-agile
● Enterprise
transformation
● Matured
58-63
Some approaches suggest even to reuse SPICE Process definitions, example:
Agile Process Maturity
T. Schweigert, M. Ekssir-Monfared, and M. Ofner, “An Agile Management Process Group for TestSPICE®,” in Systems, Software and Services Process Improvement, F.
McCaffery, R. V. O’Connor, and R. Messnarz, Eds. Springer Berlin Heidelberg, 2013, pp. 228–236.
59-63
Source: http://www.agigante.it/different-levels-of-agile-in-a-company/
Agile Maturity Matrix
Level1 – ad hoc
Agile
Level2 – doing
agile
Level3 – being
agile
Level4 – thinking
agile
Level5 – culturally
agile
Agile is not yet used
or agile practices are
used sporadically
Teams start to exhibit
some agile habits
Lean portfolio
management
Communities of
practice support agile
habits
Lean and agile are
part of organizational
culture
Variable quality Consistency across
teams is still variable
Mature embodiment
of essential
characteristics and
behaviour of agile
Successful use of
agile at scale
Perfecting waste
reduction, smooth
flow of delivery
Predominantly manual
testing
Some knowledge
sharing activities
under way
Disciplined Agile
delivery processes
and practices with
continual
improvement and
repeatable results
Success even with
teams in multiple
geographies
Sustainable pace of
innovation
Very little crossproject
knowledge and
collaboration
Use of agile tools and
practices become
commonplace
Respect for people
and continuous
improvement
Measurement
systems in place keep
track of business
value delivered
Continuous
organizational
learning and
optimization of the
work process and the
work products
Success achieved
primarily through
heroic individual
efforts
Solution quality
improves
Appropriate agile
governance
Autonomation:
automation with a
human touch
Standard work is
defined
60-63
●
One of the existing models is a simplified version of CMMI taking into
account the distributed nature of Open Source development
●
Trustworthy elements (TWE) are micro-characteristics that allow to define
maturity levels
Open Source Maturity Model (OMM)
Source: http://qualipso.icmc.usp.br/OMM/
PDOC – Product Documentation
STD – Use of Established and Widespread Standards
QTP – Quality of Test Plan
LCS – Licenses
ENV – Technical Environment
DFCT – Number of Commits and Bug Reports
MST – Maintainability and Stability
CM – Configuration Management
PP1 – Project Planning Part 1
REQM – Requirements Management
RDMP1 – Availability and Use of a (product) roadmap
RDMP2 – Availability and Use of a (product) roadmap
STK – Relationship between Stakeholders
PP2 – Project Planning Part 2
PMC – Project Monitoring and Control
TST1 – Test Part 1
DSN1 – Design Part 1
PPQA – Process and Product Quality Assurance
PI – Product Integration
RSKM – Risk Management
TST2 – Test Part 2
DSN2 – Design 2
RASM – Results of third party assessment
REP – Reputation
CONT – Contribution to FLOSS Product from SW Companies
61-63
●
Each TWE is divided in goals (G) practices (TWE) checklists metrics→ → →
●
Example: PDOC (Product Documentation) G1.Provide HQ documentation P1.Create Development→ →
Documentation C1. is detailed architectural design documentation available? (yes/no) C2…...→ →
●
The Goal Question Metric approach is used to define the elements of the current
OMM based on the TWEs
●
All levels of the GQM are aggregated according to following rating calculation:
Open Source Maturity Model (OMM)
R(Pi)=
∑ Mi
count (M )
R(Gi)=
∑ Pi
count (P)
R(TWEi)=
∑ Gi
count (G)
No weighting of metrics, Maturity Level calculated
by using practices (not TWEs)
R(ML)=
∑ Pi
max∑ Pi
P=practice, G=goal, ML=Maturity Level, R=rating
62-63
For the part about increasing returns, path dependency, if you are interested :)
[1] Arthur, W. Brian (1989). Competing Technologies, Increasing Returns, and Lock-In by Historical Events, 97
Economic Journal 642-65.
[2] Farrell, Joseph and Garth Saloner (1985). Standardization, Compatibility, And Innovation, 16 Rand Journal 70-
83.
[3] Katz, M. L., & Shapiro,C. (1985). Network Externalities, Competition, and Compatibility. The American
Economic Review, 75(3), 424-440.
[4] Katz, M. L., & Shapiro, C. (1986). Technology Adoption in the Presence of Network Externalities. Journal of
Political Economy 822-841.
[5] Liebowitz, S. J. and Stephen E. Margolis (1990). The Fable of the Keys, Journal of Law and Economics, 33:1, 1-
26.
[6] Liebowitz, S. J. and Stephen E. Margolis (1994). Path Dependency, Lock-In, and History, working paper, 1994b.
[7] Liebowitz, S. J. and Stephen E. Margolis (1994). Network Externality: An Uncommon Tragedy, 8 Journal of
Economic Perspectives 133-50.
[8] Economides, N. (1996). The Economics of Networks, International Journal of Industrial Organization, vol. 14,
no. 6, pp. 673-699
[9] Paul A. David (2000), Path dependence, its critics and the quest for ‘historical economics, in P. Garrouste and
S. Ioannides (eds), Evolution and Path Dependence in Economic Ideas: Past and Present, Edward Elgar Publishing,
Cheltenham, England.
[10] Shapiro, C. e Varian, H.R. (1999). Information Rules: A Strategic Guide to the Network Economy, Harvard
Business School Press.
[11] Windrum, P., (2003). Unlocking a lock-in: towards a model of technological succession, in Applied
Evolutionary Economics: New Empirical Methods and Simulation Techniques, P.P. Saviotti (ed.), Cheltenham:
Edward Elgar.
References
63-63
References mentioned in the slides, plus:
●
Bourque, P., & Fairley, R. E. (2014). Guide to the Software Engineering
Body of Knowledge (SWEBOK (R)): Version 3.0. IEEE Computer Society
Press.
References