SW Measurement
PA017 SW Engineering II → Aspects of SW Development
Management
Jaroslav Ráček Josef Spurný
Faculty of Informatics, Masaryk University
November 1, 2022
Definitions
Measure
Quantitative indication of extent, amount, dimension, capacity,
or size of some attribute of a product or process
A fundamental or unit-specific term
Example: Number of errors
Metric
Quantitative measure of degree to which a system, component
or process possesses a given attribute
Metric is context-specific, i.e., it has goal / intent to measure
[something]
Metric is derived from one or more measures
A handle or guess about a given attribute
Example: Number of errors found per person hours expended
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 2 / 25
Why Measure Software?
Determine and manage quality of the current product or process
Removing unnecessary complexity / size will have impact on SW
quality
Predict qualities of a product/process
Improve quality of a product/process
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 3 / 25
Example Metrics
Frequently used metrics related to SW quality and testing:
Defects rates
Errors rates
As measured per:
whole software
functional module
development phase
time unit
Errors should be categorized by:
origin (analysis, design, coding...)
type (trivial, severe, critical...)
cost (usually as a consequence of previous categories) - as
impacting customer’s business or as resources required to remove
error/defect/bug...
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 4 / 25
Metrics Classification
Products
Explicit results of software development activities
Deliverables, documentation, by products
Example: LOC, FP, errors per LOC
Processes
Activities related to production of software
Example: FP unused in final product, Man-Days, milestones,
effort
Resources
Inputs into the software development activity
Hardware, knowledge, people
Example: Server count, certifications, staff seniority, available
technologies, experience with given technology
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 5 / 25
Process vs. Product
Process Metrics
Insights of process paradigm, software engineering tasks, work
product, or milestones
Lead to long term process improvement
Product Metrics
Assesses the state of the project
Track potential risks
Uncover problem areas
Adjust workflow or tasks
Evaluate teams’ ability to control quality
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 6 / 25
Types of Measures
Direct Measures ("hard", internal attributes)
Objective, relatively easy to measure and quantify by units
Cost, Effort, LOC, Speed, Memory
Indirect Measures ("soft", external attributes)
May be less objective, have to be inferred, assessed or judged
Functionality, Complexity, Quality, Efficiency, Reliability,
Maintainability
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 7 / 25
Size Oriented Metrics
Size of the software produced
Used in COCOMO
Examples:
Lines Of Code (LOC)
1000 Lines Of Code (KLOC)
Effort measured in person months
Errors/KLOC
Defects/KLOC
Cost/LOC
Documentation Pages/KLOC
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 8 / 25
LOC Metrics
Advantage: easy and fast to calculate
Disadvantage: dependency on programmer (seniority) &
language (high vs low level)
Illustration: compare beginner vs. senior code for certain function
Usually meaningful only when team / project (technology) is
unchanged
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 9 / 25
Function Oriented Metrics
Function Point Analysis [Albrecht ’79, ’83]
International Function Point Users Group (IFPUG)
Size-based metric, complexity is used only partially (GSP general
system characteristics)
Indirect measure
Derived using empirical relationships based on countable (direct)
measures of the software system (domain and requirements)
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 10 / 25
Revisited Function Points Computation
Function points count
= (0.65 + GSP
100 ) ∗ UFP, where
GSP = sum of general system characteristics evaluation (14
adjustment values) determined for each organization via empirical
data
UFP = unmodified function points
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 11 / 25
Using FP
As oposed to LOC, FP is programmer & language independent
Focused on delivery of functionality / added value to customer
Yet usage is similar to LOC:
Errors per FP
Defects per FP
Cost per FP
Pages of documentation per FP
FP per person month
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 12 / 25
FP & Languages
As already mentioned, we can derive LOC from FP
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 13 / 25
FP Controversy
Similarly to LOC metrics, FP has proponents and opponents:
Proponents claim that:
FP is programming language independent
FP is based on data that are more likely to be known in the early
stages of a project, making it more attractive as an estimation
approach
Opponents claim that:
FP requires some skill and "mathemagic" because the computation
is based on subjective data
Counts of the information domain can be difficult to collect
FP has no direct physical meaning ...it’s just a number
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 14 / 25
Complexity Metrics
Language and programmer dependent
Halstead’s Complexity
Is dependent on actual implementation of the code
Perceives code as a sentence and investigates its "richness"
The following measures are used:
n1 - number of unique operators
n2 - number of unique operands
N1 - total number of operators
N2 - total number of operands
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 15 / 25
Example
if (k < 2)
{
if (k > 3)
x = x*k;
}
Unique operators: if ( ) { } > < = * ;
Unique operands: k 2 3 x
n1 - 10
n2 - 4
N1 - 13
N2 - 7
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 16 / 25
Halstead’s Metrics
Length = N = N1 + N2
Vocabulary = n = n1 + n2
Estimated "optimal" (well-structured) program length
Ne = n1log2n1 + n2log2n2
Purity ratio PR = Ne
N
Ideal value is close to 1
Can show how complex / "long" / "short" code a programmer
writes
Values far from 1 can cause problems with readability of code for
others – impact on future service of SW
Average "lifespan" of programmer in one company is 5 years
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 17 / 25
McCabe’s Complexity Measures
McCabe’s metrics are based on a control flow representation of
the program
A program graph is used to depict control flow
Nodes represent processing tasks (one or more code statements)
Edges represent control flow between nodes
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 18 / 25
Flow Graph Notation
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 19 / 25
Cyclomatic Complexity
Set of independent paths through the graph
V(G) = E − N + 2, where
E is number of edges
N is number of nodes
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 20 / 25
Example
i = 0;
while (i < n-1) do
j = i + 1;
while (j < n) do
if A[i] < A [j] then
swap (A [i], A [j]);
end do;
i= i+1;
end do;
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 21 / 25
Example - Flow Graph
V(G) = E - N + 2
V(G) = 9 – 7 + 2 = 4
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 22 / 25
Cyclomatic Complexity - Meaning
V(G) is the number of (enclosed) regions/areas of the planar
graph
Number of regions increases with the number of decision paths
and loops
Experimental data shows value of V(G) should be no more then
10 (for given level of decomposition)
A quantitative measure of testing difficulty and an indication of
ultimate reliability
Already tested units are considered as one node, despite they
might have their own inner complexity
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 23 / 25
Cyclomatic Complexity & Testing
Higher cyclomatic complexity usually means lower productivity
and higher potential for bugs
Application in whitebox testing
Testing for V(G) > 10 is very difficult
Cyclomatic complexity has impact on provision of support &
service of SW
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 24 / 25
Quality Model & Metrics
Jaroslav Ráček, Josef Spurný ·SW Measurement ·November 1, 2022 25 / 25