Open Source Development Course
Continuous Integration and Delivery
Vojtˇech Trefn´y
vtrefny@redhat.com
26. 3. 2020
twitter.com/vojtechtrefny
github.com/vojtechtrefny
ɭ gitlab.com/vtrefny
Pipeline
CI/CD Pipeline
• Steps that need to be performed to test and deliver new version of the software.
• Defines what needs to be done: when, how and in what order.
• Steps can vary for every project.
• Multiple pipelines or steps can run in parallel.
1 / 28
CI/CD Pipeline
1. Testing environment
Preparation of the environment to run the
tests: deploying containers, starting VMs...
2. Static Analysis
Finding defects by analyzing the code
without running it.
3. Codestyle
Checking for violations of the language or
project style guides.
4. Build
Building the project from source.
5. Tests
Running project test suite or test suites.
6. Packaging and Deployment
Building source archives, packages or
container images.
2 / 28
Testing Environment
Testing Environment
1. Preparation of VMs/containers to
run the tests
We might want to run tests in different
environments on multiple different
distributions or architectures.
2. Installation of the test dependencies
Test dependencies are usually not covered
by the project dependencies.
3. Getting the code
Clone the PR or get the latest code from
the master branch.
3 / 28
Static Analysis
Static Analysis
• Tools that can identify potential bugs by analyzing the code without running it.
• Can detect problems not covered by the test suite – corner cases, error paths etc.
• Coverity (C/C++, Java, Python, Go. . . )1
• Cppcheck (C/C++)2
• Pylint (Python)3
• RuboCop (Ruby)4
1
https://scan.coverity.com
2
http://cppcheck.sourceforge.net/
3
https://www.pylint.org
4
https://docs.rubocop.org
4 / 28
Coverity
Error: USE AFTER FREE (CWE-825):
libblockdev-2.13/src/plugins/lvm-dbus.c:1163: freed_arg: "g_free"
frees "output".
libblockdev-2.13/src/plugins/lvm-dbus.c:1165: pass_freed_arg: Passing freed
pointer "output" as an argument to "g_set_error".
# 1163| g_free (output);
# 1164| if (ret == 0) {
# 1165|-> g_set_error (error, BD_LVM_ERROR, BD_LVM_ERROR_PARSE,
# 1166| "Failed to parse number from output: ’\%s’",
# 1167| output);
5 / 28
Code Style
Code style and style guides
• Coding conventions – naming, code lay-out, comment style. . .
• Language specific (PEP 85), project specific (Linux kernel coding style6) or
library/toolkit specific (GTK coding style7).
• Automatic checks using specific tools (pycodestyle) or (partially) by the static
analysis tools.
5
https://www.python.org/dev/peps/pep-0008/
6
https://www.kernel.org/doc/html/v5.3/process/coding-style.html
7
https://developer.gnome.org/programming-guidelines/stable/c-coding-style.html.en
6 / 28
Code style and style guides
“”
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Readability counts.
Tim Peters
The Zen of Python
7 / 28
Linux kernel coding style
https://www.kernel.org/doc/html/v4.10/process/coding-style.html
8 / 28
Python and PEP8
https:
//gist.github.com/vojtechtrefny/435737417be003873a7f94aa7d53c4d2
9 / 28
Python and PEP8
10 / 28
Documentation style
• Documentation might be checked in the same way code is.
• Similar style documents and tools for checking documentations exist (for example
PEP 2578and pydocstyle9for Python).
• In some cases wrong or missing documentation (docstrings in the code) can lead
to a broken build or missing features.
8
https://www.python.org/dev/peps/pep-0257/
9
http://www.pydocstyle.org
11 / 28
Build
Build
• Building the project, a preparation to run the test suite.
• Depends on language – mostly no-op for interpreted languages, more complicated
for compiled ones.
• Build in the CI environment can detect issues with dependencies.
• Builds on different architectures can help detect issues related to endianness or
data types sizes.
12 / 28
GNU Autotools
• Helps creating portable source packages.
• Two steps:
• configure (scans the build environment)
• make (compiles the source)
• Complicated for developers, easy for users.
• Takes care of dependency checking, dynamic linking, installation destinations etc.
13 / 28
GNU Autotools
“”
I saw a book entitled ”Die GNU Autotools”and I thought ”My feelings
exactly”. Turns out the book was in German.
Tim Martin10
10
https://twitter.com/timmartin2/status/23365017839599616
Image source: https://developer.gnome.org/anjuta-build-tutorial/stable/create-autotools.html.en
14 / 28
Tests
Tests
• Running tests that are part of the project.
• New tests should be part of every change to the codebase.
• New features require new unit and integration tests.
• Bug fixes should come with a regression test.
• For some project (like libraries) running test suites of their users might be an
option.
15 / 28
Coverage
• Code coverage (or Test coverage) represents how much of the code is covered by
the test suite.
• Usually percentual value that shows how many lines of the code were “visited” by
the test.
• Generally a check that all functions and branches are covered by the suite.
• Used as a measure of the test suite “quality”.
16 / 28
Coverage
$ coverage3 report -m
Name Stmts Miss Cover Missing
---------------------------------------
div.py 5 1 80% 3
Resulting coverage is 80 %, because 1 of 5 statements is not covered.
17 / 28
Coverage
• Automated coverage tests might be part of the CI.
• Decrease in coverage can be viewed as a reason to reject contribution to the
project.
18 / 28
Delivery and Deployment
Packaging and publishing
• Delivery – releasing new changes quickly and regularly (daily, weekly...).
• Deployment – delivery with automated push to production, without human
interaction.
• Usually after merging the changes, not for the PRs.
• Building packages, container images, ISO images. . .
• Built packages can be used for further testing (manually by the Quality Assurance
or in another CI infrastructure) or directly pushed to production or included in
testing/nightly builds of the project.
19 / 28
CI Tools
Demo
Travis CI
• Probably most popular CI service nowadays.
• Can be integrated in your projects on GitHub.
• Free for opensource projects.
• Configured using .travis.yml file in the
project
• https://travis-ci.org
20 / 28
Travis CI
21 / 28
Travis CI
22 / 28
Jenkins
• Automation system, not a “true” CI/CD tool.
• Can automatically run given tasks on a node or
set of nodes.
• Tasks can be started on time basis or triggered
by an external event (like new commit or PR
on GitHub).
• https://jenkins.io/
23 / 28
Fedora CI
• Complex CI system with task to deliver an
“Always Ready Operating System”.
• Packages are tested after every change and
“gated” if the CI pipeline fails.
• The goal is to prevent breaking the
distribution. CI will stop the broken package
before it can affect the distribution.
24 / 28
Fedora CI
25 / 28
Packit
• Tool for integrating upstream projects to
Fedora.
• RPM packages are automatically build on every
pull request.
• New releases can be automatically build and
pushed to Fedora.
26 / 28
Packit
27 / 28
Questions
Questions
Thank you for your attention.
https://github.com/crocs-muni/open-source-development-course
28 / 28