During last LinuxCon, Bruno presented the continuous packaging approach used with a tool like project-builder.org to package upstream projects for hundreds of Linux distributions tuples in an automatic manner. Discussions happened there with the FOSSology project which wanted to benefit from this approach to produce Linux packages for their users. Both projects have since that worked jointly to make it a reality, and want to share their return of experience on this journey, benefits obtained, issues encountered and how they were fixed. After a reminder of the basics on continuous packing, the presentation will give a concrete example of what was setup using the infrastructure of the LinuxFoudation to enable the automatic creation of rpm and deb packages for FOSSology, launched during the continuous integration process already in place. A demo of the build process will also be made.

1. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Talk Title Here
Author Name, Company
Using Containers and Continuous Packaging to
Build Native Fossology Packages
Speakers
Bruno Cornec (bruno.cornec@hpe.com), Michael C. Jaeger (michael.c.jaeger@siemens.com)

2. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Overview: Contents
1. Introduction FOSSology
What is FOSSology
2. Motivation
What FOSSology needs
3. Introduction Project Builder
The ProjectBuilder Project
4. Build Native Fossology Packages
To get container running in the continuous build
5. Conclusion
Where to see it
2 Page 2

3. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Introduction FOSSology

4. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
The Problem Actually
Distributing open source software requires to
∙ Provide licenses of involved software
∙ Provide copyright statements of involved authors
∙ Provide disclaimers
∙ … and much more
You know these examples
Page 4

5. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
It is about finding licenses
∙ License texts
∙ References to licenses
∙ Written texts explaining licensing
∙ License relevant statements
Finding Licenses
Page 5

6. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
What is FOSSology?
A Web server application for license and copyright compliance of software components.
FOSSology Project
https://www.fossology.org/
∙ Published first in 2008, GPL-2.0
∙ 2015: Linux Foundation collaboration project
∙ Web server based and command line interfaces
∙ Scanning agents searching for license and
copyright relevant hits (and more …)
∙ A multi-user / multi-tenant Web UI for review
organizing clearing job
FOSSology Development
https://www.github.com/fossology/fossology
▪ Standard Web application stack:
▪ Linux, Apache 2, PostgreSQL, PHP,
▪ Web-based UI in PHP, but scanners
written in C / C++
▪ Two ways to interact:
▪ Web user interface
▪ Command line utilities
Page 6

7. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
How does FOSSology work?
See more details the Basic Workflow Description: https://www.fossology.org/get-started/basic-workflow
▪ Upload an open source package to the server
▪ Select scan agents that analyze the software
▪ Review what scanners have found
▪ Review license occurrences and correct findings if necessary
▪ Generate report output
▪ For example list of licenses or SPDX
Upload OSS
Package
Review and Adjust
(“Clearing”)
Generate
Page 7

8. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
What is the point of FOSSology?
See more details the Basic Workflow Description: https://www.fossology.org/get-started/basic-workflow
▪ Upload an open source package to the server
▪ Select scan agents that analyze the software
▪ Review what scanners have found
▪ Review license occurrences and correct findings if necessary
▪ Generate report output
▪ For example list of licenses or SPDX
Upload OSS
Package
Review and Adjust
(“Clearing”)
Generate
Page 8

9. Page 9© 2016-2017 Siemens AG, Linux Foundation - CC-BY-SA 4.0Open Source Summit Europe 2017
Using FOSSology with this Example
∙ It is natural that an OSS project reuses
available https://github.com/fossology/fossology
∙ Likely OSS from other projects is found
∙ For example, FOSSology will find 25 other
licensing relevant text occurrences in Apache
thrift
Open Source and Reuse
9
Page 9

10. Page 10© 2016-2017 Siemens AG, Linux Foundation - CC-BY-SA 4.0Open Source Summit Europe 2017

11. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Motivation

12. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
The Problem Actually
∙ ~ “creating binaries for linux is difficult” (starts at 5:40)
∙ https://www.youtube.com/watch?v=qHGTs1NSB1s
∙ Many linux distros with own package universe
∙ Different distros and different versions of these
∙ E.g. Packages dependencies on debian 8 change with
debian 9
∙ Even within Debian 8 postgresql changes ...
See Linus Torvalds
Page 12

13. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
FOSSology Demand
∙ Debian, Ubuntu, CentOS and Fedora
∙ To efficiently build packages for these
∙ = efficiently means not to have manual step for each distro
∙ also means dealing with specificities of each distro/version
(dependencies, availability of packages, …)
Support (at least) a basic set of Linux Distros
Page 13

14. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Technically
∙ Different Distros required (and their versions)
∙ Integration in the CI
∙ State-of-the-art: Docker
∙ Support of two main package building formats: RPM and Deb
It is about building Linux packages
Page 14

15. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Introduction to
Project-Builder.org

16. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Project-Builder.org goal
“Make upstream projects
life easier with regards to
packaging their software”
Page 16

17. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Project-Builder.org goal
“Make my
life easier with regards to
packaging my software”
Page 17

18. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Benefits from Continuous Packaging
● Packaging should be a project concern as well as coding, testing, installing, ....
especially for smaller projects
● Packaging as your only way of delivery (not a dream)
● Minimal overhead, maximum benefit:
● Consistancy and reproduceability for devs and users
● Distribution & deployment server integration,
● Consistency with distribution and avoids dependecy hell for consumers
● Packaging as a marketing activity for the upstream project. Easy way to extend
your user base, and improve your community relationship and is a “competitive
advantage”.
● New mantra: “Package early, package always”
● THE SOLUTION IS INDEED CONTINUOUS PACKAGING (whatever the tool)
Page 18

19. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Project-Builder.org goals
● VCS agnostic: no VCS but guys it's 21st century now, SVN, CVS, Mercurial, GIT
and GIT/SVN, SVK....
● OS agnostic: Linux: RPM, deb, ebuild, slack based, ... 150+ distro tuples made
and counting – repositories for yum, urpmi, apt. Solaris pkg.
● Build environment agnostic: local, VM (QEMU, KVM...), VE (Docker, chroot,
rpmbootstrap, rinse, mock, debootstrap...), RM (build farm)
● No project impact: preserves the md5sum of the delivered upstream sources. Can
be completely external to the upstream project.
● Avoids duplication of code and metadata
● THE SOLUTION IS INDEED CONTINUOUS PACKAGING (with project-builder.org !)
Page 19

20. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Project-Builder.org architecture
Page 20

21. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Build Native Fossology Packages
Demonstration !!

22. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Problem encountered
● fossology build issues
● project-builder.org bugs
● composer phar !
● build infrastructure
● introduction in CI toolset
Page 22

23. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Conclusion

24. © 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017