Presentation by Tom Mens at PackagingCon 2021 on Wednesday 10 November 2021. Abstract: Semantic versioning (semver) is a commonly accepted open source practice, used by many package management systems to inform whether new package releases introduce possibly backward incompatible changes. Maintainers depending on such packages can use this practice to reduce the risk of breaking changes in their own packages by specifying version constraints on their dependencies. Depending on the amount of control a package maintainer desires to assert over her package dependencies, these constraints can range from very permissive to very restrictive. We empirically compared the evolution of semver compliance in four package management systems: Cargo, npm, Packagist and Rubygems. We discuss to what extent ecosystem-specific characteristics influence the degree of semver compliance, and we suggest to develop tools adopting the wisdom of the crowds to help package maintainers decide which type of version constraints they should impose on their dependencies. We also studied to which extent the packages distributed by these package managers are still using a 0.y.z release, suggesting less stable and immature packages. We explore the effect of such "major zero" packages on semantic versioning adoption. Our findings shed insight in some important differences between package managers with respect to package versioning policies. Our empirical results have been published in two peer-reviewed academic journals: the IEEE Transactions in Software Engineering (https://doi.org/10.1109/TSE.2019.2918315) and Elsevier Science of Computer Programming (https://doi.org/10.1016/j.scico.2021.102656). Achknowledgments: Research conducted in the context of the SECOASSIST "Excellence of Science" Research Project.

1. Comparing semantic versioning practices
in Cargo, npm, Packagist and Rubygems
@tom_mens
www.linkedin.com/in/tommens
www.slideshare.net/TomMens

2. SECO-ASSIST
Belgian
Research Project
"Excellence of Science”
secoassist.github.io
@secoassist

3. Empirical research
on packaging ecosystems
On the impact of security vulnerabilities in the npm package dependency network
Decan, Mens, Constantinou – MSR 2018 – https://doi.org/10.1145/3196398.3196401
On the evolution of technical lag in the npm package dependency network
Decan, Mens, Constantinou – ICSME 2018 – https://doi.org/10.1109/ICSME.2018.00050
An empirical comparison of dependency network evolution in seven software packaging ecosystems
Decan, Mens, Grosjean – Empirical Software Engineering Journal 2019 – https://doi.org/10.1007/s10664-017-9589-y
What do package dependencies tell us about semantic versioning?
Decan, Mens – IEEE Transactions on Software Engineering 2019 – https://doi.org/10.1109/TSE.2019.2918315
Lost in zero space – An empirical comparison of 0.y.z releases in software packaging distributions
Decan, Mens – Science of Computer Programming 2021 – https://doi.org/10.1016/j.scico.2021.102656
Back to the past – Analysing backporting practices in package dependency networks
Decan, Mens, Zerouali, De Roover – IEEE Trans. Software Engineering 2021 – https://doi.org/10.1109/TSE.2021.3112204

4. Semantic
Versioning
https://semver.org
major minor patch
3 9 2
Breaking
changes
Backwards
compatible
changes
Bug fixes
recommended to respect
semantic versioning

5. Dependency
Constraints
https://semver.org
major minor patch
3 9 2
Breaking
changes
Backwards
compatible
changes
Bug fixes
Most
permissive
Most
Restrictive

6. Outdated
Dependencies
• 1 out of 3 packages never update their dependency
• Outdatedness is related to the type of dependency constraint being used
Strict constraints represent
about 33% of all outdated dependencies
Outdated
runtime dependencies

7. By making dependency constraints “semver-compliant”
the proportion of outdated releases might be reduced by >17%
“What if …” analysis:
Outdated
Dependencies

8. semver
in package distributions
Different package distributions interpret
dependency constraints in different ways
More restrictive than semver
More permissive than semver
What do package dependencies tell us of semantic versioning?
A Decan, T Mens (2019) IEEE Transactions on Software Engineering
semver compliant

9. semver
in package distributions
To which extent do package distributions adhere to semver?
All considered distributions become more semver-compliant over time.
mostly semver-compliant
>16% of restrictive dependency constraints,
preventing automatic adoption of backward compatible upgrades

10. semver
in package distributions
To which extent do package distributions adhere to semver?
The RubyGems team urges gem developers to follow the
semantic versioning standard for their gem’s versions. The
RubyGems library itself does not enforce a strict versioning
policy, but using an “irrational” policy will only be a disservice to
those in the community who use your gems.
https://guides.rubygems.org/patterns/

11. Cargo package
serde
Packagist package
mage2pro/core
npm package
react-scripts
Rubygems package
rails
Wisdom
of the Crowds
Maintainers of dependent packages should look at how other packages
depend on a required package to decide which version constraint to use.
Distribution of dependency constraint types of dependent packages
compliant
1
permissive
50
compliant
575
restrictive
17
compliant
1
restrictive
56
compliant
288
permissive
506
restrictive
203

12. Summary
so far
Semver reduces outdatedness
Distribution-specific semver rules are confusing
Package distributions become more semver-compliant over time
Maintainers of dependent packages could use “wisdom of the crowds”
to decide which version constraint to use for their dependencies

13. What about
major version zero?
“Major version zero (0.y.z) is for initial development. Anything MAY
change at any time. The public API SHOULD NOT be considered stable.”
https://semver.org
More permissive than semver !
Constraint Cargo npm Packagist
~0.2.3 [0.2.3, 0.3.0[ [0.2.3, 0.3.0[ [0.2.3, 0.3.0[
^0.2.3 [0.2.3, 0.3.0[ [0.2.3, 0.3.0[ [0.2.3, 0.3.0[
youtu.be/b1U4YefW24Q
proportion of dependency constraints to 0.y.z
accepting at most patches or minor releases

14. What about
major version zero?
^ constraint is misleading: it behaves differently for 0.y.z releases
Constraint Cargo npm Packagist
^1.2.3 [1.2.3, 2.0.0[ [1.2.3, 2.0.0[ [1.2.3, 2.0.0[
^0.2.3 [0.2.3, 0.3.0[ [0.2.3, 0.3.0[ [0.2.3, 0.3.0[

15. only 0.y.z
only 0.y.z
only 0.y.z
only 0.y.z
both
both
both
both
only >= 1.0.0
only >= 1.0.0
only >= 1.0.0
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Cargo
RubyGems
npm
Packagist
Stuck
in zero space?
Proportion of packages having release in a given version range
A minority of 0.y.z packages
ever crosses the 1.0.0 barrier

16. 0.0.0
0.0.0
0.0.0
0.0.0
0.0.1
0.0.1
0.0.1
0.0.1
0.1.0
0.1.0
0.1.0
0.1.0
1.0.0
1.0.0
1.0.0
1.0.0
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Cargo
RubyGems
npm
Packagist
Stuck
in zero space?
Proportion of packages created in 2019, by version number
96.3%
84.9%
44.3%
35.0%
impact of
0.1.0 default

17. Monthly proportion of 0.y.z releases
Abundance
of 0.y.z releases
The release policies of Cargo and
RubyGems should be adapted
to incite package maintainers
to move out of the zero
version space sooner.

18. Can 0.y.z
releases be trusted?
FOSDEM 2021 – Lost in ZeroSpace
0.18
Would you trust depending on a
package with major version 0?
No
6%
Only if there
is no
alternative
19%
Only after
checking
41%
Sure
34%

19. Can 0.y.z
releases be trusted?
FOSDEM 2021 – Lost in ZeroSpace
0.19
Distributions of the number of dependent packages
for required 0.y.z and ≥1.0.0 packages.
Psychological 1.0.0 barrier is mostly artificial:
• Many dependent ≥1.0.0 packages rely on
0.y.z packages.
• Many 0.y.z packages
are required by ≥1.0.0 packages.
• Little practical difference between 0.y.z and
≥1.0.0 packages
Major version zero does not imply initial development.
Move out of zero space as soon as package is production-ready.
« if your software is used in production, it should probably already be 1.0.0 »
« if you have a stable API on which users have come to depend, you should be 1.0.0 »