This document presents QuESo, a quality model for measuring the quality of open source software ecosystems. It consists of three dimensions: community quality, ecosystem network quality, and platform quality. The authors describe developing the model through a systematic literature review to identify relevant metrics, which were then organized into the quality characteristics and sub-characteristics. Examples of applying the QuESo model for risk analysis and assessing an open source community's health are provided. Future work could involve defining a complete quality assessment process based on the model.
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
QuESo Quality Model Measures OSS Ecosystems
1. QuESo: a Quality Model for Open Source Software EcosystemsICSOFT-EA 2014
Oscar Franco-Bedoya
ohernan@essi.upc.edu
David Ameller
dameller@essi.upc.edu
Dolors Costal
dolors@essi.upc.edu
Xavier Franch
franch@essi.upc.edu
2. 2
Related work
Method
QuESo
Application
example
Discussion
Future work
Conclusions
Outline
Overview
3. 3
0
500.000
1.000.000
1.500.000
2.000.000
2.500.000
2007
2009
2011
2013
2015
The number of open source
software projects is growing
Source: Black Duck Management webinar 2014
in IEEE OSS webcasts series
4. 4
A challenge arises
How can we measure the quality of these OSS projects?
Way to preventbad decisions
Avoidproblems
Verifythe compliance with requirements and business goals
Systematic monitoring to provide feedback
5. 5
The ecosystem metaphor is used to
understand software relationships
A systemformed by
the interactionof
a community of organisms
with their environment
Developers
Adopters
Software
Projects
Contributors
Software
products
6. 6
Software ecosystems definitions(Messerschmitt and Szyperski, 2005)
Business perspective
“Setof actorsfunctioning as a
unit and
interacting with a
shared market for
software and services.”
(Jansen and Cusumano, 2013)
Technical perspective
“ Acollection of
software projects which are
developedandevolve together
in the same environment.”
(Lunguet al., 2009).
7. 7
We use both definitions in our work
In order to assesssoftware ecosystem quality in its broader sense
TechnicalperspectiveBusinesspersepctiveOSSecosystem
8. 8
From the challenge:
How can we measure the quality of these OSS projects?
To the research goal of this work:
How can we measure the quality of these OSS ecosystems?
9. 9
Proposal: measure OSS ecosystem quality using a quality model
“ quality model is the set of characteristicsand the relationshipsbetween them which provide the basis for specifying quality requirements and evaluating quality”
(ISO/IEC 9126, 2001)
Our working principle:
Collect metrics proposed in the state of the art and structurethem in a quality model
10. 10
Overview
Related work
Method
QuESo
Application
example
Discussion
Future work
Conclusions
Outline
12. 12
Overview
Related work
Method
QuESo
Application
example
Discussion
Future work
Conclusions
Outline
13. 13
Gathering measures
Systematic
literature
review
1
Constructing Model
Bottom up
Top down
2
The methodology have two steps
14. 14
Gathering measures from the available literature.
Systematic literature review
(Kitchenham and Charters, 2007).
Measure inclusion criteria:
(Hartighet al., 2013; Neuet al., 2011)
User-friendlyand
operationalizable.
Non-redundant
68 measures!
BookManual searchSecondary studiesDigital libraries351116Papers with measures of OSS ecosystems17
15. 15
Method for quality model construction. RadulovicandGarcia-Castro(2011)
Sub-characteristics
Relationships
Quality measures
Derived measures
Basic measures
Align quality sub- characteristics with a quality model
QuESo
SLR measures
QualOSS
ISO Quality model
16. 16
Overview
Related work
Method
QuESo
Application
example
Discussion
Future work
Conclusions
Outline
17. 17
Quality characteristics in QuESo have
been organized into three dimensions
QuESo Community
quality
Ecosystem
network
quality
Platform
quality
Software platform in
which the projects
of the ecosystem
are built.
OSS communities
that grow inside the
ecosystem and
ecosystem’s projects
OSS ecosystem as a
network of interrelated
elements
#
11
54
8
20
18. 18
Maintenance capacity :Ability to provide
the resources necessary for maintaining
community products
Dimension
Characteristic
Sub-characteristic
From QualOSS
Measure
QuESo
Maintenance
capacity
Community
quality
Size
Activeness
number
authors
Gnome: 3.500
1997-2012
number of
commits
Gnome:
480.000
1997-2007
19. 19
Network health: How well members are connected and their impact
Ability of the nodes to establish connections between them
Capacity of the ecosystem nodes to be classified around its projects.
QuESoEcosystemnetwork qualityNetwork healthClusteringInterrelatednessnumber community projectsGnome: 1.200 1997-2012Contributor activity graphGnome:499 projects / 15.000 changes1998-2011
27. 27
Overview
Related work
Method
QuESo
Application
example
Discussion
Future work
Conclusions
Outline
28. 28
Discussion
Some observations
Completeness:To remarkthat QuESo model may not be complete
Quantitative vs. qualitative: QuESo measures are mostly quantitative.
Unbalanced distribution of measures:more research needed
Measure names:room for improvement
29. 29
Overview
Related work
Method
QuESo
Application
example
Discussion
Futurework
Conclusions
Outline
30. 30
Futurework: Perform a complete quality assessment
Define
Quality assessment process
How are the valuesof each measure interpreted?
Good and bad values?
How can the measures be merged to
provide the assessment for a particular
sub-characteristic of the quality model?
What are the principles to perform the assessment with missing, incorrect,
and/or inconsistent measure data?
31. 31
Overview
Related work
Method
QuESo
Application
example
Discussion
Futurework
Conclusions
Outline
32. 32
Conclusions
We have presented QuESo, a quality model for assessing the quality of OSS ecosystems.
This quality model has been constructed following bottom-up – Top down strategies.
QuESo coversthe platform, the community, and the ecosystem network dimensions.
This quality model can be used as a starting pointfor the quality assessment of an OSS ecosystem