Version control for models, including model diff & merge, is not only a crucial prerequisite for a wide-spread adoption of model-based engineering in industry, it also is and has been a popular and very active research topic since more than ten years. Several important algorithms and approaches emerged in the past to support the identification of differences among model versions, as well as to merge them into a new version. Many of those ideas have also been successfully transferred and implemented in proprietary and open-source modeling tools used in industry. During the last few years, especially Eclipse-based open-source modeling tools, including support for version control built on Papyrus, EMF Compare, and EGit, gained significant attention and evolved into an industry-ready platform for building modeling tools in several domains. With this increasing industrial usage, however, the involved open-source technologies are challenged in different aspects ranging from the requirement for full customizability, performance with very large models to usability supporting user with strongly varying backgrounds. In this talk, we report on our experiences gained from moving from research in the area of model diff & merge to applying and enhancing open-source technologies, such as EMF Compare, EGit, and Papyrus for industrial use. We discuss engineering and research challenges that we faced when working with industrial users and how we approach them. Several of those challenges are still open though and require a strong collaboration of researchers, technology providers, and industrial users. Open-source platforms and technologies are a great opportunity to enable such a collaboration. This is also one of the key motivations behind the Papyrus Industrial Consortium, which was founded to foster collaboration among academia and industry in the area of open-source modeling tools based on Papyrus, EMF Compare, and EGit. http://www.models-and-evolution.com/2016/?keynote

1. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 1
Version Control for Models
From Research to Industry and Back Again
Philip Langer
planger@eclipsesource.com

2. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 2
● 2009 - 2012: PhD Student and junior researcher
○ Version Control for Models, model transformation, MBE
○ First presentation at a conference was at MoDELS 2009:
Composite model operations in the context of version control
● 2012 - 2014: Post Doc and senior researcher
○ Version Control for Models, model transformation, MBE
○ Model simulation, execution, debugging, model-based testing
My Background

3. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 3
● 2014 - now: EclipseSource in Vienna
○ Open source technology and service provider
○ Support customers in applying OS Eclipse technologies
■ Sponsored open-source development
■ Integration and customization
■ Developer support, consulting, and training
○ Eclipse committer (projects in the ecosystem of EMF)
○ Main focus
■ Building Eclipse-based modeling tools
■ Industrial-quality version control support for models
■ Architecture board of the Papyrus IC
My Background

4. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 4
● Tooling that enables...
○ working on shared models
○ changing models separately in isolated branches
○ obtaining differences among model versions
○ sharing changes with others
○ merging changes with changes others have made
Version Control for Models
mastermaster
featureX

5. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 5
● Tooling that enables to efficiently use git for models
● Isn’t git on its own enough?
Version Control for Models
Taken from https://github.com/KnowitLabs/Project-Workflow-and-Conventions

6. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 6
● Tooling that enables to efficiently use git for models
Version Control for Models

7. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 7
● Tooling that enables to efficiently use git for models
Version Control for Models

8. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 8
● Tooling that enables to efficiently use git for models
Version Control for Models
origin
rightleft

9. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 9
● Tooling that enables to efficiently use git for models
Version Control for Models
origin
rightleft
What are the
differences?
What are the
differences?

10. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 10
● Tooling that enables to efficiently use git for models
Version Control for Models
origin
rightleft
Are there conflicts?
~

11. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 11
● Tooling that enables to efficiently use git for models
Version Control for Models
origin
rightleft
Apply differences (merge)
+

12. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 12
● Tooling that enables to efficiently use git
● On model-level
● For Papyrus models
→ Seamless integration of
Version Control for Models

13. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 13
● Tooling that enables to efficiently use git
● On model-level
● For Papyrus models
→ Seamless integration of
Version Control for Models

14. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 14
●
○ A generic model differencing and merging framework
○ Compare and merge the on the model level
○ Open source project in the Eclipse ecosystem
○ Generic algorithms
■ Make use of the metamodel to obtain knowledge on logical structure
■ EMF’s Reflection API to access and compare values generically
■ Support every modeling language that is specified in EMF
✔ Model matching
✔ Model comparison
✔ Model conflict detection
✔ Merging of model differences
Model Differencing with EMF Compare

15. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 15
Model Differencing with EMF Compare

16. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 16
● Model comparison phases
○ Matching
Finding corresponding model elements
○ Diffing
Identifying differences among
corresponding model elements
○ Analysis of differences
Equivalences, dependencies, and
conflicts among differences
EMF Compare’s Comparison Process

17. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 17
● Model comparison phases
○ Matching
Finding corresponding model elements
○ Diffing
Identifying differences among
corresponding model elements
○ Analysis of differences
Equivalences, dependencies, and
conflicts among differences
EMF Compare’s Comparison Process

18. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 18
● Model comparison phases
○ Matching
Finding corresponding model elements
○ Diffing
Identifying differences among
corresponding model elements
○ Analysis of differences
Equivalences, dependencies, and
conflicts among differences
EMF Compare’s Comparison Process

19. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 19
● Model comparison phases
○ Matching
Finding corresponding model elements
○ Diffing
Identifying differences among
corresponding model elements
○ Analysis of differences
Equivalences, dependencies, and
conflicts among differences
EMF Compare’s Comparison Process

20. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 20
Comparison Model
Match Model
● Matches of corresponding elements
● Scope of the comparison
Difference Model
● List of differences of each side
● Generic differences
○ Reference change
○ Attribute change
○ …
Relationships among differences
● Equivalences
● Dependencies
● Implications
● Conflicts
● Model comparison phases
○ Matching
Finding corresponding model elements
○ Diffing
Identifying differences among
corresponding model elements
○ Analysis of differences
Equivalences, dependencies, and
conflicts among differences
EMF Compare’s Comparison Process

21. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 21
EMF Compare’s Comparison Process
Comparison Model
Match Model
● Matches of corresponding elements
● Scope of the comparison
Difference Model
● List of differences of each side
● Generic differences
○ Reference change
○ Attribute change
○ …
Relationships among differences
● Equivalences
● Dependencies
● Implications
● Conflicts
Merge viewer
Merge engine

22. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 22
Selected Challenges

23. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 23
● Crucially important but challenging
○ Users get really angry if the merge breaks the model!
○ Combination of potential input is huge (changes, model elements, …)
○ Dozens of corner cases and nasty combinations
● Fuzz testing
○ Run software under test with pseudo random input
○ Monitor software under test during execution
○ Check for crashes and failures
Reliability
Software
Input-independent
postcondition
Fuzz Generator
Test coverage

24. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 24
● Goal
○ Find combination of model changes applied to an EMF model
○ Cause erroneous matching, diffing, and/or merging
● Fuzz Testing Framework for EMF tools
○ Part of EMFStore
○ Repeatedly run JUnit tests with fuzzed input
○ Pseudo-randomly generate/mutate EMF models
● Basic Idea
○ Generate an EMF model conforming to a metamodel: m1
○ Copy and mutate this model: m2
○ Compare and merge in both directions: m1→2
and m2→1
○ Compare m1
with m2→1
and m2
with m1→2
○ Assert that there are no differences
■ Between m1
and m2→1
■ Between m2
and m1→2
Reliability
m1
Fuzz Generator
m2
Compare & Merge
(r → l, l → r)
m1
m2== ==m2→1
m1→2

25. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 25
● EMF Compare and EGit (integration of git in Eclipse)
○ EMF Compare extends EGit
■ Handling of involved model resources
■ Comparisons among branches, references, tags
■ Merge, rebase, and cherry-pick
○ Git has a file-by-file way of working
○ Intercepting the merge of single model files is not enough
Model Versioning with EMF Compare and EGit

26. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 26
Model Versioning with EMF Compare and EGit

27. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 27
Model Versioning with EMF Compare and EGit

28. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 28
● EMF Compare and EGit (integration of git in Eclipse)
○ EGit had to be extended to support so-called logical model support
■ “Logical model” is a set of logically interrelated files
■ EMF Compare provides those logical models for EMF
○ EMF Compare handles connected model resources as a whole
■ Update of cross-file references
■ Propagation of model element deletions
→ Support for model fragmentation operations
→ Support for file operations in EMF Compare
Model Versioning with EMF Compare and EGit

29. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 29
Generic model comparison is on the EMF model level.
Gap between Generic Model Comparison and Papyrus

30. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 30
But this is not enough!
Gap between Generic Model Comparison and Papyrus
Papyrus

31. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 31
But this is not enough!
Papyrus
Gap between Generic Model Comparison and Papyrus
Hides complexity
of the modeling
language
Offers composite
actions and types
Provides
dedicated views
Especially with
UML-based
DSMLs

32. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 32
But this is not enough!
Gap between Generic Model Comparison and Papyrus
Papyrus
?
Gap between Modeling Editor and Model Versioning Tools

33. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 33
Gap between Generic Model Comparison and Papyrus
Papyrus EMF Compare
How a user applies a change How EMF Compare sees the change
...

34. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 34
Gap between Generic Model Comparison and Papyrus
Papyrus
“Papyrus Compare” Generic Model Comparison

35. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 35
Gap between Generic Model Comparison and Papyrus
Papyrus EMF + Papyrus Compare
How a user applies a change How Papyrus Compare shows the change
...

36. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 36
“Papyrus Compare”
● Seamless Papyrus integration
✔ Support for UML Profiles
✔ Support for UML Profile migration
✔ Grouping of several composite editing actions
✔ Specific conflict handling for those actions
✔ Direct integration of model decorations
✔ Integration of Papyrus diagram styling
✔ Specific handling of meta-information (di)
✔ ...

37. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 37
● Papyrus is a DSM platform itself
○ Custom model decoration and diagram styling
○ Custom composite model element patterns
○ Custom composite model editing actions
○ Custom conflict and merge rules
○ ...
Gap between Generic Model Comparison and Papyrus DSML

38. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 38
Gap between Model Versioning Tools and Modeling Editors

39. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 39
Future Challenges

40. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 40
● DSML customizations are redundantly and manually implemented
○ Incremental post-processing of comparison model
○ Customization of UI components
○ But this customization replicates the customization of Papyrus*
● Goal
○ Perform DSML customization once → see the effect everywhere
● Shared DSML Customization Model
○ Defines element patterns (“element types”)
○ Defines composite operations
○ Customization model is consumed by Papyrus, EMF Compare, ...
● Support for “customization of customizations” is crucial
Single-sourcing the Customizations for DSMLs

41. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 41
● Better support for diagrams and tables
○ Different domains have different requirements, backgrounds, preferences
○ Diagram-oriented users, table-oriented, model-tree-oriented users
● Diagram-oriented users want to see all changes on the diagram
○ Not only diagram changes, also model changes
○ Synchronized diagram support
■ New model element → new shape
■ Change dependencies need to be adapted
○ Integration with auto-layouting support
■ Auto-layout after merge
■ Conflicts on diagrams?
Better Concrete Syntax Comparison for DSMLs

42. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 42
● Different user roles use different views and syntaxes
○ System architect vs. electrical engineer
○ High-level vs. detailed views
○ Diagram-oriented users vs. textual-oriented users
○ But they still work on shared models
User-oriented Multi-view/Multi-syntax Comparison for DSMLs

43. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 43
User-oriented Multi-view/Multi-syntax Comparison for DSMLs

44. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 44
User-oriented Multi-view/Multi-syntax Comparison for DSMLs

45. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 45
● EUREKA project: “Hybrid Modeling”
○ Better integrating textual and graphical modeling
○ Papyrus for Real-Time as a case study
User-oriented Multi-view/Multi-syntax Comparison for DSMLs

46. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 46
● Resilience to problems in the model
○ Modeling tools are too restrictive and assume a well-formed model
○ We can’t really recover if there is something wrong (broken model, unresolved
parts)
○ Compare merge markers for text
○ More resilient modeling tools with quick-fix support
● Model review
○ Gerrit (or github) code review for models
○ Support reviewing, commenting, and annotating changes
○ Model/diagram changes, obviously
Further Topics

47. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 47
● Open consortium with a common goal
○ Developing a universal, industry-ready, open-source MBE solution
○ Based on Eclipse Papyrus and many other open-source components
● Strong collaboration
○ Tool users
○ Tool providers
○ Research and academia
● Independent of the domain
○ Systems modeling
○ Architecture modeling
○ Enterprise modeling, …
● Talk to us, if you are interested to collaborate
https://wiki.polarsys.org/Papyrus_IC
Papyrus Industrial Consortium

48. © 2016 EclipseSource | http://eclipsesource.com/vienna | Philip Langer | Models and Evolution 2016 | Version Control for Models 48
Thank you very much!
● Open source collaboration
● Follow us on collaborative-modeling.org
● Contact us
○ Papyrus IC Research & Academia Committee
○ planger@eclipsesource.com
Related Sessions
Tomorrow
OSS4MDE Workshop
Wednesday Evening
Weclome Reception