Scaling Xtext projects presented challenges for Sigasi's hardware design IDE due to large project sizes of over 250,000 lines of code. Initial performance of building and editing large files in Xtext was unacceptable, taking over 20 minutes to build and causing the editor to be unusable. Through performance analysis and optimizations including reducing workload, parallelizing builds, caching models, and implementing a lightweight editor, Sigasi was able to improve build times to under a minute and make editing large files usable. Continued work involves caching global index information and further linking and validation optimizations.

1. Scaling Xtext
Lieven Lemiengre

2. Sigasi
● IDE for Hardware Description Languages
○ VHDL, (System)Verilog
● Using Xtext for 4 years
● Large user base
○ (commercial, free, students)

3. Our company goal
● Assist hardware designer
● High quality interactive front-end compiler
○ Instant error reporting
■ parsing, semantic, linting, style checking
○ Visualisations, design exploration, ...
○ Integrate with ecosystem
■ other compilers, simulators, synthesizers

4. The challenge
● Large projects
○ > 250 KLOC is not uncommon
○ design + external libraries
○ big files
■ some libraries are distributed as 1 file
■ generated hardware cores
●

5. Adopting Xtext
● Started with the early Xtext 2.0 snapshots
● Initial performance analysis
○ Clean build performance of a big project (330k LOC)
■ > 20 minutes
■ > 2 GB
○ Editing big files (> 1 MB)
■ unusable

6. Adopting Xtext
● Started with the early Xtext 2.0 snapshots
● Initial performance analysis
○ Clean build performance of a big project (330k LOC)
■ > 20 minutes → < 1 min
■ > 2 GB → < 1 GB memory
○ Editing big files (> 1 MB)
■ unusable → usable with reduced editor

7. ● Xtext framework improvements
● Measure → analyze → improve or cheat
○ faster build
○ reduce memory usage
○ UI responsiveness
Improving performance

8. Overview
● Analysing build performance
○ Analyze the build
■ Macro build measurements
■ Key performance points
○ Reduce workload
○ Parallelize the build
● Analyzing UI issues
○ Monitoring the UI thread
○ Saveguards

9. Analyzing builds: builder overview
Global
indexing
Linking
Validation
Custom
Validation
Global
index
Eclipse
resources
warnings
errors
resource
descriptions
Builder
Participants
resource
changes
?

10. Analyzing builds: metrics
● For each build
○ # of files being build
○ timing: Global index, Linking, Validation, Individual
builder participants
● Instrument by overriding
ClusteringBuilderState & XtextBuilder
● Example: Building 134 resources, timing: {
global index=1806,
linking=378,
validation=823,
totalLinkingAndValidation=1364
}

11. Analyzing builds: resource loads
● Observation:
○ Most time spent in resource loads
○ Certain files are loaded multiple times?!
Global
indexing
Linking Validation
Builder
Participants
resources
LOAD
POTENTIAL
RELOAD
POTENTIAL
RELOAD

12. Analyzing builds: Memory footprint
Global
index
Resource
Set
● Global index
○ always loaded →limit size
○ use hashing: name + hash of contents
● Reduce EMF size
derived model -> Xcore model
http://www.sigasi.com/content/view-complexity-your-xtext-ecore-model

13. Optimize loading
● What is resource load?
○ Parse
○ build EMF model
○ build Node model
● Parallelise
○ parse multiple files simultaneously
○ ~3 time faster loads on 4 core machine
○ EMF synchronization issues
● Cache
○ serialize EMF and Node model in a cache
○ 3-4 time faster loads

14. Linking
Global
indexing
Linking Validation
Builder
Participants
● Language specific
○ VHDL vs Verilog
● Avoiding linking
○ expressions that we couldn’t handle (yet)
○ library files, only when used in user-code
● Many iterations
○ lazy linking vs eager linking
○ From 40% of build time to 20%

15. Linking
Pseudo-algorithm
● In your IScopePovider
○ Find a eContainer that can be linked eagerly
○ Link everything
■ unresolved links become UnlinkedDeclaration
● In your DefaultLinkingService
○ getLinkedObjects(...)
■ already resolved? → skip it
● Disable notifications

16. Validation
Global
indexing
Linking Validation
Builder
Participants
● Combine validations to avoid model
traversals
● Local analysis, do global validations moved
into builder participant
● Avoid validation
○ disabled validations
○ libraries: errors & warnings are suppressed anyway
● Monitor

17. Track performance
● Nightly build
● log build times

18. UI responsiveness
● Measuring: detect a blocked UI thread
○ initially Svelto https://github.com/dragos/svelto
○ now our own method & logging
○ Eclipse Mars
● Improvements
○ UI is for drawing only!
○ Make sure everything is cancellable
● Safeguards
○ certain services should never be executed on the UI
thread => check & log

19. Lightweight Editor (fallback)
● Syntax-highlighting + markers
● For files > 1 MB
● Based on ContentTypes extension point

20. Two ContentTypes (based on file size)
<extension point="org.eclipse.core.contenttype.contentTypes">
<content-type ...
describer="com.sigasi...FullVhdlContentDescriber"
name="VHDL editor"
<describer class="...FullVhdlContentDescriber" />
</content-type>
<content-type ...
describer="com.sigasi....LightweightVhdlContentDescriber"
name="Lightweight VHDL editor"
<describer class="...LightweightVhdlContentDescriber" />
</content-type>
</extension>

21. Future work
● Continuous process
● Cache global index info per resource?
● Linking without node model?
● Avoid project locking build (Xtext 2.8?)

22. Come talk to us about...
● Documentation generation
● Fancy linking algorithms / type systems
● Graphical views
● Cross-language support
● Testing Xtext-plugins
● Lexical macros
● Manage large amount of validations
● ...

23. +1 0 -1
Sign in: www.eclipsecon.org
Evaluate the sessions