Ever wondered how IDE’s are built? In this talk, we’ll skip the marketing bit and dive into the architecture and implementation of JetBrains Rider.
We’ll look at how and why we have built (and open sourced) a reactive protocol, and how the IDE uses a “microservices” architecture to communicate with the debugger, Roslyn, a WPF renderer and even other tools like Unity3D. We’ll explore how things are wired together, both in-process and across those microservices. Let’s geek out!
5. JetBrains
Founded 2000 in Prague (Czech Republic)
2000 IntelliJ Renamer
2001 IntelliJ
2004 ReSharper
2019 20+ IDE’s and other developer tools
6.
7. ReSharper IDE
Project halted (but not gone to waste)
Several concepts and architecture remained
Keep functionality separate from the actual IDE
Same core, IDE interoperability layer on top
Visual Studio 2010, 2013, 2015, 2017 and 2019
ReSharper command line tools (CLI)
9. Why build a .NET IDE?
Many reasons!
“When will JetBrains come with its own .NET IDE?”
ReSharper constrained by Visual Studio environment
32 bit process resource constraints
Changes in VS impact ReSharper
.NET Core
No good cross-platform IDE existed at the time
10. Cross-platform...
A cross-platform UI toolkit is needed!
ReSharper UI elements in WinForms and WPF
Existing ReSharper UI would need converting, new UI to be built
WinForms? (Mono sort of has it)
GTKSharp?
Qt?
11. IntelliJ Platform
Foundation of all of our IDE’s
Provides base infrastructure
Project view, code completion, UI toolkit
+ Platform plugins such as version control, terminal, ...
+ JetBrains <product name> IDE plugins
Open source (build your own IDE – e.g. Android Studio, Comma IDE & others)
https://github.com/JetBrains/intellij-community
Windows, Linux, Mac – already cross-platform thanks to JVM
12. IntelliJ Platform + R# ?
IntelliJ Platform
Great foundation to build on
Windows, Linux, Mac
JVM
ReSharper (R#)
All of those .NET inspections, refactorings, code generation, project model, ...
.NET
13. Options!
Rewrite R# in Java?
14 years of implementation and knowledge
Would bring 2 R# implementations... Automatic conversion?
Run R# as a command-line process
Already possible (thanks, 2004!)
“Just need our own UI on top”
14. IntelliJ Platform + R# !
Headless R# as a language server
Cross-platform (.NET / Mono)
No constraints
It is ReSharper! 2 products, 1 code base
IntelliJ as a thin UI
Control the R# process
15. Both sides are an IDE...
Is IntelliJ really a thin UI?
Three cases...
Features where IJ handles everything
Features where R# handles almost everything
Features where both IDE’s make a more awesome IDE
17. How to make them talk?
Inter-process communication
18. Example: Context actions (Alt+Enter)
1. IntelliJ provides text editor, caret(s) and lets us Alt+Enter
2. Ask current document’s language for items to display in Alt+Enter menu
For C#, this language is a facade to the R# process
3. IntelliJ renders list of items, may add its own entries
Data: a tree of names and icons
19. 1. IntelliJ provides text editor and plumbing to display squiggles
2. IntelliJ notifies R# that a document was opened (or modified)
3. R# does its thing (analyze, inspect, summarize that data)
4. R# publishes this to IntelliJ
5. IntelliJ displays this info
Data: a set of name, icon, severity, tooltip, text range
Can’t do RPC! Analysis can take any amount of time, so IJ should not wait for it.
~~~~~~
Example: Inspections and highlighting
20. 1. Bi-directional
User can be typing
A refactoring or completion may be injecting code at the same time
2. Can be implemented with delta’s
IntelliJ pushes delta to R#
R# pushes delta to IntelliJ
Concurrency?
Data: a delta (from line + column, to line + column, text to insert)
Example: Writing code
21. Types of data
Context actions
A tree of names and icons
Inspections
A set of name, icon, severity, tooltip, text range
Writing code
Delta with from line + column, to line + column, text to insert
Fairly simple messages! Can we make this generic enough?
Make one inspection work make them all work
22. Which protocol do we use?
Re-use Language Server Protocol (LSP)?
Great in itself – IDE concepts like window, editor, language, diagnostics, ...
We would need customizations for R# feature set
Or build a custom REST-like protocol?
Experimented with JSON, ProtoBuf, request/response style
23. Request-action-response
LSP and custom protocol are request/response mostly
Conflict resolution...
What if both human and a refactoring make a change to code?
How to keep things synchronized and in a healthy state?
Realization: Why a “request-action-response” flow? Why RPC?
Both IDE’s share a similar model and architecture
Messages are simple, but for RPC they would need context
(which solution, which file, state info, ...) – overhead!
24. Model-View-ViewModel (MVVM)
IntelliJ is our view, ReSharper provides the model
Protocol is the ViewModel, sharing lightweight data
Project.Files.Add("Foo.cs")
Project.Files["Foo.cs"].Inspections.Add(
"Possible null reference", "Warning", 20, 30, 20, 42);
Both processes can react to such change (observable + observer)
25. Conflict resolution...
Changes to data in shared model can come from IJ and R#
Can still cause conflicts due to features or timing/GC issues
IntelliJ: “I just deleted file foo.cs”
R#: “I just refactored foo.cs”
Solutions!
Locking? (freezes, how to handle deadlocks?)
Conventions!
26. Conflict conventions!
View + Model (or client: IntelliJ + server: ReSharper)
Each value stored in the view model has a version
Updates by the view/client increment the version
Updates by the model/server do not
Only accept changes if version is the same or newer
If not, the change is discarded
29. Rider protocol
“Reactive Distributed communication framework for .NET, Kotlin, JS, C++”
Open source - https://github.com/jetbrains/rd
1. Include protocol libraries
and build tools on all sides
2. Write view model in special DSL
3. Generate code
4. Work with generated model
.NET/Kotlin/JS/... code generator
Model definition DSL
Primitives
Conflict resolution, serialization, ...
Sockets, batching, binary wire protocol
30. Rider protocol
Only need to know about a few primitives
Conflict resolution, wire protocol, timeouts, ... handled by protocol
Code generated based on the defined view model
Bonus points: no reflection/introspection needed on every run
Hierarchical + lifetimes
31. Signal (event)
Producers/subscribers
Observable/observer
Using lifetime to manage subscription
// Produce event
interface ISource<T> {
void Fire(T value);
}
// Subscribe to event
interface ISink<T> {
void Advise(Lifetime l, Action<T> handler);
}
// Event
interface ISignal<T> : ISource<T>, ISink<T> { }
32. Property
Signal implementation
Using lifetime to manage subscription
To changes to propery in general
To changes to specific value
// Observable property
interface IProperty<T> : ISink<T> {
T Value { get; set; }
void Advise(Lifetime l, Action<T> handler);
void View(Lifetime l, Action<Lifetime, T> handler);
}
33. Primitives
Primitive Description
Signal Event that is fired when something happens
Property Observable value
List/set/map Observable collections
Field Immutable value
Call/callback RPC-style call, needed from time to time
byte, short, int, long, float,
double, char, boolean, string,
securestring, void, enum, ...
Primitives and special types
Aggregatedef/classdef/structdef A node in the viewmodel
34. Hierarchical + lifetimes
Cleanup and resource management
Objects attach to lifetime
Lifetime destroys attached objects
Parent lifetime destroys children
NuGet: JetBrains.Lifetimes
public class Lifetime : IDisposable {
private Stack<Action> resources = new Stack<Action>();
public void Attach(Action resource) {
resources.Push(resource);
}
public void Attach(IDisposable disposable) {
resources.Push(disposable.Dispose);
}
public void Dispose() {
while (resources.Count > 0) {
var resource = resources.Pop();
resource();
}
}
}
35. Hierarchical + lifetimes
Solution
NuGet host
Project
Document
Inspections
PSI (Program Structure Interface)
Class
Field
Method
Document
Inspections
...
Project
Local history
NuGet tool window
Project
Editor tab
Inspections
Language
Editor tab
Inspections
Language
viewmodel(Riderprotocol)
37. Rider protocol
Very extensible through Kotlin-based DSL
Easy to work with for our developers
Update view model, generate code, work with generated code
Find Usages, Navigation, ... work while crafting model
No need to think about multiple processes, state, conflict resolution, ...
Cross-language, cross-platform
Plugin model for Rider is more complex (IJ and R# parts may be needed)
https://github.com/JetBrains/fsharp-support
https://github.com/JetBrains/resharper-unity
41. Multiple processes...
What if certain features were
running in their own process?
No need to run all the time
Own memory constraints
Start/stop/crash independently
42. Shared view model
Pass around a shared view model
to interested parties
Example: Roslyn analyzers/inspections
Pass around “reference” of
[
{ name, icon, severity,
tooltip, text range }
]
48. Model the view as well
public CSharpInteractiveOptionsPage(Lifetime lifetime, ...)
: base(lifetime, ...) {
AddHeader("Tool settings");
AddToolPathFileChooserOption(lifetime, commonFileDialogs);
AddEmptyLine();
AddStringOption((CSIOptions s) => s.ToolArguments,
"Tool arguments:", "Additional tool arguments");
AddHeader("Tool window behavior");
AddBoolOption((CSIOptions s) => s.FocusOnOpenToolWindow,
"Focus tool window on open");
AddBoolOption((CSIOptions s) => s.FocusOnSendLineText,
"Focus tool window on Send Line");
AddBoolOption((CSIOptions s) => s.MoveCaretOnSendLineText,
"Move caret down on Send Line");
// ...
FinishPage();
}
49. Multiple machines
WPF on macOS/Linux
Rendering on Windows
Front-end on one machine,
back-end on another
...
50. Every IDE as both a client and server
Front-end and back-end: separate process & memory
Whatever happens in the backend, the frontend can process the user's typing
Bring this technology to other IDE’s?
Reuse WebStorm's HTML/CSS/JS functionality in ReSharper
(e.g. Visual Studio + R# using WebStorm in back-end mode)
52. Conclusion
Rider is an IDE built on
two IDE’s
two technology stacks
Rich and easy programming model was needed to bridge the two
Protocol gave rise to
more than two processes
more than one machine
micro UI
Free trial! www.jetbrains.com/rider
This talk is about:
How Rider came to be
How protocol allowed us to gain additional knowledge and ideas of separating parts of our IDEs and building TOWARDS microservices
This talk is also about building a new product based on many years of previous investments
Open ContosoUniversity in Rider
Show solution explorer
Show editor where you can type, show inspections, navigation, refactoring
Mention debugging
Mention tools like database
Mention cross platform
Obligatory marketing sldie
Now that we have seen a bit of the IDE, let’s look at some history first.
Talk about history of JetBrains a bit, mention Eclipse in 2002, need for a new product.
Mention ReSharper plugin to VS..
ReSharper 1.0 -> 2.0 – let’s do a full IDE
Rely on being a plugin to VS? Or build a full .NET IDE?
It was never released, but a fully functional prototype.
Provided a solution explorer, an editor, find usages, code completion and refactorings.
Built on .NET WinForms and Windows Presentation Foundation (WPF) wasn’t around.
Project halted – VS plugin seemed best way to go
Project halted (but not gone to waste)
Concepts and architecture remained
Action system
Text control implementation
Several tool windows and toolbar controls
Unit test runner
ReSharper command line tools (CLI)
Keep functionality separate from the actual IDE
Helped future versions of ReSharper: Visual Studio 2010, 2013, 2015 and 2017
Same core, IDE interoperability layer on top
Headless R# as a language server
Cross-platform (.NET on Windows, Mono on Linux and macOS)
No constraints (64 bit process, and its own memory space)
It is ReSharper! 2 products, 1 code base
IntelliJ as a thin UI
Control the R# process
Client/server or IPC communication
Is IntelliJ really a thin UI?
Full IDE for its own languages, no need for R# there
Combined languages (e.g. JS/TS/HTML in IJ and R#)
Change tracking, VCS, REST client, tool windows, ...
Three cases...
Features where IJ handles everything
Features where R# handles almost everything
Features where both IDE’s make an awesome IDE
Both sides are an IDE!
Same concepts, but not the same knowledge about the project
Open ContosoUniversity in Rider
Navigate to *.html – Navigation already includes all flows! Files in IJ, symbols in IJ, symbols in R#
HTML editor is purely IntelliJ
Navigate to *.cshtml – Again both IDE’s at work
CSHTML is both C# and HTML – now what? Both IDE’s!
Navigate to HomeController
C# is al ReSharper. Or is it? Show database tools + language injection with database query.
Mention local history – tracked by IJ but R# needs to know about this too, e.g. in a big refactoring
If we are going to make them talk, let’s look at how we could model the data that goes over the wire.
Re-use Language Server Protocol (LSP)?
Great in itself – IDE concepts like window, editor, language, diagnostics, ...Built for/with VS Code and the ideas in that IDE.Not bad! PoSh plugin uses this, and works fine! But feature set is a bit more limited than what we have in R#.
We would need customizations...
LSP is lowest common denominator – some R# refactorings can not be done
Mixed languages? e.g. CSHTML which can be HTML + CSS + JS + C#/VB.NET
Build a custom REST-like protocol?
Experimented with JSON, ProtoBuf, request/response style
Slow, hard to customize, hard to develop with when all is in motion
Objects bind to other objects, instead of parent keeping track of just direct children
Open empty project in Rider, install a NuGet package, show log tab
Open Rider in IntelliJ IDEA
NuGetModel.kt – explain it Extends the solution node in the model
Has a bunch of inner classes, and properties
Interesting is sink(“log”)
Generated version: RdNuGetHost – Kotlin implementation of our view model
In RiderNuGetLogPanel – init -> facade.host.log.advise(facade.lifetime)
Open ReSharperHost.Generated.sln in Rider
Generated version: RdNuGetHost – C# implementation of our model
In NuGetNativeLogger, public override void Log(ILogMessage message) => myHost.Log.Fire( new NuGetLogMessage(message.Time, NuGetLogContext.NuGet, Convert(message.Level), message.Message));
Open Rider in IntelliJ IDEA
NuGetModel.kt – find property("configFiles", immutableList(classdef("RdNuGetConfigFile") {
RiderNuGetSourcesPanel – init - facade.host.configManager.configFiles.advise(facade.lifetime) { newFiles ->
Similar construct to subscribe to sources being added to the view model
Open ReSharperHost.Generated.sln in Rider
In NuGetCredentialProviderHost, show lifetime example – when solution closes the lifetime closes, so this thing cleans up as well
Mention WPF, WinForms
Thought experiments: WPF/XAML renderer
Rider and Unity Editor can be started independently, but this does not prevent us from both processes to look for each other’s Rider Protocol connection.
When both are launched and the connection is allowed, Rider can share its view model with Unity Editor and vice-versa.
This lets Rider control play/pause/and stop buttons in Unity, and also provides the ability to debug code in Rider, even though it is running in the Unity Editor.
Rider and Unity Editor can be started independently, but this does not prevent us from both processes to look for each other’s Rider Protocol connection. When both are launched and the connection is allowed, Rider can share its view model with Unity Editor and vice-versa. This lets Rider control play/pause/and stop buttons in Unity, and also provides the ability to debug code in Rider, even though it is running in the Unity Editor.