This document summarizes Yusuke Endoh's talk on plans for Ruby 3 types. It discusses Matz's plan to include type signatures in Ruby 3 to enable optional static type checking. Two approaches are proposed: 1) A level-1 type checker without signatures that can detect possible errors through type inference. 2) A level-2 checker with signatures that verifies code complies with provided signatures. The talk introduces Type Profiler, an experimental tool that uses abstract interpretation to infer types in Ruby code and detect possible errors, as an example of approach 1. The goal is to enable static analysis with little impact on the Ruby programming experience.

1. A Plan towards Ruby 3 Types
Yusuke Endoh
EuRuKo '19 - 22rd Jun. 2019
1

2. Yusuke Endoh (@mametter)
•A full-time Ruby committer at Cookpad Inc.
•with @ko1 (Ruby's speed freak)
2

3. PR: Cookpad Inc.
•Mission: "Make Everyday Cooking Fun"
•cookpad.com: A recipe sharing service
•Monthly Average Users: 96 million
3

4. PR: Cookpad Inc.
We're hiring!
HQ is in Bristol, UK
Aim to be No.1 in 100 countries
(Now in 29 Languages and 72 Countries)
4

5. Yusuke Endoh (@mametter)
•A full-time Ruby committer at Cookpad Inc.
• with @ko1 (Ruby's speed freak)
•My contributions:
• Keyword arguments since Ruby 2.0
• Test coverage measurement
• Optcarrot: a benckmark for Ruby 3x3
• Recent: Beginless/endless range: (1..), (..1)
5

6. Ruby 3 goals
•Performance (Ruby 3x3)
•Concurrency
•Static analysis  My mission
6

7. This Talk
•Matz's plan for Ruby 3 Types
•An introduction of Type Profiler
7

8. Matz's plan for Ruby 3 Types
8

9. Ruby 3 Types'
•Objective: Find a possible type bug
•To help development
•Requirement: Keep Ruby's experience
•Type annotation is optional
9
# s is String
def foo(s)
...
end
type
annotation

10. Items for Ruby 3 Static Analysis
1. Standard type signature format
2. Level-1 type checker without signatures
+ Type inference to suggest type signature for
non-annotated Ruby code
3. Level-2 type checker with signatures
10

11. 1. Type Signature Format (.rbi)
•What types a method accepts/returns
class Array[A]
include Enumerable
def []: (Integer) -> A?
def each: { (A)->void } -> self
...
end
mix-in
generics
option type
interface
any type
Proposal:
ruby-signature
11

12. 2. Type Checker without Signature
•Finds possible NoMethodError/TypeErrors
•Work without application's type signature
•May report false positive
def foo(s)
s.gsuub!(//, "")
s + 42
end
foo("foo")
NoMethodError?
TypeError?
Proposals:
• Type Profiler
• mruby's JIT compiler
12

13. 2'. A kind of Type Inference
•Suggest a prototype of type signature for
non-annotated Ruby code
• Possible approach
• virtually simulating
the code in type-level
def foo(s)
s.to_s
end
foo("foo")
foo(42)
(String | Integer)
-> String ?Proposals:
• Type Profiler
• mruby's JIT compiler
13

14. 3. Type Checker with Signatures
•Conservatively find NoMethod/TypeErrors
•Verifies that the code complies with signature
• Possible implementation
• Gradual type checking
• Tool-defined annotation def foo(s)
s.gsuub!(//, "")
s + 42
end
TypeError!
def foo:
(String) -> Integer
Proposals:
• Steep
• Sorbet
• RDL
NoMethodError!
14

15. Items for Ruby 3 Static Analysis
Library code
type signature
Sorbet
Steep
RDL
Type error
warnings
Application code
Type Profiler
(mruby JIT)
Type error
warnings
type signature
15

16. Use Cases
• I want to find a possible bug
• Write a code
→Use 2 (level-1 type checker without signature)
• I want to verify my code
• Write a code
→ Use 2' (type inference)
→ Use 3 (level-2 type checker with signature)
• I want to write a code in "type-driven" style
• Hand-write a signature and then a code
→ Use 3 (level-2 type checker with signature)
16

17. Current Development Status
•ruby-signature: discussed by tool developers
• Any comment is welcome:
https://github.com/ruby/ruby-signature
•Type-Profiler: Very experimental...
•Steep: Trial use in Sider, Inc.
• Good for duck typing / requires many annotations
•Sorbet: Trial use in some companies
• Gradually applicable / less support for duck typing
17

18. What Ruby 3 will Ship
•Matz wants to bundle "type signatures"
•A parser library for the format
•Type signature files for stdlibs
•RubyGems with type signature support
•No plan to bundle the checkers
•They will be released as external gems
18

19. An introduction of Type Profiler
19

20. Type Profiler for Checking
•Finds NoMethodError, TypeError, etc.
def foo(n)
if n < 10
n.timees {|x|
}
end
end
foo(42)
Type
Profiler
t.rb:3: [error] undefined method:
Integer#timees
Typo
20

21. Type Profiler for Inference
•Generates a prototype of type definition
def foo(n)
n.to_s
end
foo(42)
Type
Profiler
Object#foo ::
(Integer) -> String
21

22. Demo: Overloading
def my_to_s(x)
x.to_s
end
my_to_s(42)
my_to_s("STR")
my_to_s(:sym)
Type
Profiler
Object#my_to_s :: (Integer) -> String
Object#my_to_s :: (String) -> String
Object#my_to_s :: (Symbol) -> String
22

23. Demo: Recursive Method
def fib(n)
if n > 1
fib(n-1) + fib(n-2)
else
n
end
end
fib(10000)
Type
Profiler
Object#fib ::
(Integer) -> Integer
23

24. How Type Profiler Does
•Runs a Ruby code in "type-level"
Normal interpreter
def foo(n)
n.to_s
end
foo(42)
Calls w/
42
Returns
"42"
Type profiler
def foo(n)
n.to_s
end
foo(42)
Calls w/
Integer
Returns
String
Object#foo ::
(Integer) -> String
24

25. Type Profiler and Branch
•"Forks" the execution
def foo(n)
if n < 10
n
else
"error"
end
end
foo(42)
Fork!
Now here
We cannot tell
if n<10 or not
Returns
Integer
Returns
String
Object#foo ::
(Integer) ->
(Integer | String)
25

26. The Problem: State Explosion
•Analysis time (at April, at RubyKaigi)
•Analyzing the code of Type Profiler: 10 min.
•Analyzing optcarrot: 3 min.
a=b=c=d=e=nil
a = 42 if n < 10
b = 42 if n < 10
c = 42 if n < 10
d = 42 if n < 10
e = 42 if n < 10
Fork!
Fork!
Fork!
Fork!
Fork!
2
4
8
16
32
State numbers
26

27. The Progress since RubyKaigi
•Revamped the analysis algorithm
•"state merging" technique called in
symbolic execution
27

28. Basic Analysis Algorithm
•"Environment": A map from variable to type
•Example:
•TP runs each instruction iteratively
•by propagating the environments
•until no environments
are updated
28
x = 1
x = 1
x = 1
x y
{ int, str } { int }
x y
{ nil } { int }
∅ ∅
x y
{ nil } { int }
{ int } { int }

29. Example of Analysis
1: def foo(a)
2: if a < 10
3: b = 42
4: else
5: b = "str"
6: end
7: c = b
8: c
9: end
10:
11: ret = foo(42)
Line# a b c
1 ∅ ∅ ∅
2 ∅ ∅ ∅
3 ∅ ∅ ∅
4 - - -
5 ∅ ∅ ∅
6 -
7 ∅ ∅ ∅
8 ∅ ∅ ∅
Line# a b c
1 { Int } ∅ ∅
2 ∅ ∅ ∅
3 ∅ ∅ ∅
4 - - -
5 ∅ ∅ ∅
6 -
7 ∅ ∅ ∅
8 ∅ ∅ ∅
Line# a b c
1 { Int } ∅ ∅
2 { Int } ∅ ∅
3 ∅ ∅ ∅
4 - - -
5 ∅ ∅ ∅
6 -
7 ∅ ∅ ∅
8 ∅ ∅ ∅
Line# a b c
1 { Int } ∅ ∅
2 { Int } ∅ ∅
3 { Int } ∅ ∅
4 - - -
5 { Int } ∅ ∅
6 -
7 ∅ ∅ ∅
8 ∅ ∅ ∅
Line# a b c
1 { Int } ∅ ∅
2 { Int } ∅ ∅
3 { Int } ∅ ∅
4 - - -
5 { Int } ∅ ∅
6 -
7 { Int } { Int } ∅
8 ∅ ∅ ∅
Line# a b c
1 { Int } ∅ ∅
2 { Int } ∅ ∅
3 { Int } ∅ ∅
4 - - -
5 { Int } ∅ ∅
6 -
7 { Int } { Int } ∅
8 ∅ ∅ ∅
Line# a b c
1 { Int } ∅ ∅
2 { Int } ∅ ∅
3 { Int } ∅ ∅
4 - - -
5 { Int } ∅ ∅
6 -
7 { Int } { Int, Str } ∅
8 ∅ ∅ ∅
Line# a b c
1 { Int } ∅ ∅
2 { Int } ∅ ∅
3 { Int } ∅ ∅
4 - - -
5 { Int } ∅ ∅
6 -
7 { Int } { Int, Str } ∅
8 ∅ ∅ ∅
Line# a b c
1 { Int } ∅ ∅
2 { Int } ∅ ∅
3 { Int } ∅ ∅
4 - - -
5 { Int } ∅ ∅
6 -
7 { Int } { Int, Str } ∅
8 { Int } { Int, Str } { Int, Str }
Object#foo :: (Int) -> (Int|Str) 29

30. Experiment result
•Type Profiler: 10 min. ➔ 2.5 sec.
•optcarrot: 3 min. ➔ 6 sec.
30

31. Analysis Time of Type Profiler
0
200
400
600
800
self-profiling optcarrot
seconds
old new
31

32. Other Problems of Type Profiler
•It requires a test
•False positive and false suggestion
•Some Ruby features cannot be handled
• e.g., Object#send, singleton classes
# b: Integer or String
c = b
# c: Integer or String
# We lose the correspond between b and c
b + c # "May call Integer#+(String)!"
32

33. Development Progress: Done
•Design the basic analysis algorithm
•Based on abstract interpretation
•Support basic language features
•Variables, methods, user-defined classes, etc.
•Blocks and arrays (maybe)
•A limited set of built-in classes
33

34. Development Progress: To Do
•Support more features
•More built-in classes (Hash!)
•Complex arguments (optional/rest/keyword)
•Exception
•Modules
•Input/output type signature format
•A lot of improvements for practical use...
34

35. Acknowledgement
•Hideki Miura
•Matz, Akr, Ko1
•PPL paper co-authors
• Soutaro Matsumoto
• Katsuhiro Ueno
• Eijiro Sumii
•Stripe team & Jeff Foster
•And many people
35

36. Conclusion
•Explained Matz's plan for Ruby 3 static analysis
•Introduced Type Profiler
• A type analyzer for Ruby 3
applicable to a non-annotated Ruby code
• Based on abstract interpretation technique
• Little change for Ruby programming experience
•Any comments and/or contribution are welcome!
• https://github.com/mame/ruby-type-profiler
36