SwiftcKaigi #1

1. Integrating libSyntax into
the compiler pipeline
1 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

2. Hi, I'm Yusuke
@kitasuke
2 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

3. Swift Compiler
3 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

4. Parser
4 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

5. Overview of Parser
5 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

6. Parser
6 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

7. TokenKind
SYNTAX_TOKENS = [
# Keywords that start decls
DeclKeyword('Associatedtype', 'associatedtype', serialization_code=1),
DeclKeyword('Class', 'class', serialization_code=2),
DeclKeyword('Deinit', 'deinit', serialization_code=3),
DeclKeyword('Enum', 'enum', serialization_code=4),
DeclKeyword('Extension', 'extension', serialization_code=5),
DeclKeyword('Func', 'func', serialization_code=6),
...
]
7 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

8. Parser
8 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

9. Syntax tree
9 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

10. SyntaxKind
enum class SyntaxKind {
TypealiasDecl,
AssociatedtypeDecl,
ClassDecl,
StructDecl,
ProtocolDecl,
FunctionDecl,
...
};
10 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

11. Parser
11 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

12. AST
12 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

13. AST base objects
/// Stmt - Base class for all statements in swift.
class alignas(8) Stmt {...}
/// Decl - Base class for all declarations in Swift.
class alignas(1 << DeclAlignInBits) Decl {...}
/// Expr - Base class for all expressions in swift.
class alignas(8) Expr {...}
/// Pattern - Base class for all patterns in Swift.
class alignas(8) Pattern {...}
13 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

14. AST objects
/// Integer literal with a '+' or '-' sign, like '+4' or '- 2'.
///
/// After semantic analysis assigns types, this is guaranteed to have
/// a BuiltinIntegerType or be a normal type and implicitly be
/// AnyBuiltinIntegerType.
class IntegerLiteralExpr : public NumberLiteralExpr {
public:
/// Returns the value of the literal, appropriately constructed in the
/// target type.
APInt getValue() const;
/// Returns the raw value of the literal without any truncation.
APInt getRawValue() const;
...
};
14 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

15. Integrating libSyntax into
the compiler pipeline
15 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

16. 16 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

17. Goal
To have the Swift parser fully embrace libSyntax and
stop emitting the semantic AST
17 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

18. Problem
Semantic AST is completely ignored and wasted
when SwiftSyntax invokes the Swift parser to get the
libSyntax tree
18 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

19. Existing pipeline
19 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

20. Existing pipeline + SwiftSyntax
20 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

21. How's Swift parsed in
existing pipeline?
21 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

22. parseTopLevel
// ParseDecl.cpp
bool Parser::parseTopLevel() {
parseBraceItems(Items,
allowTopLevelCode() ? BraceItemListKind::TopLevelCode
: BraceItemListKind::TopLevelLibrary);
....
}
22 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

23. parseBraceItems
// ParseStmt.cpp
ParserStatus Parser::parseBraceItems(SmallVectorImpl<ASTNode> &Entries,
BraceItemListKind Kind,
BraceItemListKind ConditionalBlockKind) {
ASTNode Result;
ParserStatus Status = parseExprOrStmt(Result);
...
}
23 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

24. parseExpr
// ParseStmt.cpp
ParserStatus Parser::parseExprOrStmt(ASTNode &Result) {
ParserResult<Expr> ResultExpr = parseExpr(diag::expected_expr);
...
}
24 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

25. parseExpr
// ParseStmt.cpp
ParserStatus Parser::parseExprOrStmt(ASTNode &Result) {
ParserResult<Expr> ResultExpr = parseExpr(diag::expected_expr);
...
}
// ParseExpr.cpp
ParserResult<Expr> parseExpr(Diag<> ID) {
return parseExprImpl(ID, /*isExprBasic=*/false);
}
25 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

26. parseExpr
// ParseStmt.cpp
ParserStatus Parser::parseExprOrStmt(ASTNode &Result) {
ParserResult<Expr> ResultExpr = parseExpr(diag::expected_expr);
...
}
// ParseExpr.cpp
ParserResult<Expr> parseExpr(Diag<> ID) {
return parseExprImpl(ID, /*isExprBasic=*/false);
}
ParserResult<Expr> Parser::parseExprImpl(Diag<> Message,
bool isExprBasic) {
auto expr = parseExprSequence(Message, isExprBasic,
/*forConditionalDirective*/false);
...
}
26 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

27. parseExprSequence
// ParseExpr.cpp
ParserResult<Expr> Parser::parseExprSequence(Diag<> Message,
bool isExprBasic,
bool isForConditionalDirective) {
// Parse a unary expression.
ParserResult<Expr> Primary =
parseExprSequenceElement(Message, isExprBasic);
...
}
27 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

28. parseExprSequence
// ParseExpr.cpp
ParserResult<Expr> Parser::parseExprSequence(Diag<> Message,
bool isExprBasic,
bool isForConditionalDirective) {
// Parse a unary expression.
ParserResult<Expr> Primary =
parseExprSequenceElement(Message, isExprBasic);
...
}
ParserResult<Expr> Parser::parseExprSequenceElement(Diag<> message,
bool isExprBasic) {
return parseExprUnary(message, isExprBasic);
}
28 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

29. parseExprElement
// ParseExpr.cpp
ParserResult<Expr> Parser::parseExprUnary(Diag<> Message, bool isExprBasic) {
return parseExprPostfix(Message, isExprBasic);
}
29 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

30. parseExprElement
// ParseExpr.cpp
ParserResult<Expr> Parser::parseExprUnary(Diag<> Message, bool isExprBasic) {
return parseExprPostfix(Message, isExprBasic);
}
ParserResult<Expr> Parser::parseExprPostfix(Diag<> ID, bool isExprBasic) {
return parseExprPrimary(ID, isExprBasic);
}
30 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

31. parseIntegerLiteralExpr
// ParseExpr.cpp
ParserResult<Expr> Parser::parseExprPrimary(Diag<> ID, bool isExprBasic) {
// Syntax
SyntaxParsingContext ExprContext(SyntaxContext, SyntaxContextKind::Expr);
switch (Tok.getKind()) {
case tok::integer_literal: {
StringRef Text = copyAndStripUnderscores(Tok.getText());
SourceLoc Loc = consumeToken(tok::integer_literal);
// Syntax
ExprContext.setCreateSyntax(SyntaxKind::IntegerLiteralExpr);
// AST object
return makeParserResult(new (Context)
IntegerLiteralExpr(Text, Loc,
/*Implicit=*/false));
}
}
}
31 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

32. parseIntegerLiteralExpr
// ParseExpr.cpp
ParserResult<Expr> Parser::parseExprPrimary(Diag<> ID, bool isExprBasic) {
// Syntax
SyntaxParsingContext ExprContext(SyntaxContext, SyntaxContextKind::Expr);
switch (Tok.getKind()) {
case tok::integer_literal: {
StringRef Text = copyAndStripUnderscores(Tok.getText());
SourceLoc Loc = consumeToken(tok::integer_literal);
// Syntax
ExprContext.setCreateSyntax(SyntaxKind::IntegerLiteralExpr);
// AST object
return makeParserResult(new (Context)
IntegerLiteralExpr(Text, Loc,
/*Implicit=*/false));
}
}
}
32 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

33. parseIntegerLiteralExpr
// ParseExpr.cpp
ParserResult<Expr> Parser::parseExprPrimary(Diag<> ID, bool isExprBasic) {
// Syntax
SyntaxParsingContext ExprContext(SyntaxContext, SyntaxContextKind::Expr);
switch (Tok.getKind()) {
case tok::integer_literal: {
StringRef Text = copyAndStripUnderscores(Tok.getText());
SourceLoc Loc = consumeToken(tok::integer_literal);
// Syntax
ExprContext.setCreateSyntax(SyntaxKind::IntegerLiteralExpr);
// AST object
return makeParserResult(new (Context)
IntegerLiteralExpr(Text, Loc,
/*Implicit=*/false));
}
}
}
33 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

34. parseIntegerLiteralExpr
// ParseExpr.cpp
ParserResult<Expr> Parser::parseExprPrimary(Diag<> ID, bool isExprBasic) {
// Syntax
SyntaxParsingContext ExprContext(SyntaxContext, SyntaxContextKind::Expr);
switch (Tok.getKind()) {
case tok::integer_literal: {
StringRef Text = copyAndStripUnderscores(Tok.getText());
SourceLoc Loc = consumeToken(tok::integer_literal);
// Syntax
ExprContext.setCreateSyntax(SyntaxKind::IntegerLiteralExpr);
// AST object
return makeParserResult(new (Context)
IntegerLiteralExpr(Text, Loc,
/*Implicit=*/false));
}
}
}
34 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

35. parseBraceItems
// ParseStmt.cpp
ParserStatus Parser::parseBraceItems(SmallVectorImpl<ASTNode> &Entries,
BraceItemListKind Kind,
BraceItemListKind ConditionalBlockKind) {
SyntaxParsingContext ItemListContext(SyntaxContext,
SyntaxKind::CodeBlockItemList);
auto *TLCD = new (Context) TopLevelCodeDecl(CurDeclContext);
ASTNode Result;
ParserStatus Status = parseExprOrStmt(Result);
auto Brace = BraceStmt::create(Context, Result.getStartLoc(),
Result, Result.getEndLoc());
TLCD->setBody(Brace);
Entries.push_back(TLCD);
return Status;
}
35 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

36. parseBraceItems
// ParseStmt.cpp
ParserStatus Parser::parseBraceItems(SmallVectorImpl<ASTNode> &Entries,
BraceItemListKind Kind,
BraceItemListKind ConditionalBlockKind) {
SyntaxParsingContext ItemListContext(SyntaxContext,
SyntaxKind::CodeBlockItemList);
auto *TLCD = new (Context) TopLevelCodeDecl(CurDeclContext);
ASTNode Result;
ParserStatus Status = parseExprOrStmt(Result);
auto Brace = BraceStmt::create(Context, Result.getStartLoc(),
Result, Result.getEndLoc());
TLCD->setBody(Brace);
Entries.push_back(TLCD);
return Status;
}
36 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

37. parseTopLevel
// ParseDecl.cpp
bool Parser::parseTopLevel() {
SF.ASTStage = SourceFile::Parsing;
parseBraceItems(Items,
allowTopLevelCode() ? BraceItemListKind::TopLevelCode
: BraceItemListKind::TopLevelLibrary);
// Add newly parsed decls to the module.
SmallVector<ASTNode, 128> Items;
for (auto Item : Items)
if (auto *D = Item.dyn_cast<Decl*>())
SF.Decls.push_back(D);
// Note that the source file is fully parsed and verify it.
SF.ASTStage = SourceFile::Parsed;
verify(SF);
// If we are done parsing the whole file, finalize the token receiver.
if (Tok.is(tok::eof)) {
SyntaxContext->addToken(Tok, LeadingTrivia, TrailingTrivia);
TokReceiver->finalize();
}
}
37 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

38. parseTopLevel
// ParseDecl.cpp
bool Parser::parseTopLevel() {
SF.ASTStage = SourceFile::Parsing;
parseBraceItems(Items,
allowTopLevelCode() ? BraceItemListKind::TopLevelCode
: BraceItemListKind::TopLevelLibrary);
// Add newly parsed decls to the module.
SmallVector<ASTNode, 128> Items;
for (auto Item : Items)
if (auto *D = Item.dyn_cast<Decl*>())
SF.Decls.push_back(D);
// Note that the source file is fully parsed and verify it.
SF.ASTStage = SourceFile::Parsed;
verify(SF);
// If we are done parsing the whole file, finalize the token receiver.
if (Tok.is(tok::eof)) {
SyntaxContext->addToken(Tok, LeadingTrivia, TrailingTrivia);
TokReceiver->finalize();
}
}
38 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

39. parseTopLevel
// ParseDecl.cpp
bool Parser::parseTopLevel() {
SF.ASTStage = SourceFile::Parsing;
parseBraceItems(Items,
allowTopLevelCode() ? BraceItemListKind::TopLevelCode
: BraceItemListKind::TopLevelLibrary);
// Add newly parsed decls to the module.
SmallVector<ASTNode, 128> Items;
for (auto Item : Items)
if (auto *D = Item.dyn_cast<Decl*>())
SF.Decls.push_back(D);
// Note that the source file is fully parsed and verify it.
SF.ASTStage = SourceFile::Parsed;
verify(SF);
// If we are done parsing the whole file, finalize the token receiver.
if (Tok.is(tok::eof)) {
SyntaxContext->addToken(Tok, LeadingTrivia, TrailingTrivia);
TokReceiver->finalize();
}
}
39 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

40. parseTopLevel
// ParseDecl.cpp
bool Parser::parseTopLevel() {
SF.ASTStage = SourceFile::Parsing;
parseBraceItems(Items,
allowTopLevelCode() ? BraceItemListKind::TopLevelCode
: BraceItemListKind::TopLevelLibrary);
// Add newly parsed decls to the module.
SmallVector<ASTNode, 128> Items;
for (auto Item : Items)
if (auto *D = Item.dyn_cast<Decl*>())
SF.Decls.push_back(D);
// Note that the source file is fully parsed and verify it.
SF.ASTStage = SourceFile::Parsed;
verify(SF);
// If we are done parsing the whole file, finalize the token receiver.
if (Tok.is(tok::eof)) {
SyntaxContext->addToken(Tok, LeadingTrivia, TrailingTrivia);
TokReceiver->finalize();
}
}
40 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

41. Syntax tree creation &
AST generation
41 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

42. parseExprParen
ParserResult<Expr> Parser::parseExprParen() {
SyntaxParsingContext ParenCtxt(SyntaxKind::ParenExpr, SyntaxContext);
// In consumeToken(), a RawTokenSyntax is added to the context.
consumeToken(tok::l_paren)
// On returning from parseExpr(), a Expr Syntax node is created and added to the context.
ParserResult<Expr> Result = parseExpr();
consumeToken(tok::r_paren)
// Now the context holds { '(' Expr ')' }.
// From these parts, it creates ParenExpr node and add it to the parent.
return makeParserResult(new (Context) ParenExpr(Result.get()));
}
42 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

43. Syntax tree
ParserResult<Expr> Parser::parseExprParen() {
SyntaxParsingContext ParenCtxt(SyntaxKind::ParenExpr, SyntaxContext);
// In consumeToken(), a RawTokenSyntax is added to the context.
consumeToken(tok::l_paren)
// On returning from parseExpr(), a Expr Syntax node is created and added to the context.
ParserResult<Expr> Result = parseExpr();
consumeToken(tok::r_paren)
// Now the context holds { '(' Expr ')' }.
// From these parts, it creates ParenExpr node and add it to the parent.
return makeParserResult(new (Context) ParenExpr(Result.get()));
}
43 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

44. AST
ParserResult<Expr> Parser::parseExprParen() {
SyntaxParsingContext ParenCtxt(SyntaxKind::ParenExpr, SyntaxContext);
// In consumeToken(), a RawTokenSyntax is added to the context.
consumeToken(tok::l_paren)
// On returning from parseExpr(), a Expr Syntax node is created and added to the context.
ParserResult<Expr> Result = parseExpr();
consumeToken(tok::r_paren)
// Now the context holds { '(' Expr ')' }.
// From these parts, it creates ParenExpr node and add it to the parent.
return makeParserResult(new (Context) ParenExpr(Result.get()));
}
44 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

45. parseExprParen
ParserResult<Expr> Parser::parseExprParen() {
SyntaxParsingContext ParenCtxt(SyntaxKind::ParenExpr, SyntaxContext);
// In consumeToken(), a RawTokenSyntax is added to the context.
consumeToken(tok::l_paren)
// On returning from parseExpr(), a Expr Syntax node is created and added to the context.
ParserResult<Expr> Result = parseExpr();
consumeToken(tok::r_paren)
// Now the context holds { '(' Expr ')' }.
// From these parts, it creates ParenExpr node and add it to the parent.
return makeParserResult(new (Context) ParenExpr(Result.get()));
}
45 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

46. How optionally creating
syntax tree?
46 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

47. LangOptions
/// A collection of options that affect the language dialect and
/// provide compiler debugging facilities.
class LangOptions {
/// Whether to parse syntax tree. If the syntax tree is built, the generated
/// AST may not be correct when syntax nodes are reused as part of
/// incrementals parsing.
bool BuildSyntaxTree = false;
}
47 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

48. SyntaxParsingContext
class alignas(1 << SyntaxAlignInBits) SyntaxParsingContext {
// If false, context does nothing.
bool Enabled;
...
}
/// Add Token with Trivia to the parts.
void SyntaxParsingContext::addToken(Token &Tok,
const ParsedTrivia &LeadingTrivia,
const ParsedTrivia &TrailingTrivia) {
if (!Enabled)
return;
...
}
/// Add Syntax to the parts.
void SyntaxParsingContext::addSyntax(ParsedSyntax Node) {
if (!Enabled)
return;
...
}
48 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

49. SyntaxParsingContext
class alignas(1 << SyntaxAlignInBits) SyntaxParsingContext {
// If false, context does nothing.
bool Enabled;
...
}
/// Add Token with Trivia to the parts.
void SyntaxParsingContext::addToken(Token &Tok,
const ParsedTrivia &LeadingTrivia,
const ParsedTrivia &TrailingTrivia) {
if (!Enabled)
return;
...
}
/// Add Syntax to the parts.
void SyntaxParsingContext::addSyntax(ParsedSyntax Node) {
if (!Enabled)
return;
...
}
49 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

50. Existing pipeline + SwiftSyntax
50 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

51. Recap
51 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

52. Recap
→ Syntax tree is created by SyntaxParsingContext
51 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

53. Recap
→ Syntax tree is created by SyntaxParsingContext
→ AST is generated while syntax tree is created
51 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

54. Recap
→ Syntax tree is created by SyntaxParsingContext
→ AST is generated while syntax tree is created
→ if Enabled, parser creates syntax tree AND
generates AST
51 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

55. Goal
To have the Swift parser fully embrace libSyntax and
stop emitting the semantic AST
52 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

56. New pipeline
53 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

57. How's Swift pared in new
pipeline?
54 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

58. Parser
55 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

59. ParsedSyntaxBuilders
class ParsedParenExprSyntaxBuilder {
public:
explicit ParsedParenExprSyntaxBuilder(SyntaxParsingContext &SPCtx)
: SPCtx(SPCtx) {}
ParsedParenExprSyntaxBuilder &useLeftParen(ParsedTokenSyntax LeftParen);
ParsedParenExprSyntaxBuilder &useExpression(ParsedExprSyntax Expression);
ParsedParenExprSyntaxBuilder &useRightParen(ParsedTokenSyntax RightParen);
ParsedParenExprSyntax build();
...
};
56 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

60. Parser
ParsedSyntaxResult<Expr> Parser::parseExprParenSyntax() {
ParsedParenExprSyntaxBuilder builder(*SyntaxContext);
// (
auto leftParen = consumeTokenSyntax(tok::l_paren);
builder.useLeftParen(std::move(leftParen));
// Expr
auto expression = parseExpressionSyntax();
builder.useExpression(expression);
// )
auto rightParen = consumeTokenSyntax(tok::r_paren);
builder.useRightParen(std::move(rightParen));
return makeParsedResult(builder.build());
}
57 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

61. Parser
ParsedSyntaxResult<Expr> Parser::parseExprParenSyntax() {
ParsedParenExprSyntaxBuilder builder(*SyntaxContext);
// (
auto leftParen = consumeTokenSyntax(tok::l_paren);
builder.useLeftParen(std::move(leftParen));
// Expr
auto expression = parseExpressionSyntax();
builder.useExpression(expression);
// )
auto rightParen = consumeTokenSyntax(tok::r_paren);
builder.useRightParen(std::move(rightParen));
return makeParsedResult(builder.build());
}
58 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

62. Parser
ParserResult<Expr> Parser::parseExprParen() {
auto parsed = parseExprParenSyntax();
SyntaxContext->addSyntax(parsed.get());
...
}
59 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

63. ASTGen
60 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

64. ASTGen
/// Generates AST nodes from Syntax nodes.
class Parser;
class ASTGen {
public:
Expr *generate(const syntax::ExprSyntax &Expr, const SourceLoc Loc);
Expr *generate(const syntax::ParenExprSyntax &Expr, const SourceLoc Loc);
...
}
61 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

65. ASTGen
ParserResult<Expr> ASTGen::parseExprParen() {
auto syntax = SyntaxContext->topNode<ParenExprSyntax>();
auto result = generate(syntax, leadingLoc);
return makeParserResult(result);
}
62 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

66. ASTGen
ParserResult<Expr> ASTGen::parseExprParen() {
auto syntax = SyntaxContext->topNode<ParenExprSyntax>();
auto result = generate(syntax, leadingLoc);
return makeParserResult(result);
}
63 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

67. Parser + ASTGen
64 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

68. Benefits
65 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

69. Benefits
→ Robust architecture
65 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

70. Benefits
→ Robust architecture
→ SwiftSyntax parsing performance improvements
65 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

71. Benefits
→ Robust architecture
→ SwiftSyntax parsing performance improvements
→ Enable future work for making the compiler
pipeline more suitable for interactive contexts
65 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

72. Pipeline + SwiftSyntax
66 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

73. Current scope
Parser should be fully converted first to have layered
architecture for syntax tree and AST
67 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

74. Current scope
68 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

75. Parser
ParsedSyntaxResult<Expr> Parser::parseExprParenSyntax() {
ParsedParenExprSyntaxBuilder builder(*SyntaxContext);
// (
auto leftParen = consumeTokenSyntax(tok::l_paren);
builder.useLeftParen(std::move(leftParen));
// Expr
auto expression = parseExpressionSyntax();
builder.useExpression(expression);
// )
auto rightParen = consumeTokenSyntax(tok::r_paren);
builder.useRightParen(std::move(rightParen));
return makeParsedResult(builder.build());
}
69 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

76. Parser
ParserResult<Expr> Parser::parseExprParen() {
auto parsed = parseExprParenSyntax();
SyntaxContext->addSyntax(parsed.get());
auto syntax = SyntaxContext->topNode<ParenExprSyntax>();
auto result = Generator.generate(syntax, leadingLoc);
return makeParserResult(result);
}
70 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

77. TODOs
71 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

78. TODOs
→ Performance issue
71 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

79. TODOs
→ Performance issue
→ Module separation
71 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

80. TODOs
→ Performance issue
→ Module separation
→ SwiftSyntax hookup
71 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

81. syntax-parse branch
https://github.com/apple/swift/tree/
syntax-parse
72 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

82. Summary
73 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

83. Summary
→ libSyntax integration is still WIP
73 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

84. Summary
→ libSyntax integration is still WIP
→ Definitely better change for many reasons
73 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)

85. Summary
→ libSyntax integration is still WIP
→ Definitely better change for many reasons
→ Huge performance improvements for SwiftSyntax
73 — Integrating libSyntax into the compiler pipeline, Yusuke Kita (@kitasuke)