Sharing three EMNLP 2020 papers on conversational semantic parsing on Watch Party

1. Jie Cao
Dec 04, 2020
*many content are borrowed from the original papers
Task-Oriented Conversational
Semantic Parsing
EMNLP 2020 Watch Party@Amazon Lex

2. Outlines
• Background
• Related Works
• Recent Advances on representation
• Conversational Semantic Parsing[1](Facebook)
• Conversational Semantic Parsing for Dialog State Tracking[2](Apple)
• Task-oriented Dialogue as Data
fl
ow Synthesis[3](Microsoft Semantic Machines)
• Summary

3. Background
Conventional Task-oriented Dialog System
Key Issues on Intent/Slot Fillin
g
◦ Poor Scalabilit
y
◦ Unseen intent/slot/slot values(even the same
domain
)
◦ Lacking knowledge sharing across domai
n
◦ Poor Compositionalit
y
◦ Complex intent/slot/system ac
t
◦ Multiple intent
s
◦ Nested intent/slo
t
Other Issues
:
◦ Dialog State Tracking Issue
s
◦ Coreferenc
e
◦ Multi-domain: slot carryove
r
◦ Dialog Polic
y
◦ Complex actio
n
◦ Low Resource

4. Related Works
A. delexiconlization with semantic dictionaries[4][5]
B. neural belief tracker[6][7]
C. dual-strategy, generative DST(DST as QA)[8,9,10,11]
D. Zero(few)-shot
E. ….
Better modeling methods
◦ Poor Scalabilit
y
◦ Dialog State Tracking Issues

5. Related Works
A. For intent/slot tags
• Decomposable multipoint representation for intent/slot
names[2,11,12]
• Schema-guided dialog (supported with natural language
description)[13,14]
B. For better Intent/Slot composition
A. Hierarchical Representation[1,2,3,15,16]
C. Beyond Intent/Slot Representation[3]
Better representation design
◦ Poor Scalabilit
y
◦ Poor Compositionalit
y
◦ Dialog State Tracking/Policy Issue
s

6. Conversational Semantic Parsing[1] (Facebook, SBTOP)
• Utterance-level Hierarchical Intent/Slot Representation[15, 16](TOP, TOPV2)
Background

7. Conversational Semantic Parsing[1] (Facebook, SBTOP)
Background(utterance-level TOP)
Pros:
1. Hierarchical queries
2. Easy Annotation: labeling the span anchors
3. Easy parsing: constituent tree parsing
4. Compatible(following traditional intent/slot framework)
Cons:
1. Only utterance level (TOP, TOPV2)
2. In-order constraint
1. Must reconstruct the sentence.
3. Toy dataset:
1. Shallow tree (2.54 avg depth)
2. Short sentences(9 tokens per utterance)
3. Few domains(2 in TOP, 6 new in TOPV2)
4. Limited Composition
1. Support only nested intent, not conjunction for multiple
intents.

8. Conversational Semantic Parsing[1] (Facebook, SBTOP)
Limitations of In-order constraint:
1.Discontinuous
2.Strict Word Order
3.Not scalable to Session-based
1.Intent, dialog recovery
On Monday, set an alarm for 8am [SL DATETIME 8am on Monday]
Solutions: Decouple form
•removing all text that does not
appear in a leaf slot.
•Easy for aggerating for session-
based

9. Conversational Semantic Parsing[1] (Facebook, SBTOP)
Session-based hierarchical representation
Additional Support for Session-based: extra REF label
• Coreferences (REF: EXPLICIT)
• Slot-carryover (REF: IMPLICIT)
what artist is this ? | this is mozart opus 3 | what movement is this
[IN:QUESTION_MUSIC
[SL:MUSIC_TYPE movement ]
[SL:REF_IMPLICIT [IN:GET_REF
[SL:MUSIC_ALBUM_TITLE opus 3 ] ] ]
[SL:REF_IMPLICIT [IN:GET_REF
[SL:MUSIC_ARTIST_NAME mozart ] ] ] ]
Session-based aggregation
[IN:QUESTION_MUSIC_ARTIST
[SL:MUSIC this]]

10. Conversational Semantic Parsing[1] (Facebook, SBTOP)
Take-aways
1.Main Goal: TOP -> Session-based TOP
2.Main contributions:
1.In-order constraint blocks the session-based: resolved by decouple form
2.Additional Support for Session-based: extra REF label
Remaining Issues:
1. Very Poor dataset:
1. Few domains(2 in TOP, 8 in TOPV2, only 4 in SBTOP)
2. Short dialog, as the table 4 statistics
3. Low quality annotation
1. 55% annotator agreement, 94% parsing correct?
2. Limited Composition:
1. Only nested intent, no nested slot
2. No conjunction

11. Conversational Semantic Parsing for Dialog State Tracking[2](Apple)
Main Issues on
fl
at representation
• Poor expressiveness in multiple levels
• Intent/slot representation,
fl
at name tags
• Slot value(nested properties)
• No conjunctions and nested intents.
• Session-based
• Coreference/ Slot CarryOver

12. Conversational Semantic Parsing for Dialog State Tracking[2](Apple)
1. Hierarchical intent/slot names by semantic decoupling
I want a
fl
ight ticket departure at 5 AM tomorrow
1.Context-aware turn-level representatio
n
• both user and system tur
n
2.Non-terminals
:
• domains: a group of activitie
s
• verbs: used for a user turn, the verb part of s inten
t
• actions: used for a system turn, the dialog act to respond the user
.
• slot
s
• operators: equals
• types: Person, Time, Location
.
3. Terminal value nod
e
• categorical value e.g day of-week
• open value (in context, anchored)
• reference node

13. Conversational Semantic Parsing for Dialog State Tracking[2](Apple)
2. Nested/Conjunction properties(e.g time range): slot-operator-(argument1, argument2)
argument:
1.sub-slot (time in date, hour in time
)
2.terminal value nod
e
1. Canonical categorical label , e.g. day of wee
k
3.referece nod
e
1.reference to a whole intent(nested intent to
fi
nd the event
fi
rst
)
2.reference to co-reference in the previous turn: sub-tree copy

14. Conversational Semantic Parsing for Dialog State Tracking[2](Apple)
Take-aways
• Hierarchical in multiple levels
• semantic decomposition for intent/slot name
• (domain.verb.slot)
• slot-operator-(argument1, argument2)
• Nested intent(slot-intent), conjunctions
• Support nested slots(slot-slot)
• Session-based
• User-turn-level state and system act, context-aware
• Reference to intent
• Copy subtree from previous existed state(inline)
no (intent-intent) nested cases ?
Cons
1. Seems not much semantic for operators now: only
“equal”
2. Still focus on intent/slot value state, not a meta
computation graph
3. Their experiments didn’t investigate the impact of
semantic decomposition
4. Intent/slot name decomposition may be not easy to
deploy for large amount of services.

15. Task-oriented Dialogue as Dataflow
Synthesis[3]
• At each turn, translate the most recent user
utterance into a program (Not a resultant
value, or meaning for user utterance).
• The predicted program is direct contextual
appropriate (executable) response
• Predicted programs nondestructively
extend the data
fl
ow graph
Beyond Intent-Slot framework
ASR
TTS
Data
fl
ow Synthesis
Generation
New Pipelines
Common Ground
Data
fl
ow
U1
P1
U2
P2
S1 S2
….
U_n
P_n
S_n

16. Task-oriented Dialogue as Dataflow
Synthesis[3]
Reference: refer to previous entity
Predicted Program
• Solid border: return program value
• Refer to some salient previously mentioned node
Data-
fl
ow graph
• Shaded node means evaluated
• Evaluated node has a dashed result edge
• Exception will cause unevaluated nodes
dayOfWeek
refer
Here refer will try to
fi
nd a previous node with constraints(DataTime type),
Here, it is the top-level result of evaluated start node
Constraints:
Type Constraint:
refer(Constraint[Event]())
Property Constraint:
refer(Constraint[Event](date= Constraint[DateTime](weekday=thurs)))
Role Constraint: (keyword named argument, like slot or subplot)
refer(RoleConstraint([date,weekday])).

17. Task-oriented Dialogue as Dataflow
Synthesis[3]
Revision: refer to subgraph Predicted Program
• Solid border: return program value
• Refer to some salient previously mentioned node
• Light gray means previous program
Data-
fl
ow graph
• Shaded node means evaluated in order
• Evaluated node has a dashed result edge
• Exception will cause unevaluated nodes
Revisie operator take three arguments
• rootLoc, a constraint to
fi
nd the top-level
node of the original computation;
• oldLoc, a constraint on the node to replace
within the original computation;
• new, a new graph fragment to substitute there.
The
fi
nal result is the root of revised subgraph,
the new start node
New nodes will be re-evaluated
fi
nally
Recover is implemented Revision

18. Task-oriented Dialogue as Dataflow
Synthesis[3]
Take-aways
ASR
TTS
Data
fl
ow Synthesis
Generation
New Pipelines
Common Ground
Data
fl
ow
U1
P1
U2
P2
S1 S2
….
U_n
P_n
S_n
• Translate the most recent user utterance into a program
• Not a resultant value, or meaning for user utterance).
• The predicted program is direct contextual appropriate (executable)
response
• Predicted programs nondestructively extend the data
fl
ow graph
• Graph node are evaluated in order once new predicted program added in
• Saving evaluated values for quick reference value
• Saving meta graph for revision to subgraphs
• Recover and revision

19. Summary
• Previous work are mainly about
fl
at frame presentation with intent/slot
• All three papers are dialog hierarchical presentation (session-based,
compositional)
• SBTOP and TreeDST follow the intent/slot presentation
• While Data
fl
ow exploit program transformation to translate utterance into
program then build data-
fl
ow graph
Symbol
Semantic
Intent/slot composition Act Session-based
Name
decomposition
n
Intent
Conjunc
tion
Slot-intent
nested
Slot-
subslot
nested
System
act
Corefere
nce
Carryover
Meta-
computation
SBTOP N N Y N N Y Y N
TreeDST Y Y Y Y Y Y Y N
Data
fl
ow* N Y Y Y Y Y Y Y
* Data
fl
ow are not strictly comparable with intent/slot framework

20. Q&A
Thanks!

21. References
1. Aghajanyan, Armen, et al. "Conversational Semantic Parsing." Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.
https://www.aclweb.org/anthology/2020.emnlp-main.408.pd