Deep learning architectures for the conversational intelligence.

1. iPavlov:
Deep learning architectures for
the conversational intelligence
Mikhail Burtsev, PhD
Neural Networks and Deep Learning Lab,
Moscow Institute of Physics and Technology
(MIPT)

2. Neural Networks and
Deep Learning Lab, MIPTR&D Goals and Directions
06/06/2017 2
• Conversational Intelligence
- Complex real world problem
- Can be decomposed into simpler tasks
- Big amount of data is available
- Immediate application in industry
- A step towards solving AI
• General approach is development of deep learning architectures combining :
- recurrent neural networks for the generation of sequences and memory,
- reinforcement learning for the dialogue planning .

3. Neural Networks and
Deep Learning Lab, MIPTiPavlov
06/06/2017 3
Deep learning architectures for the conversational
intelligence
• Duration of the project is 2017-2020
• Joint project with Sberbank the largest bank in Russia
(operating income $20 billion, total assets $400 billion (2014))
• Budget of the project is $8 mln
• Up to 20 researchers and engineers

4. Neural Networks and
Deep Learning Lab, MIPTiPavlov
06/06/2017 4
• Technology outcomes
- Opensource deep
learning NLP library
DeepPavlov.
- AI platform DeepReply
implementing NLP
services on top of
DeepPavlov library for
the chat-bot and
dialogue systems
products.
Technology
Stack
Project
Outcome
Description Examples
AI APPLICATIONS
Out of the scope
of iPavlov project
Third party AI applications
in the domain of
conversational
intelligence.
Google Now,
Digital Genius
AI SERVICES DeepReply
AI conversational services
on top of the neural
network models trained
for specific domains.
API.ai, wit.ai,
Google NLP API
DEEP LEARNING
ARCHITECTURES
DeepPavlov
Core components for
neural conversational
intelligence. Basic NLP
functions and major
neuroarchitectures for the
dialogue systems.
MemNN, HRED
CORE DEEP
LEARNING
ALGORITHMS
CNN, RNN,
LSTM, RL
COMPUTATIONAL
LIBRARIES
Out of the scope
of iPavlov project
ThensorFlow,
Torch
DRIVERS GPU/FPGA
C/C++,Python,
Julia
CPU/GPU/FPGA
NVIDIA GPU, Intel
CPU, Google TPU

5. Neural Networks and
Deep Learning Lab, MIPTiPavlov project
06/06/2017 5
• R&D Roadmap
Integrated Neuroarchitectures
Memory
Short-term
memory
Episodic
memory
Working
memory
Goal-
directed
behaviour
Reinforcement
learning
Functional
systems New
research
directionsNeuroevolution
Neurogenesis

6. web:
iPavlov.ai
06/06/2017 6