A very short (5-min) talk on important things in AI now. Given at Global AI Student conference 2020

1. What’s new in AI in 2020
Grigory Sapunov – CTO, Intento
@che_shr_cat

2. Transformers
● Large Models, including cross-lingual
○ GPT-3 (up to 175B parameters), ruGPT-3 (1.3B+), Chinese CPM
(2.6B)
○ T5 (up to 11B), mT5 (up to 13B, multilingual)
○ BART, mBART (680M, multilingual)
○ Turing-NLG (17B), T-ULRv2 (multilingual, 550M)
● Architectural innovations
○ Efficiency: Reformer, Linformer, Longformer, Big Bird,
Performer, Axial Transformers
○ Images: iGPT, Vision Transformer (ViT), image processing
transformer (IPT)
○ Memory: Compressive Transformer, memory-augmented
models
○ many other improvements!

3. Biology’s “ImageNet moment”
AlphaFold 2

4. Hardware Innovations
● NVIDIA Ampere A100
● NVIDIA+ARM
● Google TPU v4, Edge TPU
● Apple M1
● Neuromorphic (Intel Loihi)
● Habana
● Graphcore
● Cerebras
● etc

5. Cloud AI — Democratizing AI
● 100s of AI APIs for NLP,
Computer Vision, Video,
Speech, Structured Data,
and other cases
● Customized models in
the Cloud — you can
train your own models
with an easy to use GUI.

6. AI Ethics
● Face recognition bans
● Malicious use of AI
● Biases in ML models
● Interpretability &
Explainability
● ...