10. 陳昇瑋 / 台灣產業 AI 化 - 如何跨出第一步?
機器學習
27
Training a prediction machine by
showing examples instead of
programming it.
-Yann LeCun
(prediction machine: 可基於已知預測未知的數學模型)
11. 陳昇瑋 / 台灣產業 AI 化 - 如何跨出第一步?
機器學習的定義
28
讓電腦能從資料裡頭淬取出
規則的演算法。
Find the common patterns
from the left waveforms
It seems impossible to
write a program for
speech recognition
你好 你好
你好 你好
You quickly get lost in the
exceptions and special cases.
(Slide Credit: Hung-Yi Lee)
12. 陳昇瑋 / 台灣產業 AI 化 - 如何跨出第一步?
就放棄教電腦規則,讓它自己學吧!
你好
大家好
人帥真好
You said
“你好”
很多訓練資料
機器學習演算法
從訓練資料中
找到規則
(Slide Credit: Hung-Yi Lee)
13. 陳昇瑋 / 台灣產業 AI 化 - 如何跨出第一步?
機器學習學到的規則跟你想的不太一樣
31
35. 0.95
F-score
Algorithm Ophthalmologist
(median)
0.91
“The study by Gulshan and colleagues
truly represents the brave new world in
medicine.”
“Google just published this paper in
JAMA (impact factor 44.405) [...] It
actually lives up to the hype.”
Dr. Andrew Beam, Dr. Isaac Kohane
Harvard Medical School
Dr. Luke Oakden-Rayner
University of Adelaide
36. 陳昇瑋 / 人工智慧在台灣
Deep Learning for Detection of Diabetic Eye Disease
76
Algorithm’s F1-score: 0.95
Median F1-score of 8 ophthalmologists : 0.91
43. 陳昇瑋 / 人工智慧在台灣 87
Deep Learning for Kidney Function Classification and
Prediction using Ultrasound-based Imaging
Chin-Chi Kuo1, Chun-Min Chang2, Kuan-Ting Liu2, Wei-Kai Lin2,
Chih-Wei Chung1, and Kuan-Ta Chen2
1Big Data Center, China Medical University Hospital, China Medical University, Taichung, Taiwan
2Institute of Information Science, Academia Sinica, Taiwan
eGFR
(腎功能指數)
45. Cardiologist-Level Arrhythmia Detection with
Convolutional Neural Networks
94
Goal: diagnose irregular heart rhythms, also known as
arrhythmias, from single-lead ECG signals better than a
cardiologist
46. 陳昇瑋 / 人工智慧在台灣
Input and Output
Input: a time-series of raw ECG signal
The 30 second long ECG signal is sampled at 200 Hz
From 29,163 patients
Output: a sequence of rhythm classes
The model outputs a new prediction once every second
Total 14 rhythm classes are identified
95
48. 陳昇瑋 / 人工智慧在台灣
Model
34 layers NN
16 residual blocks
2 conv layers per block
Filter length = 16 samples
# filter = 64*k, k start from 1 and is
incremented every 4-th residual block
Every residual block subsamples
its input by a factor of 2
97
51. Predictive tasks for healthcare
Given a large corpus of training data of de-identified medical records, can we
predict interesting aspects of the future for a patient not in the training set?
● will patient be readmitted to hospital in next N days?
● what is the likely length of hospital stay for patient checking in?
● what are the most likely diagnoses for the patient right now? and why?
● what medications should a doctor consider prescribing?
● what tests should be considered for this patient?
● which patients are at highest risk for X in next month?
Collaborating with several healthcare organizations, including UCSF,
Stanford, and Univ. of Chicago. Have early promising results.
79. 陳昇瑋 / 人工智慧民主化在台灣
Natural Language Processing (NLP): Some Sentence
Generation Examples by GTP-2
149
Bad news:
The technology can be used in generating fake news
Transformer based
(Slide credit: HT Kung)
80. 陳昇瑋 / 人工智慧民主化在台灣
More text generation samples
150https://openai.com/blog/better-language-models/
81. 陳昇瑋 / 人工智慧民主化在台灣
BERT and GPT-2
151
● Both BERT (from Google) and GPT-2 (OpenAI) are general purpose
pretrained NLP Feature Extractors based on the Transformer trained on
enormous amounts of text data.
● These models can be fine-tuned on small-data NLP tasks (like question
answering), resulting in substantial accuracy improvements compared to
training on these datasets from scratch
Pretrained BERT/GPT-2
Additional classifier for
your own tasks
The pretrained feature
extractor
Based on
Transformer
82. 陳昇瑋 / 人工智慧民主化在台灣
Advances of NLP Models in Recent Years
152
Deeper models, Larger Datasets
Better Features!
Performance Improvement: ~20%
Reuse of pre-training BERT/GPT-2: models.
Impact is similar to that of ImageNet in computer vision
Paper on theTransformer:Vaswani, A. et al, “Attention Is AllYou Need,” NIPS 2017
One-hot
Word
Embedding
ELMo BERT GPT-2
milestone
Transformer
based
BooksCorpus
(800M words) +
Wikipedia
(2,500M words)
scraped content from
the Internet of 8
million web pages
WMT 2011
(800M words)
WMT 2011
(800M words)
Large-scale dataset like ImageNet
RNN
based
Development of pretrained
‘feature extractor’ for NLP tasks:
89. 陳昇瑋 / 人工智慧民主化在台灣
Typical Applications of RL
Play games: Atari, poker, Go, ...
Explore worlds: 3D worlds, Labyrinth, ...
Control physical systems: manipulate, walk, swim, ...
Interact with users: recommend, optimize, personalize, ...
167
(Slide credit: David Silver)
90. 陳昇瑋 / 人工智慧民主化在台灣
More RL Applications
Flying Helicopter
Driving
Google Cuts Its Giant Electricity Bill With DeepMind-Powered AI
Parameter tuning in manufacturing lines
Text generation
Hongyu Guo, “Generating Text with Deep Reinforcement Learning”, NIPS, 2015
Marc'AurelioRanzato,SumitChopra,Michael Auli,Wojciech Zaremba, “Sequence Level
Training with Recurrent Neural Networks”, ICLR, 2016
168(Slide Credit: Hung-Yi Lee)
94. 陳昇瑋 / 人工智慧民主化在台灣 174
Mobile computing, inexpensive sensors collecting terabytes of data, and
the rise of machine learning that can use that data will fundamentally
change the way the global economy is organized.
- Fortune, “CEOs:The Revolution is Coming,” March 2016
110. Convolution Neural Networks + Transfer
Learning
Pre-trained using 14-million image dataset
ResNet with > 8-million parameters
Input
images
Model training /
inference
OK
OK
以深度學習進行自動瑕疵檢測
116. 台灣人工智慧學校首屆開學典禮
Especially important for equipment with high failure cost (such as motors in machine
tools)
Also important for expensive consumables (such as blades used in precision cutting
machines)
204
產業共通挑戰 #3-預測性維護
169. 陳昇瑋 / 人工智慧民主化在台灣
Execute pilot projects to gain momentum
Build an in-house AI team
Provide broad AI training
Develop an AI strategy
Develop internal and external communications
296
170. 陳昇瑋 / 人工智慧民主化在台灣
AI 導入三步驟
步驟 一
借助外部資源
步驟二
利用外部資源,
培養內部 AI 團隊
步驟三
將內部 AI 團隊與
各個有效結合
297
CEO
AI 業務部門 業務部門 業務部門
AI 培訓
171. 陳昇瑋 / 人工智慧民主化在台灣
State of AI In The Enterprise, 2018
303
Deloitte interviewed 1,100 IT and line-of-business executives from US-based companies
in the 3rd quarter of 2018.
82% of enterprise AI early adopters are seeing a positive ROI from their production-level
projects this year.
69% of enterprises are facing a “moderate, major or extreme” skills gap in finding skilled
associates to staff their new AI-driven business models and projects.
63% of enterprises have adopted machine learning, making this category the most
popular of all AI technologies in 2018.
190. 陳昇瑋 / 從大數據走向人工智慧
持續的團隊支援
330
A common data platform and workflow is
crucial for enterprise success.
Data Engineer ML Engineer Biz Analyst DevOps DevOps +
ML Engineer
App
Developer
(Credit: IBM Systems Lab Services)
(all under the supervision of Data Scientist)
191. 陳昇瑋 / 從大數據走向人工智慧
AI vs. BI
AI systems suggest decisions for users by making
predictions
BI systems support users make decisions based on data
visualization
Key difference
AI systems are based on generalizable models
BI systems require humans to generalize
Best practice
AI: fast, massive, error-tolerant, ML-capable problems
BI: otherwise
AI+BI: making sense of AI decisions
333
197. 陳昇瑋 / 人工智慧民主化在台灣
What we can and cannot today
What we can have
Safer car, autonomous car
Better medical image analysis
Personalized medicine
Adequate language translation
Useful but stupid chatbots
Information search, retrieval,
filtering
Numerous applications in
energy, finance, manufacturing,
commerce, law, …
What we cannot have (yet)
Machine with common sense
Intelligent personal assistants
“Smart” chatbots
Household robots
Agile and dexterous robots
Artificial General Intelligence
(AGI)
359
(Credit:Yann LeCun)
198. 陳昇瑋 / 人工智慧民主化在台灣
Strong AI Weak AI
Can think
Own conscious
Act as it can think
Consciousless
(1980)
201. 陳昇瑋 / 人工智慧民主化在台灣
AI Don’t Know What They are Talking About
372
https://www.facebook.com/playgroundenglish/videos/629372370729430/?hc_ref=ARQ
HCaS2GZ9jUgZermEupF5yerADq2X9F9P40OR3n70poUiCy7R0X3oHrGxyLSrWVdI
202. Change is the only constant.
- Heraclitus (535 BC - 475 BC)
(Slide Credit: Albert Chen)