What is Smart Speaker MEMS Microphone Array Technologies Cloud Voice Service

1. 智能音箱技術簡介
Victor Sue

2. Agenda
• What is Smart Speaker
• MEMS Microphone Array Technologies
• Cloud Voice Service

3. Agenda
• What is Smart Speaker
• MEMS Microphone Array Technologies
• Cloud Voice Service

4. What is Smart Speaker?

5. What is Smart Speaker? (cont.)
"Alexa, what's the
weather?"
"Alexa, show my
calendar."
"Alexa, show me my
timers."
"Alexa, play some
music."
"Alexa, read my
notifications"
"Alexa, what's in the
news?"
"Alexa, find me a nearby
pizza restaurant"
"Alexa, ask Uber to
request a ride"
Voice Portal

6. What is Smart Speaker? (cont.)

7. What is Smart Speaker? (cont.)
ASR
STT
TTS
Hotword
Detection
MEMS
MIC
MIC
Array
NLP
DNN
RNN
DOA
VAD
Audio
Beamfr-
oming
SVM

8. VAD (Voice activity detection)

9. Hotword Detection
Raw Audio
16KHz
16bit
mono
Precise
Engine
Combine
Score
Pre-Emphasis
Framing and
Windowing
FFT
Mel Filter
Bank
Logarithm DCT MFCC Features
MFCC
DNN
Feature Extraction
Posterior Handling
Deep Neural Network

10. Agenda
• What is Smart Speaker
• MEMS Microphone Array Technologies
• Cloud Voice Service

11. Microphone Application
Laptop PC
Mobile Phone
Digital Camera
Gaming
Voice Assistant

12. ECM Microphone
 A type of electrostatic capacitor-based microphone which eliminates the need for
a polarizing power supply by using a permanently charged material.
 Built-in a FET as the amplifier.
 Condenser diaphragm and plastic chassis are heat sensitive and should be careful
while soldering.
ECM Mic Circuit
ECM Mic Appearance ECM Mic Structure

13. MEMS Microphone
 3 types of MEMS microphone technology
 Piezoelectric (壓電式) (low sensitivity, high system noise)
 Piezoresistive (壓組式) (low sensitivity, high system noise)
 Capacitive (high sensitivity, low power consumption and low system noise)
 Capacitive type of MEMS is the main stream in the market.

14. ECM Mic. vs MEMS Mic.
 MEMS mics could offer the same SNR with much smaller size compare to traditional ECM mics.
 MEMS mics also provide much better consistent response to sound across all operating temperatures.
Reference of Eetimes: http://www.eetimes.com/document.asp?doc_id=1280170

15. ECM Mic. vs MEMS Mic.
Reference of DIGITIMES
比較項目 MEMS麥克風 ECM麥克風
元件尺寸 較小 較大
組裝方式 SMT自動組裝 人工組裝為主
操作溫度 可至攝氏200度以上 攝氏85度以上失真
防震抗撞 優 差
防EMI 優 差
防RFI 優 差
產品價格 較高 較低

16. MEMS Microphone Advantage
• MEMS Mic vs. Traditional Mic
• Increase performance
• Enhance manufacturability
• Reduce size
• Reduce costs

17. MEMS Microphone Suppliers
 Knowles
 InvenSense
 STMicroelectronics
 AAC Technologies
 Goertek

18. MEMS Microphone Market

19. MEMS Microphone Market (cont.)
• MEMS Microphone Applications

20. Microphone Array System diagram

21. Microphone Array Key Feature
• Higher Directivity
• Higher SNR
• Full Receiving Range
• Locate the Signal Source
• Smaller Size
• Acoustic Software Technologies
• Echo Cancellation
• Beamforming
• Sound Source localization
• Sensor Assistant

22. Beamforming Microphone Design
• Beamforming Filter(Delay of Sum)

23. Beamforming Microphone Design (cont.)
• Beamformer is the Spatial Filtering.
• Base on these two idea
• Narrow band signal
• Far field plane wave
• Selectively amplify a sound source
at a particular location
• Take advantage of sound
propagation through space
• Use Delay Sum Beamforming

24. Beamforming Microphone Design (cont.)
• Endfire Microphone array
• Algorithm: sum of signal in front microphone
and inverted delay rear microphone.
• Distance: microphones shall be match to
sample rate for correct delay of sampling
Distance: Sound speed * sample time * n of sample
• Pattern: cardioid :180o : no Signal
( frequency < Aliasing frequency )
34300*1/(48000)*3=2.14
(cm)
A
B
SUM

25. Beamforming Microphone Design (cont.)
2 Microphone endfire
6db 12db

26. 42.5mm
1
2
3
4
5
6
Beamforming Microphone placement design
• 2-Order Endfire Array *6
• Sample Rate: 22Khz
• Distance: 42.5mm (3 sample delay)
• Null Frequency : 4.2Khz
• Coverage: 360 degree
• Cost: Reused the central Mic

27. MIC array ID consideration

28. Agenda
• What is Smart Speaker
• MEMS Microphone Array Technologies
• Cloud Voice Service

29. Cloud Voice Service
• Speech Recognition service(Speech to Text)
• NLP service (Natural Language Processing)
• TTS service (Text to Speech)

30. Cloud Voice Service (cont.)
Voice
Text
Intent
Feedback Activity
Device
Cloud

31. Cloud Voice Service (cont.)
• Amazon Alexa Voice Service
• Google Voice Service
• Baidu DuerOS
• Microsoft Voice Services
• IBM Watson Voice Service
• Nuance Communications
• Internal of Amazon AVS

32. Amazon Alexa Voice Service
• Alexa Voice Service(AVS)
• voice recognition service
• natural language understanding
service
• For voice-enable connected device
• Alexa Skills Kit
• API for voice application
• Include the ability to play music,
answer general questions, set an
alarm or timer, and more.

33. Amazon Alexa Voice Service (cont.)

34. Amazon Alexa Voice Service (cont.)
• Application Case
Amazon Echo Vehicle Connectivity
( BMW / Ford / Hyundai )

35. Google Voice Service
• Actions on Google
• Design VUI
• work with the Google Assistant
• Support Google Home
• Support DiaglogFlow
• Support Firebase
• Google Speech Recognition
• Google TTS
• Google Natural Language API

36. Google Voice Service (cont.)
• Application Case
Google Home

37. Baidu DuerOS
• DuerOS Open Platform
• 2017年1月由百度度秘推出
• 結合人工智慧對話系統與智慧家電的作業系統
• DIDP(DuerOS Intelligent Devices Platform)
• 為提供智慧設備可對話的能力，通過集成DuerOS
智能硬件與開放接口的能力，提供使用者以下的
操作體驗
1. 通過語音控制設備來播放音樂、查詢天氣及
最新新聞，獲取交通情況以及通用知識詢問
2. 通過語音來設置鬧鐘、提醒
3. 通過語音來獲取服務，如叫車、訂外賣等
4. 通過語音來獲取來自百度第三方合作夥伴創
建的技能

38. Baidu DuerOS (cont.)
• 技術架構
應用層
小度智能設備
開放平台
核心層
小度對話
核心系統
能力層
小度技能
開放平台
場景應用參考設計
核心接入組件
晶片模組
開發套件
SDK
麥克風陣列
機構設計
工業設計
音響設計
對話服務(DuerOS Conversational Service)
技能框架 (DuerOS Bot Framework)
語音識別 語音播報 屏幕顯示
原生技能 第三方技能
技能開發工具

39. Baidu DuerOS (cont.)
• 開發流程
開發者認證
選擇
場景
手機
音箱
冰箱
電視
故事機
輕量設備
Android
Linux
mbedOS
服務配置
裝置名稱
基本配置
OAUTH
配置
下載
SDK

40. Microsoft Voice Service
• Cognitive API with Microsoft Azure
• Speech API
• Speech to Text
• Text to Speech
• Speaker Recognition
• Speech Translation
• Language API
• Text Analytics
• Translator Text
• Bing Spell Check
• Language Understanding
• Content Moderator

41. Microsoft Voice Service (cont.)
• Microsoft Cortana

42. IBM Watson Voice Service
• Voice Agent with Watson
• Improve telephone-based customer
service
• based on IBM Voice Gateway
• Support Service Orchestration Engine
• Speech to Text API
• Text to Speech API
• Natural Language Classifier
• Interpret and classify natural
language with confidence.

43. IBM Watson Voice Service (cont.)
• Application case
IBM Watson-powered
driverless electric bus
Softbank Pepper robot

44. Nuance Communications
• The company provide the software voice technology
used for Samsung’s S Voice and Apple’s Siri.
• Voice Search
• Intelligent Personal Assistance
• Knowledge Navigator
• Natural Language User Interface
• Dragon Software Developer Kit
• Dragon Mobile SDK
• PC Recognition Software

45. Internal of Amazon AVS

46. Internal of Amazon AVS (cont.)
Amazon Skill Store

47. Internal of Amazon AVS (cont.)
• AVS flow

48. Internal of Amazon AVS (cont.)
• AVS flow (cont.)
• AVS response
HTTP/1.1 200 OK
Content-Type: multipart/related; boundary={BOUNDARY TERM}
--{BOUNDARY TERM}
Content-Type: application/json; charset=UTF-8
{ "messageHeader": {}
"messageBody" {
"directives": { {
"namespace": "Control",
"name": "TurnOnOffRequest",
"action": "TURN_ON",
"applianceID: "4704F880-6026-11E5", : {
. . .

49. Internal of Amazon AVS (cont.)
• AVS flow (cont.)
Voice
Intent

50. Internal of Amazon AVS (cont.)
• AVS flow (cont.)
Alexa Connected Home (CoHo) Skills
WeMo Lighting Skills
Intent
Activity

51. Internal of Amazon AVS (cont.)
• Custom Skill
• Control LED ON/OFF on
Realtek Ameba board
Voice Intent
Trigger
Skill
MQTT