声質変換の概要と最新手法の紹介

Copyright © DeNA Co.,Ltd. All Rights Reserved.
Strictly confidential

n G
n
n
n -
n A C
2

n a
/ []eb A D a C
/ D 4 1 /0 , 1 6
3

n I N
)
n A
B B
(
(
(
(
(

5
3 0 2 . /3 0 7 3 1 . 0 0 7 3

6
3 0 2 . /3 0 7 3 1 . 0 0 7 3
:

n
A
B A
7
B
( A )
( ; F0)
( ; bap)
B

8
(F0 )
bap
•
→ Vocoder
•
• STRAIGHT [Kawahara+; ’99]
• WORLD [Morise+; ’16]

Vocoder
9

vocoder
10
F0bap
F0bap
F0bap
1 frame
frame
Frame

[Abe+; ’90][stylianou+; ’98]
n
A B
11
F0bap
F0bap
GMM DNN

[Abe+; ’90][stylianou+; ’98]
n
B
12
AF0bap
AF0bap
GMM DNN
• F0, bap
→ A
• F0 bap

13
Parallel-data
B
A
frame
A
B

o s e P P
• 6C C E C AA 6-- ( y
• 6-- K K E 2; ] p e
d r aKvt 6-- (
3 E AA A ; G
• g K V P[P PN g kO
• E AA A ; G P Po - A 1 70 C + )8
i h ced
• nQc i h ʻ] d l ]SP
• nQc 6 6 . 7 ; 2CE; + )8
14

ig l hV vNVQ] V
• 3 7 7 E C 7;6 Nk
16 6 6 6Nyo
• Ns V PV cK
• A6 6 6 6 V i + 7 - .6 G
n a N []
• n a r pdNʻ O
• t V e [n 32 3 , E6 0 G
15

Voice Conversion Challenge 2016
n
n 7
7
7
n 5 5
n 01
16

Results of listening tests in VCC 2016
17
cf. http://vc-challenge.org/vcc2016/summary.html

ig l hV vNVQ] V
• 3 7 7 E C 7;6 Nk
16 6 6 6Nyo
• Ns V PV cK
• A6 6 6 6 V i + 7 - .6 G
n a N []
• n a r pdNʻ O
• t V e [n 32 3 , E6 0 G
18

CycleGAN [Zhu+; ’17]
n
19
cf. https://junyanz.github.io/CycleGAN/

n
n
Forward-inverse mapping Inverse-forward mapping
GX→Y GY→X G L real/fake loss
[Kaneko+; ‘17]
M
mapping loss

n
n
21
GX→Y adversarial loss

n
n
22
= "#~%&'(' # log,- . + "0~%&'(' 0 log 1 − ,- 34→- .
GY→X adversarial loss

n
n
23
L1loss

n
n
24
λcyc 10.0

CycleGAN parallel-data-free [Kaneko+, ’17]
n NG NG
n
C
25
CycleGAN
copy
A
A
A
A
A
t1 t2 tTbap
bap
bap
bap
bap
F0
F0
F0
F0
F0
bap
bap
bap
bap
bap
F0
F0
F0
F0
F0
A
A
A
A
A
t1 t2 tT

VC
. 1 1 2 r
• d c U R l U
• c U
• t pt t G l em
- 1 1 a ) ( A . (1 1 ( l
• y sv cG X
• g I ni L UI o
26

Network architecture
n . / - . /
n :
. / / . / / / .
27

ig l hV vNVQ] V
• 3 7 7 E C 7;6 Nk
16 6 6 6Nyo
• Ns V PV cK
• A6 6 6 6 V i + 7 - .6 G
n a N []
• n a r pdNʻ O
• t V e [n 32 3 , E6 0 G
28

Variational autoencoder (VAE) [Hinton+; '06]
n
z
29
x
Encoder
qθ(z|X)
Decoder
pθ(X|z)
z
!"
# $; 0, 1
Input feature Generated feature

VQ-VAE [van den Oord+, ‘17]
n
- -( ) E V A
n
30
x
Encoder
p(ze(x)|x)
Decoder
p(x|zq(x))
ze(x)
!"
A
A
e1 e2 e3 eK
zq(x)
x LQ loss VQ loss Encoder loss

VQ-VAE
n [van den Oord+; ’16]
t v o G a r
• N x h G d
• λ W l lg d
• l m r e
31
! " # = %
&'(
)
* +&|+&-),+&-)/0, ⋯ +&-0, #
λ : d lg c d
, " = +(, +0, ⋯ +&-0

VQ-VAE [van den Oord+, ’17]
n
32
Encoder WaveNet
ze(x)
e1 e2 e3 eK
zq(x)
id
• zq(x) id
• ze(x) zq(x) id
• zq(x)
( )
https://avdnoord.github.io/homepage/vqvae/

VQ-VAE [van den Oord+; ‘17]
n
33
Encoder WaveNet
ze(x)
e1 e2 e3 eK
zq(x)
id
cf. https://www.slideshare.net/YukiSaito8/saito18sp03
• zq(x) id
• ze(x) zq(x) id
• zq(x)
( )
https://avdnoord.github.io/homepage/vqvae/

n A
n C
-
34

H9J J JZJ d.-I 6 9J J JZJ 7 J[]MJ 9J[][N JVM 0 MN 1 N NRPVN e N[ Z]L ]ZRVP [XNNL ZNXZN[NV J RWV[ ][RVP J
XR L JMJX R N RUN OZNY]NVLa [UWW RVP JVM JV RV[ JV JVNW][ OZNY]NVLa KJ[NM 4 N ZJL RWV W[[RKTN ZWTN WO J ZNX R R N
[ Z]L ]ZN RV [W]VM[ f XNNL 1WUU]VRLJ RWV , XX -, , ...
H WZR[N c +I WZR[N 4 FWSWUWZR JVM 9 bJ J eD :2 J WLWMNZ KJ[NM RP Y]JTR a [XNNL [aV N[R[ [a[ NU OWZ
ZNJT RUN JXXTRLJ RWV[ f 73713 ZJV[JL RWV[ WV RVOWZUJ RWV JVM [a[ NU[ WT 3.. 2 VW , XX -,, --) +
H0KN . I 0KN JSJU]ZJ 9 RSJVW JVM 6 9] JKJZJ eCWRLN LWV NZ[RWV ZW]P NL WZ Y]JV RbJ RWV f
, ,+
H[ aTRJVW] c.-I F aTRJVW] 1JXXh JVM 3 W]TRVN[ e1WV RV]W][ XZWKJKRTR[ RL ZJV[OWZU OWZ WRLN LWV NZ[RWV f
( ) ..-
H9JVNSW ,I A 9JVNSW JVM 6 9JUNWSJ f JZJTTNT 2J J 4ZNN CWRLN 1WV NZ[RWV [RVP 1aLTN 1WV[R[ NV 0M NZ[JZRJT
N WZS[ f JZER ,
HG ] c ,I 8 F G ] A JZS 7[WTJ JVM 0 0 3OZW[ e VXJRZNM RUJPN W RUJPN ZJV[TJ RWV ][RVP LaLTN LWV[R[ NV
JM NZ[JZRJT VN WZS[ f
H6RV WV c +I 5 3 6RV WV JVM JTJS ] MRVW e NM]LRVP N MRUNV[RWVJTR a WO MJ J R VN]ZJT
VN WZS[ f ,-+ ) , +
H JV MNV WZM c ,I 0 JV MNV WZM JVM CRVaJT[ e N]ZJT MR[LZN N ZNXZN[NV J RWV TNJZVRVP f 7V
XX +( . +( - ,
H JV MNV WZM d +I 0 JV 2NV WZM 2RNTNUJV 6 GNV 9 RUWVaJV CRVaJT[ 0 5ZJ N[ JVM 9 9J ]SL]WPT]
eDJ NVN 0 PNVNZJ R N UWMNT OWZ ZJ J]MRW f
35

声質変換の概要と最新手法の紹介

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to 声質変換の概要と最新手法の紹介

Similar to 声質変換の概要と最新手法の紹介 (20)

More from Kentaro Tachibana

More from Kentaro Tachibana (6)

Recently uploaded

Recently uploaded (20)

声質変換の概要と最新手法の紹介