Presentation given by Janani Mukundan and Richard Daskas at Watson DevCon 2016.
As AI advances, scientists are pushing the boundaries of computational creativity. IBMers from Research are teaching Watson to compose original music based on human emotion. Using machine learning algorithms, the Researchers are teaching artificial neural networks to understand music theory, structure (pitch, time signature, key signature), and emotional intent to co-create music with a human partner. "Watson Beat" generates new and unique musical song combinations based on the beat played by the individual and the emotional tone the human chooses. Here’s how it works.
10. Example 1: Create new music by adding perturbation
10
h: Hidden Layer
(Holds extracted features of visible
Demo: Learned
“Mary” (less perturbation)
Demo: Learned
“Mary” (more perturbation)
x: Visible layer
(Holds initial pitch information)
h1: Hidden Layer
(Holds extracted features of visible
C# rand
B
Time
x: Visible layer
(Holds perturbed pitch information)
x~: Learned visible layer
(Holds learned pitch information)
p(h|x) p(x|h)
E rand E# rand`
B
Time
A rand`
p1(h|x)
C# E B
Time
Demo: Original “Mary
had a little lamb”
11. 11
h: Hidden Layer
(Holds extracted features of visible
Demo: Spooky
String Quartet
x: Visible layer
(Holds initial pitch information)
h1: Hidden Layer
(Holds extracted features of visible
C#
Oct
B
Time
x: Visible layer
(Holds perturbed pitch information)
x~: Learned visible layer
(Holds learned pitch information)
p(h|x) p(x|h)
E Oct
E# Oct`
B
Time
A Oct`
p1(h|x)
C# E B
Time
Demo: Original “String
Quartet”
Example 2: Create new music by steering learning based on emotional intent
12. Example 3: Create new music by learning from two songs and adding perturbation
C# minor
B
Time
Visible layer
(Holds pitch + bias information)
Hidden Layer
(Holds extracted features of visible layer)
minor
E
Song A
D#
Song B
Weights
12
Demo: Learned
“Willie Nelson”
and
”Miley Cyrus”
14. 14
x1: Visible layer for RBM1
x2: Visible layer for RBM2
h1: Hidden Layer for RBM 1
W1: weights for RBM1
x3: Visible Layer for RBM3
h2: hidden layer for RBM2
h3: hidden layer for RBM3
W2: weights for RBM2
W3: weights for RBM3