The document discusses Watson and open source software. It provides an overview of Watson's question answering system and some of the key open source software used in Watson, including UIMA, Hadoop, MapReduce, Lucene, and Indri. The presentation describes Watson's architecture and how it uses machine learning to analyze questions, generate hypotheses, retrieve and score evidence from sources to determine confident answers.
1. Watson & Open Source
Software
Ivan Portilla
IT Architect
5/8/12
portilla@gmail.com
Sunday, May 20, 12 1
2. If I have seen further it is by standing on the
shoulders of giants.
Isaac Newton, Letter to Robert Hooke, February 5, 1675
Sunday, May 20, 12 2
3. Objectives
By the end of this session, you
should be able to:
ü Describe the main characteristics
of Watson QA system.
ü Identify the key open source SW
used in Watson.
ü Recognize examples of Agile
development best practices.
3
Sunday, May 20, 12 3
5. Disclaimer 1
ü This presentation represents the view of the
author and does not represent the view of IBM.
ü All opinions expressed in this presentation are
strictly of the speaker, and do NOT represent
those of IBM, IBM management, or anyone else.
ü IBM and IBM (logo) are trademarks or registered
trademarks of International Business Machines
Corporation in the United States and/or other
countries.
Sunday, May 20, 12 5
6. Disclaimer
2
I
(We)
do
not
work
for
the
Watson
team.
Sunday, May 20, 12 6
8. Let’s Play Jeopardy
BEFORE & AFTER: The Jerry Maguire star who
automatically maintains your vehicle’s speed.
Sunday, May 20, 12 7
9. Let’s Play Jeopardy
BEFORE & AFTER: The Jerry Maguire star who
automatically maintains your vehicle’s speed.
Sunday, May 20, 12 7
10. Let’s Play Jeopardy
BEFORE & AFTER: The Jerry Maguire star who
automatically maintains your vehicle’s speed.
COMMON BONDS: trout, loose change in your pocket,
and compliments.
Sunday, May 20, 12 7
11. Let’s Play Jeopardy
BEFORE & AFTER: The Jerry Maguire star who
automatically maintains your vehicle’s speed.
COMMON BONDS: trout, loose change in your pocket,
and compliments.
Diplomatic Relations: Of the four countries in the
world that the United States does not have diplomatic
relations with, the one that’s farthest north.
Sunday, May 20, 12 7
12. Let’s Play Jeopardy
BEFORE & AFTER: The Jerry Maguire star who
automatically maintains your vehicle’s speed.
COMMON BONDS: trout, loose change in your pocket,
and compliments.
Diplomatic Relations: Of the four countries in the
world that the United States does not have diplomatic
relations with, the one that’s farthest north.
Sunday, May 20, 12 7
13. Let’s Play Jeopardy
BEFORE & AFTER: The Jerry Maguire star who
automatically maintains your vehicle’s speed.
COMMON BONDS: trout, loose change in your pocket,
and compliments.
Diplomatic Relations: Of the four countries in the
world that the United States does not have diplomatic
relations with, the one that’s farthest north.
Geography: Chile shares its longest land border with
this country
Sunday, May 20, 12 7
17. A Brief History of Watson
§ Deep Blue Ended in 1997
§ Looking for a new research challenge
§ 2004, IBM Research manager Charles Lickel,
§ Ken Jennings
§ Started in 2005
• David Ferrucci
• DeepQA in 2007
• Won Jeopardy Match, Feb 2011
hAp://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717
Sunday, May 20, 12 11
18. A Brief History of Watson
§ Deep Blue Ended in 1997
§ Looking for a new research challenge
§ 2004, IBM Research manager Charles Lickel,
§ Ken Jennings
§ Started in 2005
• David Ferrucci
• DeepQA in 2007
• Won Jeopardy Match, Feb 2011
hAp://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717
Sunday, May 20, 12 11
20. What
is
Watson?
ü Understands
natural
language.
ü Generates
&
evaluates
hypothesis
for
beAer
outcomes.
ü Adapts
&
learns
from
user
selec@ons
and
responses.
hAp://www.ibm.com/innova@on/us/watson/
Sunday, May 20, 12 13
21. Watson
metrics
Development Team: 25 people
Project Duration: 4 years
Hardware: 90 IBM Power-750 servers
2880 Power7 cores @ 80+ TFLOPS
20 TB Disk, 16 TB RAM (memory)
10 Gbps network
hAp://na11.apachecon.com/talks/19932
hAp://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717
Sunday, May 20, 12 14
26. Learning To Rank (Basic Architecture)
Hypothesis
Keywords Evidence
Scoring
Watson
by
R.Yates
Sunday, May 20, 12 19
27. Watson
Architecture
hAp://en.wikipedia.org/wiki/Watson_(computer)
Sunday, May 20, 12 20
28. Who is the 44th President of the
United States?
Sunday, May 20, 12 21
29. Who is the 44th President of the United States?
Who is the 44th President
of the United States?
Watson
by
R.Yates
Sunday, May 20, 12 22
30. Who is the 44th President of the United States?
'Who' is the '44th' 'President' of the 'United States'?
Who is the 44th President
of the United States?
Lexical Focus Keywords
Answer
Type Can be replaced
by the correct
answer to make a
→ Person
true statement
Sunday, May 20, 12 23
31. Who is the 44th President of the United States?
Keywords:
44th President United States
Question
Question Hypothesis
Analysis Generation
Watson
by
R.Yates
Sunday, May 20, 12 24
32. Who is the 44th President of the United States?
Keywords:
44th President United States
Question
Question Hypothesis
Analysis Generation
Sunday, May 20, 12 25
33. Who is the 44th President of the United States?
Primary
Search
Question
Question Hypothesis
Analysis Generation
Sunday, May 20, 12 26
34. Who is the 44th President of the United States?
Barack Obama
George W. Bush
Harvard Law School
Illinois
Primary
Search
Question
Question Hypothesis
Analysis Generation
Watson
by
R.Yates
Sunday, May 20, 12 27
35. Who is the 44th President of the United States?
Who is the 44th President of the United States?
Barack Obama
Who is the 44th President of the United States?
George W. Bush
Answer
Who is the 44th President of the United States? Scoring
Harvard Law School
Who is the 44th President of the United States?
Illinois Question
Scoring
Question Hypothesis Scoring
Analysis Generation
Scoring
Sunday, May 20, 12 28
36. Who is the 44th President of the United States?
Who is the 44th President of the United States?
Barack Obama
Who is the 44th President of the United
States?
Who is the 44th President of the United States?
George W. Bush Answer
Answer
Scoring Contextual → Person
Scoring
Answer Answer
Who is the 44th President of the United States? Answer
Scoring Scoring Is Barack Obama a Person? .90
Harvard Law School Scoring Is George W. Bush a Person? .90
Is Harvard Law School a Person? .10
Who is the 44th President of the United States? Is Illinois a Person? .15
Illinois Question
Scoring
Question Hypothesis Scoring
Analysis Generation
Scoring
Watson
by
R.Yates
Sunday, May 20, 12 29
37. Who is the 44th President of the United States?
Barack Obama is the 44th President of the United States
George W. Bush is the 44th President of the United States
Harvard Law School is the 44th President of the United States
Illinois is the 44th President of the United States
Contextual
Answer
Scoring
Question
Scoring
Question Hypothesis Scoring
Analysis Generation
Scoring
Unstructured
Informa@on
Management
Applica@ons
-‐
UIMA
Sunday, May 20, 12 30
38. Who is the 44th President of the United States?
Barack Hussein Obama II (i/bəәˈrɑːk huːˈseɪn oʊˈbɑːməә/;
born August 4, 1961) is the 44th and current President of
the United States.
George Walker Bush (born July 6, 1946) is an American
politician who served as the 43rd President of the United
States from 2001 to 2009 and the 46th Governor of Texas
from 1995 to 2000.
Barack Obama is the 44th President of the United States
George W. Bush is the 44th President of the United States
Harvard Law School is the 44th President of the United States
Illinois is the 44th President of the United States
Question Barack Obama .95
George W. Bush .80
Scoring Harvard Law School .05
Question Hypothesis Scoring Illinois.10
Analysis Generation
Scoring
Unstructured
Informa@on
Management
Applica@ons
-‐
UIMA
Watson
by
R.Yates
Sunday, May 20, 12 31
39. Who is the 44th President of the United States?
Candidate Answer Answer Evidence retrieval & Confidence
Scoring scoring
Barack Obama 0.90 0.90 .95
George W. Bush 0.90 0.80 .65
Evidence
Harvard Law School 0.10 0.05 Retrieval .05
Illinois 0.15 0.10 .10
Trained
Question Models
Scoring
Question Hypothesis Scoring
Analysis Generation
Scoring
Unstructured
Informa@on
Management
Applica@ons
-‐
UIMA
Watson
by
R.Yates
Sunday, May 20, 12 32
40. DeepQA
Massively Parallel Probabilistic Evidence-Based Architecture
Learned Models
help combine and
weigh the Evidence
Evidence Balance
Answer Sources & Combine
Models Models
Sources
Question Answer Evidence
Models Models
Scoring Retrieval
Primary Candidate
& Scoring Models Models
Search Answer
Generation
Ques@on
&
Final
Confidence
Ques@on Hypothesis Hypothesis
and
Evidence
Topic
Synthesis Merging
&
Decomposi@on Genera@on Scoring
Analysis Ranking
Hypothesis Hypothesis and Merging & Answer &
Evidence Scoring Ranking
Genera@on Confidence
...
ApacheCon
2011,
Watson,
a
Reasoning
System:
based
on
Apache
Inside!,
David
Boloker
Sunday, May 20, 12 33
42. OSS
in
Watson
üUIMA,
UIMA-‐AS
üHadoop,
Map
Reduce
üLucene,
Indri
ApacheCon
2011,
Watson,
a
Reasoning
System:
based
on
Apache
Inside!,
David
Boloker
Sunday, May 20, 12 35
43. UIMA
hAp://uima.apache.org/
Sunday, May 20, 12 36
44. UIMA-‐Asynchronous
Scaleout
UIMA
AS
provides
more
flexible
and
powerful
scale
out
capability.
Innovate 2011, How Does It Work? The Architecture of Watson. Grady Booch
Sunday, May 20, 12 37
45. Think
Hadoop
A
framework
for
storing
&
processing
big
data.
üUp
to
4,000
machines
üUp
to
20
PB
High
reliability
done
in
soiware:
üAutomated
failover
for
data
&
computa@on
üImplemented
in
Java
hAp://hadoop.apache.org/mapreduce/
Sunday, May 20, 12 38
46. Map
reduce
hAp://hadoop.apache.org/mapreduce/
Sunday, May 20, 12 39
48. Architecture
hAp://lucene.apache.org
Sunday, May 20, 12 41
49. Indri
&Lemur
Indri
is
a
text
search
engine
developed
at
Umass
&
CMU.
Indri
is
part
of
the
Lemur
project.
hAp://lemurproject.org/indri/
Sunday, May 20, 12 42
51. Other
OSS
in
Watson
hAps://www.ibm.com/developerworks/mydeveloperworks/blogs/InsideSystemStorage/entry/
ibm_watson_how_to_build_your_own_watson_jr_in_your_basement7?lang=en
Sunday, May 20, 12 44
52. J-‐Archive
data
hAp://www.j-‐archive.com/showgame.php?game_id=3577
Sunday, May 20, 12 45
53. Development Process
üWar room setting with
continuous collaboration.
üWeekly integration.
üResults driven with E2E
regression testing.
ü About 8,000 experiments
ü 10 GBs of test data/wk.
ü Agile development
Innovate 2011, How Does It Work? The Architecture of Watson. Grady Booch
Sunday, May 20, 12 46
54. Other
OSS
hAp://manning.com/ hAp://www.apache.org
Sunday, May 20, 12 47
55. Take
Away
OSS
is
powerful
and
scalable
enough
for
the
Watson
team,
what
about
your
project?
Sunday, May 20, 12 48
56. Resources
IBM
Journal
of
Research
and
Development
hAp://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717
IBM
Watson
hAp://www.ibm.com/innova@on/us/watson/
hAp://www.research.ibm.com/deepqa/index.shtml
Nova
hAp://www.pbs.org/wgbh/nova/tech/smartest-‐machine-‐on-‐earth.html
Sunday, May 20, 12 49
57. Review of Objectives
Now that you have completed this session, you are able to:
ü Describe the main characteristics of Watson QA system.
ü Identify the key open source tools used in Watson.
ü Recognize examples of Agile development best practices.
51
Sunday, May 20, 12 50