The Jeopardy match between the two best human players of all time and the IBM Deep Q/A software, “Watson,” captured the spotlight and stimulated the imagination of the entire world. The subsequent announcement of IBM’s involvement in the creation of “Dr. Watson” has created a high level of interest in the healthcare community about the potential of this breakthrough technology as well as the potential pitfalls of the use of “artificial intelligence” in medicine. Dr. Siegel is currently working together with IBM engineers to explore how Dr. Watson can work together with physicians and medical specialists. His presentation, which was delivered on March 28th, provided a high level overview of the uniqueness of Deep Q/A Software and how it differs from other previous artificial intelligence applications.

1. CBIIT Speaker Series
Watson and Deep Q/A Software In
Pursuit of Personalized Medicine
Eliot Siegel, M.D., FACR, FSIIM
Professor and Vice Chair University of Maryland Department of Diagnostic
Radiology
Chief Imaging VA Maryland Healthcare System
1

2. caBIG Mission
• Widespread, sustainable availability of critical standards-based,
interoperable academic/commercial biomedical capabilities
• Large and diverse cancer research data sets sustainably available for
analysis, integration, and mining
• Rather than 2% or 3% of patients’ data captured in clinical trials, capture
all patient data for decision and treatment support and data driven
research
2

3. Clinical Scenario
3

4. Clinical Challenge
VINCI – VA Informatics and Computing Infrastructure
4

5. Year of Artificial Intelligence in Medicine
• 2011 will likely be remembered as the year of the re-emergence of
artificial intelligence in medicine with Watson and of course, Siri,
arguably the best feature of the new iPhone 4S
• 2011 may well be the year that AI finally gets real traction in the
medical informatics community and in medicine in general including
the lay population
• Biggest contribution of Dr. Watson software in addition to Deep Q/A
may be excitement to overcome inertia of the past
5

6. IBM and Jeopardy: A New Era?
• The Jeopardy match between the two best human players of all time
and the IBM Deep Q/A software, “Watson” captured the spotlight and
stimulated the imagination of the entire world
• The subsequent announcement of IBM’s involvement in the creation of
“Dr. Watson” has created an incredible interest in the healthcare
community about the potential breakthrough technology as well as the
potential pitfalls of the use of “artificial intelligence” in medicine.
6

7. Dr. Watson Overview and History
• Initially had opportunity to visit IBM team about a year and a half ago
• Engaged Jeopardy team and discussed the potential for medical
applications as next steps after Jeopardy Challenge
• Began initial research with IBM approximately one year ago
• Current grant with IBM for initial exploratory work with physician
helping team to understand the medical domain and challenges
• Worked together on deeper understanding of the medical domain
using multiple resources
7

8. Introduction
• Deep Q/A is unique and exciting because it represents a fundamentally
new approach that creates tools to rapidly mine a dynamic and non-
predefined database
• Represents a potential fundamental change in opportunities for
Artificial Intelligence applications in medicine
• But in some ways Watson is a “special needs” student
• How does one train a system that is so remarkable at Jeopardy!
questions and apply to medicine?
8

9. • Watson can process 500 gigabytes, the equivalent of a million books,
per second
• Hardware cost has been estimated at about $3 million
• 80 TeraFLOPs , 49th in the Top 50 Supercomputers list
• Content was stored in Watson's RAM for the game because data
stored on hard drives too slow to process
9

10. Deep Q/A
• Massively parallel, component based pipeline architecture
• Uses extensible set of structured and unstructured content sources
• Uses broad range of pluggable search and scoring components
10

11. Deep Q/A
• These allow integration of many different analytic techniques
• Input from scorers is weighed and combined using machine learning
to generate a set of ranked candidate answers and associated
confidence values
• Each answer is linked to its supporting evidence
11

12. Deep Q/A
• Does not map question to database of answers
• Represents software architecture for analyzing natural language
content in both questions and knowledge sources
• Discovers and evaluates potential answers and gathers and scores
evidence for those answers using unstructured sources such as
natural language documents and structured sources such as
relational and knowledge databases
12

13. Hardware
• Cluster of ninety IBM Power 750 servers (plus additional I/O, network
and cluster controller nodes in 10 racks) with a total of 2880 POWER7
processor cores and 16 Terabytes of RAM
• Each Power 750 server uses a 3.5 GHz POWER7 eight core processor,
with four threads per core
• The POWER7 processor's massively parallel processing capability is
an ideal match for Watson's IBM DeepQA software which
is embarrassingly parallel (that is a workload that is easily split up into
multiple parallel tasks)
13

14. Software
• Watson's software was written in both Java and C++ and uses Apache
Had0op framework for distributed computing
• Apache UIMA (Unstructured Information Management Architecture)
framework
• IBM’s DeepQA software and SUSE Linux Enterprise Server
11 operating system
• “More than 100 different techniques are used to analyze natural
language, identify sources, find and generate hypotheses, find and
score evidence, and merge and rank hypotheses.”
14

15. High Level View of DeepQA Architecture
15

16. Deep QA Process
• Analyzes input question and generates many possible candidate
answers through broad search of volumes of content
• Hypothesis is formed based on considerate of each candidate answer
in context of original question and topic
• For each of these, DeepQA spawns independent thread attempting to
prove it
• Searches content sources for evidence supporting or refuting each
hypothesis
• Applies hundreds of algorithms for each evidence hypothesis pair that
dissects and analyzes along different dimensions of evidence
16

17. Types of Dimensions of Evidence
• Type classification
• Time
• Geography
• Popularity
• Passage support
• Source reliability
• Semantic relatedness
17

18. Dimensions of Evidence for Jeopardy!
18

19. Scoring Features
• These features/scores are then combined based on their learned
potential for predicting the right answer resulting in a ranked list of
candidate answers, each with a confidence score indicating degree to
which the answer is believed to be correct, along with links back to the
evidence
19

20. Deep QA for Differential Diagnosis
20

21. Advantages of Dr. Watson Approach
• Represents new architecture for evaluating unstructured content
• Different from traditional expert systems using forward reasoning
(data to conclusions) or backward reasoning
• Unlike systems such as Stanford’s Mycin that used If-Then
statements:
• If
• The stain of the organism is grampos and the morphology of the organism is
coccus and the growth conformation of the organism is chains
• Then
• There is suggestive evidence that the identity of the organism is
streptococcus
21

22. Advantages of Watson Approach
• If then approach is costly and difficult to develop and maintain
• Traditional expert systems are brittle because underlying reasoning
engine requires perfect match between input data and existing rule
forms
• Not all rule forms can be known in advance for all forms that the input
data may take
22

23. Advantages of Watson Approach
• Watson uses NLP and variety of search techniques to generate likely
candidate answers in hypothesis generation (analogous to forward
chaining”)
• Uses evidence collection and scoring (analogous to “backward
chaining”)
• These make DeepQA more flexible, maintainable, and scalable as well
as cost efficient in terms of staying current with vast amounts of new
information
23

24. Clinical Setting
• Deep QA can develop diagnostic support tool using the context of an
input case (information about patient’s medical condition)
• Generates ranked list of differential diagnoses with associated
confidences
• The dimensions of evidence include
• Symptoms
• Findings
• Patient history
• Family history
• Demographics
• Current medications
• Many others
24

25. Is There A Need for Artificial Intelligence In Medicine?
Do Physicians Need Assistance?
25

26. Motivation for Artificial Intelligence Software in
Medicine
• Schiff
• Diagnostic errors far outnumber other medical errors by 2-4X
• Elstein
• Diagnostic error rate of about 15% in line with autopsy studies
• Singh and Graber
• Diagnostic errors are single largest contributor to ambulatory
malpractice claims (40% in some studies) and cost about $300,000 per
claim
• Graber
• Literature review of causes of diagnostic error suggest 65% system
related (e.g. communication) and 75% had cognitive related factors
26

27. Cognitive Errors
Graber et al Diagnostic Error in Internal Medicine, Arch
Intern Med 2005; 165:1493-1499
• Cognitive errors primary due to “faulty synthesis or flawed processing
of the available information”
• Predominant cause of cognitive error was premature closure
(satisfaction of search in diagnostic imaging)
• Failure to continue considering reasonable alternatives after an initial
diagnosis was reached
27

28. Cognitive Errors
• Other contributors to cognitive errors
• Faulty context generation – lack of awareness of aspects of patient info
relevant to diagnosis
• Misjudging salience of a finding
• Faulty detection or perception
• Failed use of heuristics – assuming single rather than multifactorial
cause of patient symptoms
28

29. Cognitive Errors
• Graber suggested augmenting “a clinician’s inherent metacognitive
skills by using expert systems”
• Suggested that clinicians continue to miss diagnostic information and
“one likely contributing factor is the overwhelming volume of alerts,
reminders, and other diagnostic information in the Electronic Health
Record”
29

30. Previous Attempts at Artificial Intelligence in Medicine
• Mycin- Stanford
• Doctoral dissertation of Edward Shortliffe designed to identify bacterial
etiology in patients with sepsis and meningitis and to recommend
antibiotics
• Had simple inference engine and knowledge base of 600 rules
• Proposed acceptable therapy in 69% of cases which was better than
most ID experts
• Never actually used in practice largely due to lack of access and time
for physician entry >30 minutes
• Caduceus – similar inference engine to Mycin and based on Harry
Pope from U of Pittsburgh’s interviews with Dr. Jack Myers with
database of up to 1,000 diseases
30

31. Previous Attempts at Artificial Intelligence in Medicine
• Internist I and II – Covered 70-80% of possible diagnoses in internal
medicine, also based on Jack Myers’ expertise
• Worked best on only single disease
• Long training and unwieldy interface took 30 to 90 minutes to interact
with system
• Was succeeded by “Quick Medical Reference” which was
discontinued ten years ago and evolved into more of a reference
system than diagnostic system
• Each differential diagnosis includes linkes to origin evidence to
provide meaningful use of EMR’s and supports adoption of evidence
based medicine/practice
31

32. Medical Diagnostic Systems
• Dxplain used structured knowledge similar to Internetist I, but added
hierarchical lexicon of findings
• Iliad system developed in ‘90s added probabilistic reasoning
• Each disease had associated a priori probability of disease in population
for which it was assigned
32

33. Diagnostic Systems using Unstructured Knowledge
• ISABEL uses information retrieval software developed by “Autonomy”
• First CONSULT allows search of medical books, journals, and
guidelines by chief complaints and age group
• PEPID DDX is diagnosis generator
33

34. Diagnosis Systems Using Clinical Rules
• Acute cardiac ischemia time insensitive predictive instrument uses
ECG features and clinical information to predict probability of
ischemia and is incorporated into heart monitor/defibrillator
• CaseWalker system uses four item questionnaire to diagnose major
depressive disorder
• PKC advisor provides guidance on 98 patient problems such as
abdominal pain and vomiting
34

35. Reasons Current Diagnostic Systems Aren’t Widely
Used
• They aren’t integrated into day to day operations and workflow of
health organizations and patient information is scattered in outpatient
clinic visits and hospital visits and their primary provider and
specialists
• Entry of patient data is difficult – requires too much manual entry of
information
• They aren’t focused enough on recommendations for next steps for
follow up
• Unable to interact with practitioner for missing information to increase
confidence and more definitive diagnosis
• Have difficulty staying up to date
35

36. Watson in the News This Week As Oncology Librarian
• March 22, 2012 -- Memorial Sloan-Kettering Cancer Center (MSKCC)
and IBM plan to collaborate on the development of a powerful tool
built on IBM's Watson artificial intelligence platform that will provide
medical professionals with improved access to current and
comprehensive cancer data and practices, MSKCC said.
• The initiative will combine the computational and language-processing
ability of IBM Watson with MSKCC's clinical knowledge, existing
molecular and genomic data, and repository of cancer case histories
in order to create an outcome and evidence-based decision-support
system, according to MSKCC
36

37. Watson in the News as Research Librarian
• Development work has begun for the first applications, which include
lung, breast, and prostate cancers. The goal is to begin testing the tool
with a small group of oncologists in late 2012, with wider distribution
planned for late 2013, MSKCC said.
• The computer will assist doctors in making diagnoses and treatment
decisions by mining current information and alerting doctors to new
developments and research,
37

38. • "Sloan-Kettering and IBM are already developing the first applications
using Watson related to lung, breast, and prostate cancers, and aim to
begin piloting the solutions to some oncologists in late 2012, with
wider distribution planned for late 2013.”
38

39. Watson as Diagnostician
39

40. 40

41. Google Search
41

42. 42

43. 43

44. Google Search: Uveitis Cause
44

45. 45

46. Google Search: Uveitis, Arthritis, Circular Rash,
Headache
46

47. 47

48. 48

49. 49

50. User Information Design for Decision Support
50

51. My Involvement in Helping to Train Dr.
Watson
• Initial research and grant to help educate Watson in medical domain
• Could Watson software for Jeopardy! be successfully ported into the
medical domain?
• Began discussing challenge associated with NEJM Clinico-Pathological
Conference
• Talked about books and journals and other sources that could augment
the general knowledge built into the Jeopardy! playing software
51

52. After Jeopardy! Match:
Initial Reactions/Expectations
• E-mails and interviews from all over the world:
• Most were incredibly impressed with potential for medicine and
opportunities for the future
• Some however:
• SKYNET and end of world as we know it
• Pre-medical students speculating that it really doesn’t make sense to attend
medical school any more
• Physicians writing blogs predicting that they would be replaced by the
computer within a short period of time
52

53. Taking Watson to Medical School
• Want 3 components similar to medical students education
• Book knowledge
• Sim Human Model
• Experiential learning from actual EMR
53

54. Book Learning
• Textbook, journal, and Internet resource knowledge. Quiz materials
• Like medical student this alone not enough don’t want to make
hypochondriac
54

55. Advancing Deep Q/A’s Medical Knowledge
• Continue to develop medical knowledge database
• Harrison’s
• Merck
• Current Medical Diagnosis and Treatment
• American College of Physicians Medicine
• Stein’s Internal Medicine
• medical Knowledge Self Assessment Program
• NLM’s Clinical Question Repository
55

56. Advancing Deep Q/A’s Medical Knowledge
• Use New England Journal of Medicine 130 CPC cases and quiz
material
• Additional CPC cases at U of Maryland
• Begin developing interactive capability to develop hypotheses and
refine them depending on the answer to those questions
• Develop a tool that allows for physician feedback to the system for
various hypotheses so community can interact and teach Watson
56

57. SIM Human
• SIM Human model of physiology – work done at the University of
Maryland School of Medicine and UMBC by Dr. Bruce Jarrell and
colleagues
• Want to have understanding from model of physiology
• Work has been done to create simulations of disease processes and
then observe how it affects other physiology in the body
57

58. Clinical/Hospital “Experience”
• Consumption of electronic medical record which is largely just paper
represented digitally, cannot search for “rash” for example
• Access to records at U of Maryland and VA but also larger repositories
from the VA in de-identified manner
58

59. Electronic Medical Record Challenges and Limitations
• Epic system at the University of Maryland
• VA’s VISTA System
• University of Maryland EPIC system
• EMR
• Electronic version of paper records
• Review large number of discharge summaries
• Review progress notes and structured and unstructured additional
information from EMR
59

60. IBM and VA Team Review of EMR
• Patient EMR such as VA’s highly publicized and praised VISTA
revealed numerous challenges
60

61. Despite the fact that virtually 100% of patient information is available in the
electronic EMR with records going back more than 15 years
• Not possible to search for a term within or among patient records
such as “rash”
• Majority of data is unstructured and in free text format
• Much of the text in progress notes and other types of notes is highly
redundant since interns and residents and attending physicians
typically cut and paste information from lab and radiology and other
studies and other notes
• Information is entered with abbreviations that are not consistent and
misspellings
61

62. Patient Problem List
• Patient problem list has no “sheriff” and each physician is free to add
“problems” but very few delete them for “problems” that are temporary
• The problem lists themselves often have contradictory information
62

63. Medical Domain Adaptation
• 5000 questions from American College of Physicians Doctor’s
Dilemma competition
• E.g.
• The syndrome characterized by joint pain, abdominal pain, palpable
purpura, and a nephritic sediment
• Henoch-Schonlein Purpura
• Familial adenomatous polyposis is caused by mutations of this gene:
APC gene
• Syndrome characterized by narrowing of the extrahepatic bile duct from
mechanical compression by a gallstone impacted in the cystic duct:
Mirizzi’s Syndrome
63

64. 3 Areas of Adaptation for Deep QA
• Content
• Organizing domain content for hypothesis and evidence generation
such as textbooks, dictionaries, clinical guidelines, research articles
• Tradeoff between reliability and recency
• Training
• Adding data in the form of sample training questions and correct
answers from the target domain so system can learn appropriate
weights for its components when estimating answer confidence
• Functional
• Adding new question analysis, candidate generation, and hypothesis
evidencing analytics specialized for the domain
64

65. Content Adaptation
• Text content is converted into XML format used as input for indexing
• Text analyzed for medical concepts and semantic types using Unified
Medical Language System terminology to provide for structured query
based lookup
• “Corpus expansion technique” used by DeepQA searches web for
similar passages given description of symptoms for example and
generates pseudo documents from web search results
65

66. Medical Content Sources for Watson
Include:
• ACP (American College of Physicians)
Medicine
• Merck Manual of Diagnosis and Therapy
• PIER (collection of guidelines and evidence
summaries)
• MKSAP (Medical Knowledge Self Assessment
Program study guide from ACP)
• Journals and Textbooks
66

67. Discovering the Untapped, Disconnected
Gold Mines of Clinical and Research Data
• Despite all of the advances in computer technology we
are arguably still at the paper stage of research as far
as ability to discover and combine important data
• Research data including those associated with major
medical journals and clinical trials are typically created
for a single purpose and beyond a one or two
manuscripts, remain largely locked up or inaccessible
• Even when the data are made accessible, they are
typically associated with limited access through a
proprietary Internet portal or even by requesting data on
a hard drive
• Often requires submission of a research plan and data
and then a considerable wait for permission to use the
data which is often not granted
67

68. ADNI
• Alzheimer’s Disease Neuroimaging Initiative
• Excellent example of patient data and associated images with great
sharing model
• However requires access through their own portal and requires
permission from ADNI Data Sharing and Publications Committee
68

69. CTEP (NIH Cancer Therapy Evaluation Program)
Pediatric Brain Tumor Consortium
One of the Better Sources of Data
• As an NCI funded Consortium, the Pediatric Brain Tumor Consortium
(PBTC) is required to make research data available to other investigators
for use in research projects
• An investigator who wishes to use individual patient data from one or
more of the Consortium's completed and published studies must submit in
writing:
• Description of the research project
• Specific data requested
• List of investigators involved with the project
• Affiliated research institutions
• Copy of the requesting investigator's CV must also be provided.
• The submitted research proposal and CV shall be distributed to the PBTC
Steering Committee for review
• Once approved, the responsible investigator will be required to complete a
Material and Data Transfer Agreement as part of the conditions for data
release
• Requests for data will only be considered once the primary study analyses
have been published
69

70. Institutional
Database
General
Practice
Research
Database
70

71. Institutional Database:
VA’s Corporate Data Warehouse Vinci
71

72. Disease Specific Databases Alzheimer’s, Parkinson’s,
Schizophrenia
72

73. Cornucopia of Sources of Data for Dr.
Watson
• University hospital databases
• Large medical system e.g. Kaiser Permanente data warehouse
• Insurance databases such as WellPoint
• State level databases
73

74. Discovering and Consuming Databases
• At best, freely sharable databases are accessed using their own
idiosyncratic web portal
• Currently no index of databases or their content
• No standards exist to describe how databases can “advertise” their
content and availability (free or business model) and their data
provenance and sources and peer review, etc.
• Would be wonderful project for AMIA or NLM to investigate the
creation of an XML standard for describing the content of databases
• This will be critical to the continuing success of the Dr. Watson project
in my opinion
74

75. Medical Guidelines
• Medical guidelines are increasingly being put into machine intelligible
form although this is not an easy process
• Incorporating these into Watson software could serve multiple
purposes including health surveillance, could factor into diagnostic
decision making, and could be an early implementation of the Watson
technology
75

76. Peleska et al: General Graphic Model
Making Guidelines “Formal” and Machine Readable 2003
European Guidelines on Cardiovascular Disease Prevention
and 2003 ESH/ESC Hypertension Guidelines
76

77. The Electronic Medical Record
• The transition to the 3rd year of medical school begins a new phase in
education from theoretical to empirical
• Medical students are exposed for the first time to the wards and of
course, importantly, to one of their major jobs for the next few years:
• Maintenance and review of patient charts, nowadays the Electronic
Medical Record
77

78. Introducing Dr. Watson to the Electronic Medical
Record
78

79. Watson and the EMR
• Despite the tremendous strides we have made toward an electronic
medical record, we are really just at the 1.0 stage and arguably most
current EMR systems really represent just a digital form of paper
• The Watson development team was really surprised when we reviewed
the EMR at how primitive it was, even in 2011
• Lack of ability to search for terms within a patient’s record
• Lack of ability to search across patient records
• Lack of ability to perform basic statistics or have access to basic
decision support tools in EMR
79

80. EMR
• The diagnosis of a specific type of pneumonia, for example, can be
made according to patient signs and symptoms using journal articles
and textbooks
• But it can also be made more reliably by a system such as Watson by
also mining the local EMR database as to what diagnoses have been
made over the past few days, weeks, months, etc. locally
80

81. EMR
• It can then be further refined by not necessarily being constrained to
tentative diagnoses that have been made but the
microbiology/pathology proven causes of pneumonia
• The EMR provides empirical data about the association of these signs
and symptoms with diagnoses and the means to verify what was
found by lab tests etc.
81

82. EMR Challenges
• Challenges mining EMR
• Unstructured free text with abbreviations, variable terms (e.g. MRI
terminology)
• Difficulty in having Watson technology analyze large databases such as
VA’s EMR due to PHI concerns and need to stay within the firewall
• Watson needs to incorporate the concept of changing signs and
symptoms in a patient over time which creates added dimension to
diagnosis of a single patient presentation
• Challenge is the fragmentation of electronic medical records by multiple
hospitals, clinics, outpatient settings, etc.
82

83. EMR Opportunities
• Watson can gain empirical knowledge of vast numbers of physicians
and patients in a way that would not be possible for any single
practitioner
• Watson could use EMR to perform research and discovery in
healthcare such as unanticipated drug responses and interactions and
factors impacting patient response to therapy
• Watson can be impetus to medical community for the development of
more structured EMR in a more friendly machine readable format
83

84. Personal Health Records May Help
Ameliorate Fragmentation of EMR’s Hospital
and Clinics and Offices
• PHR’s will enable Watson to get all information in one place when
patients centralize and take control of their own electronic health
records
• Patients will be able to control level of access to their information
84

85. ADDITIONAL APPLICATIONS
FOR DR. WATSON
85

86. Additional Applications for Dr. Watson
• Surveillance – e.g. Los Alamos Labs
• Bioterrorism
• Drug
• Infectious Disease
86

87. Chart Review and Patient Problem List Sheriff
• Review for patient safety issues
• Computerized patient problem list
• IBM team and I found patient problem list typically poorly maintained
and updated
• Problems not deleted when they are no longer important
• Contradictions in patient problem list
• Patient on medications not corresponding to problems on the list
87

88. Personalized Medicine
• Dr. Watson software can utilize genomic and proteomic information in
addition to patient signs and symptoms to provide personalized
diagnostic and treatment information
• Will be able to utilize an increasing number of genomic and proteomic
databases such as The Cancer Genome Atlas and The Million Veteran
Program
88

89. Synthesis/Display of Complex Information in EMR
89

90. Utilizing the NCI caBIG Semantics and Technologies To
Support Phenotype/Genotype Clinical Analysis for
Personalized Medicine in the Diagnosis of Glioblastoma
Multiforme
90

91. Current Dr. Watson Opportunities for Improvement
• Need to understand to listen and human speech including accents
• Needs to have improved ability to understand abbreviations and
medical jargon
• Needs mechanism to obtain feedback (learn) from physicians using it
• Continue to refine and improve user interface to allow feedback and
refinement of algorithms
91

92. Interactive
• Emergency Department Scenario
• Requires “real-time” decision making
• Cannot use same model with all information entered
into the chart before Watson makes its assessment
and recommendations
• Need better systems to capture information at point
of care
• Vital signs and lab and signal monitoring
• Do we need additional methods of inputting data?
• Do we need to capture live conversations with
providers and patients?
92

93. Current Opportunities for Improvement
• Could use more personality
• Female voice chosen for Siri after much research and feedback
• Needs to understand nuances of communication such as patients
questions expressing emotions such as fear etc.
93

94. Siri: Artificial Intelligence Devices Say the
Darnest Things
94

95. Watson Opportunity:
As Unifier for Interoperability and Test Bed
• Potential for Watson to be bridge to allow connectivity and
interoperability since so many islands currently being set up with
health information exchanges at city and state and other levels
• Watson or Watson like technology may provide test bed for standards
in medicine and may improve interoperability
95

96. Teaching Dr. Watson Bedside Manners
• According to a study done by the Mayo Clinic in 2006, the most
important characteristics patients feel a good doctor must possess
are entirely human
• According to the study, the ideal physician is confident, empathetic,
humane, personal, forthright, respectful, and thorough
• Watson may have proved his cognitive superiority, but can a computer
ever be taught these human attributes needed to negotiate through
patient fear, anxiety, and confusion? Could such a computer ever
come across as sincere?
96

97. Turing Test
• Introduced by Alan Turing in his 1950 paper
“Computing Machinery and Intelligence”
• Opens with the words “I propose to consider the
question, ‘Can machines think?”
• Asks whether a computer could fool a human
being in another room into thinking it was a human
being
• Modified Dr. Watson Turing Test might ask: Can a
computer fool a human being into thinking it was a
doctor?
97

98. Ultimate Challenge: Medical Imaging
Scientific American June 2011
Testing for Consciousness
Alternative to Turning Test
Christof Koch and Giulio Tononi
98

99. Imaging May Be Ultimate/Future Frontier For Dr.
Watson
99

100. Does Watson Obviate Need for
Standards and Structure?
• No, in order to achieve their full potential we
will need to make our medical records more
structured and standardized, and rethink how
we can make our clinical trial and other
research databases more readily discoverable
and reusable
• These changes will also accelerate
interoperability and information exchange
which will improve healthcare
100

101. Conclusions
• I am absolutely convinced that natural language processing and
Artificial Intelligence applications such as IBM’s Dr. Watson will have a
major impact on the practice of medicine in the very near future
• It will result in more cost effective, higher quality care and will help to
decrease the disparities of care that we currently see geographically,
socioeconomically, and according to subspecialty
• It will also allow us to finally achieve true personalized medicine,
taking clinical signs and symptoms and history and laboratory
information and diagnostic imaging and genomics and proteomics
into account to personalize treatment recommendations
101

102. Conclusion
• Dr. Watson will evolve as an amiable, knowledgeable, fast, and reliable
assistant
• If there are any pre-med students out there in the audience, please do
plan to attend medical school and rest assured that Dr. Watson will
require your wisdom, common sense, and humanity in order to be a
continuing and evolving success
102

103. • The Watson Q/A technology and Jeopardy demonstration have
captured the imagination of many people including those in healthcare
and this may provide a critical springboard to revive many of the
excellent initiatives on artificial intelligence applications in medicine
• The potential of these to revolutionize medicine is tremendous and
exciting
103

104. DR. WATSON – A
PROMISING STUDENT IN
PURSUIT OF SMARTER
MEDICINE
Eliot Siegel, M.D.
Professor and Vice Chair University of Maryland Department
of Diagnostic Radiology
104