Dan Lee from Dentuit AI presented an Intro to Deep Learning for Medical Image Analysis at the Maryland AI meetup (https://www.meetup.com/Maryland-AI), May 27, 2020. Visit https://www.youtube.com/watch?v=xl8i7CGDQi0 for video.

1. Introduction to Deep Learning for
Medical Image Analysis
MD-AI Meetup // May 27, 2020
Dan Lee Co-founder & CEO // Dentuit Imaging

2. Motivation?
1. Challenge myself to refresh, learn, and clearly communicate
my understanding of a rapidly evolving technology
2. Attempt to derail the AI hype train - and share some real world
use cases
3. Share a glimpse into the future of medicine/dentistry.
4. Inspire you to pursue self-learning on a particular detail that
interested you (links for further reading are shared at the end)
5. And finally ...

3. a quick plug on our awesome
startup & partners:
Dentuit Imaging

4. Today

5. Current digital “solutions”

6. But what if ….

7. “... having work done
at a dental school
clinic inherently
involves a huge
amount of oversight:
every diagnosis and
filling is checked over
by several students
and professors ...”
link to Vox article

9. Why Medical Imaging?
Meaningful information drives
better decision making.
❖ screen for possible health conditions before
symptoms appear.
❖ diagnose the likely cause of existing symptoms.
❖ monitor health conditions that have been diagnosed,
or the effects of treatment for them.

10. Types of Medical Imaging
❖ Plain X-ray
❖ Computed tomography (CT
scans)
❖ Nuclear medicine imaging
including positron-emission
tomography (PET)

11. Types of Medical Imaging
Medical Imaging not always Radiographic:
❖ Magnetic resonance imaging (MRI)
❖ Ultrasound (sound waves)
❖ Endoscopy
… and many more. Okay … So What?
(we’ll come back to answer that question in a later slide)

12. Neural Networks (NNs)
Neural networks are a set of algorithms, modeled loosely after the human
brain, that are designed to recognize patterns.
They interpret sensory data through a kind of machine perception, labeling or
clustering raw input.
The patterns they recognize are numerical, contained in vectors, into
which all real-world data, be it images, sound, text or time series, must be
translated.

13. [link]
A node combines input from the
data with a set of coefficients, or
weights, that either amplify or
dampen that input, thereby
assigning significance to inputs with
regard to the task the algorithm is
trying to learn; e.g. which input is
most helpful is classifying data
without error?
These input-weight products are
summed and then the sum is
passed through a node’s so-called
activation function, to determine
whether and to what extent that
signal should progress further
through the network to affect the
ultimate outcome, say, an act of
classification. If the signals passes
through, the neuron has been
“activated.”

14. Quick detour for a History lesson
❖ 1950s: Artificial Neural Network (ANNs) were introduced
➢ loosely modeled after the neurons in a biological brain (see slides on NNs)
➢ lack of computing power + absence of sufficient data
■ remember the earlier slide on advances in medical imaging?
❖ 1980s: Machine learning (ML) algorithms introduced for classification tasks
➢ (See next slide)
❖ Early 2000s: computer-aided detection (CAD) systems were developed and
introduced in the clinical workflow
➢ More false-positives than human readers
➢ Led to a greater assessment time and additional biopsies
❖ Present: Deep Neural Networks (DNNs)

15. Classical ML to Modern DL

16. Deep Learning (DL)
Deep learning is a subtype of representation learning which aims to
describe complex data representations using simpler hierarchized structures
defined from a set of specific features. [link]
Let’s specifically examine CNNs to better understand this definition.

17. A Convolutional Neural Network (CNN) is a type of neural network
that specializes in image recognition and computer vision tasks.
CNNs have two main parts:
1. A convolution/pooling
mechanism that breaks up
the image into features
and analyzes them
2. A fully connected layer
that takes the output of
convolution/pooling and
predicts the best label to
describe the image

18. convolution/pooling mechanism that breaks up the image into
features and analyzes them
[link]

19. (visualize the convolution/pooling in action)

20. Neural Networks - Architecture?
The term neural network architecture refers to the:
(a) arrangement of neurons into layers
(b) and the connection patterns between layers, activation
functions, and learning methods.
Therefore: the neural network model and the architecture of a
neural network determine how a network transforms its input into
an output.

21. some examples typically used for medical image analysis
● AlexNet
● VGG
● GoogLeNet
● ResNet
● Highway nets
● DenseNet
● GANs
● ResNext
● SENets
● NASNet
● YOLO
● Siamese Nets
● U-net
● V-net
Graphing their accuracy versus amount
of operations required for a single forward
pass.

23. clinical uses of DL techniques

24. Convolutional Neural
Networks are successful
for simpler image tasks
such as classification
Not for more complex
ones like localization
and segmentation.
BUT ...

25. In biomedical cases, it requires us not only to distinguish whether
there is a disease, but also to localise the area of abnormality.
This is where other algorithms like U-Net and ResNet
architectures come into play.

26. U-net: first realized as suggestion for better
segmentation on biomedical images.
In CNN, the image is converted
into a vector which is largely
used in classification problems.
But in U-Net, an image is
converted into a vector and
then the same mapping is used
to convert it again to an image.
SIMPLY PUT: It converts the
segmentation problem into a
classification problem where
we need to classify each
pixel to one of the classes.

27. Residual Networks (ResNet) addresses the
“vanishing gradient” problem
In traditional neural
networks, each layer feeds
into the next layer. But in
a network with residual
blocks, each layer feeds
into the next layer and
directly into the layers
about some hops away.

28. Notable Breakthroughs: CheXNet (Nov 2017)
“CheXNet” is a type of image
analysing AI called a DenseNet
(a variant of a ConvNet, similar
to a ResNet) that was trained to
detect abnormalities on chest
x-rays, using the ChestXray14
dataset.

29. Notable Breakthroughs:
Google Health/DeepMind (Jan 2020)

30. Challenges: Limited Datasets
● Medical images are protected by privacy & legal frameworks e.g., HIPAA
● Medical annotations are Costly & specialized labelers
● Siloed databases with little to no interoperability
The Workarounds ...
● Data augmentation
● De-identification protocol (Safe Harbor, Expert determination)
● Crowdsourced (from medical community) annotation platforms
● Distributed & federated learning
● U-nets, etc.
More better approaches are still TBD in current and future research

31. Challenges: Real world Performance

32. Other Noteworthy Challenges
● Data Standards
● Algorithmic Bias
● Privacy and Legal Issues
● Uninterpretable Black Box Model
… and many more - just not enough time to cover them all here.

33. ● A Comprehensive Introduction to Different Types of Convolutions
in Deep Learning [link]
● JAMA’s Machine Learning For Medical Image Analysis - How It
Works [link]
● Udacity’s Virtual Conference on AI for Healthcare (webinar
recordings) [link]
● Stanford’s AIMI Center - Shared Datasets [link]
● Deep Learning Book [link]
● NVIDIA Deep Learning Institute (DLI) [link]
● Introduction to U-Net and Res-Net for Image Segmentation [link]

34. Q&A
Dan Lee
hello@dentuit.ai
Website launching soon - stay tuned.