Timo Honkela: Linking Cognitive Systems, Digital Humanities and Brain Cancer Experiences

Timo Honkela, Neuroscience Seminar presentation, 18.9.2015
Timo Honkela
Neuroscience seminar series, Host Prof. Eero Castrén
University of Helsinki, 18 Sep 2015
Linking Cognitive Systems,
Digital Humanities and Brain
Cancer Experiences
timo.honkela@helsinki.fi

Speaker's
Background

Summary of study & work career
● M.Sc. on human-oriented information systems
development at University of Oulu
● Sitra's Kielikone project
● VTT Information Technology
● Neural Networks Research Center, Helsinki University of
Technology, PhD
● Media Lab, University of Art and Design Helsinki,
professor
● TKK > Aalto University, head of Cognitive Systems group
● University of Helsinki and National Library of Finland,
professor

Hand-crafted,
symbol manipulation
based AI

Early personal experiences on
rule-based natural language processing
● H. Jäppinen, T. Honkela, H. Hyötyniemi & A. Lehtola (1988):
A Multilevel Natural Language Processing Model.
Nordic Journal of Linguistics 11:69-87.
What is the turnover of the ten largest stock exchange companies in forestry?
Morphological analysis
Dependency parsing
Logical analysis
Database query formation
Result from the SQL database

Neural network /
machine learning based AI

Classical example: Learning meaning from context:
Maps of words in Grimm fairy tales
Honkela, Pulkki & Kohonen 1995

Chemistry
Natural sciences
and engineering
Bio- and
environmental
sciences
Health
Culture and
society
Map of Finnish Science

Multimodally
grounded AI

Labeling movements: Associating
high-dim. kinesthetic time series
with linguistic labels
Förger & Honkela 2014

RUNNING
WALKING
LIMPING
JOGGING
Förger & Honkela 2014

Cognitive
Systems

Ambiguity (homography & polysemy)
and contextuality: case “ALUSTA”
● “ALUSTA”
"alku" N ELA SG
"alusta" N NOM SG
"alustaa" V PRES ACT NEG
"alustaa" V IMPV ACT SG2
"alustaa" V IMPV ACT NEG SG
"alunen" N PTV SG
"alus" N PTV SG
FINTWOL: Finnish Morphological Analyser
Copyright © Kimmo Koskenniemi & Lingsoft Oy 1995 – 2012
http://www2.lingsoft.fi/cgi-bin/fintwol
Alusta
Monta alusta
Näin monta alusta
Näin monta alusta
satamassa

and contextuality: case “ALUSTA”
● “ALUSTA”
"alku" N ELA SG
"alusta" N NOM SG
"alustaa" V PRES ACT NEG
"alustaa" V IMPV ACT SG2
"alustaa" V IMPV ACT NEG SG
"alunen" N PTV SG
"alus" N PTV SG
FINTWOL: Finnish Morphological Analyser
Copyright © Kimmo Koskenniemi & Lingsoft Oy 1995 – 2012
http://www2.lingsoft.fi/cgi-bin/fintwol
Alusta
Monta alusta
Näin monta alusta
Näin monta alusta
satamassa alas
taivaalta
http://favim.com/image/92863/

and contextuality: case “GET”
●
“ S: (v) get, acquire (come into the possession of something concrete or abstract) "She got a lot of paintings from her uncle"; "They acquired a new pet"; "Get your results the next day"; "Get permission to take a few days off from work"
●
S: (v) become, go, get (enter or assume a certain state or condition) "He became annoyed when he heard the bad news"; "It must be getting more serious"; "her face went red with anger"; "She went into ecstasy"; "Get going!"
●
S: (v) get, let, have (cause to move; cause to be in a certain position or condition) "He got his squad on the ball"; "This let me in for a big surprise"; "He got a girl into trouble"
●
S: (v) receive, get, find, obtain, incur (receive a specified treatment (abstract)) "These aspects of civilization do not find expression or receive an interpretation"; "His movie received a good review"; "I got nothing but trouble for my good intentions"
●
S: (v) arrive, get, come (reach a destination; arrive by movement or progress) "She arrived home at 7 o'clock"; "She didn't get to Chicago until after midnight"
●
S: (v) bring, get, convey, fetch (go or come after and bring or take back) "Get me those books over there, please"; "Could you bring the wine?"; "The dog fetched the hat"
●
S: (v) experience, receive, have, get (go through (mental or physical states or experiences)) "get an idea"; "experience vertigo"; "get nauseous"; "receive injuries"; "have a feeling"
●
S: (v) pay back, pay off, get, fix (take vengeance on or get even) "We'll get them!"; "That'll fix him good!"; "This time I got him"
●
S: (v) have, get, make (achieve a point or goal) "Nicklaus had a 70"; "The Brazilian team got 4 goals"; "She made 29 points that day"
●
S: (v) induce, stimulate, cause, have, get, make (cause to do; cause to act in a specified manner) "The ads induced me to buy a VCR"; "My children finally got me to buy a computer"; "My wife made me buy a new sofa"
●
S: (v) get, catch, capture (succeed in catching or seizing, especially after a chase) "We finally got the suspect"; "Did you catch the thief?"
●
S: (v) grow, develop, produce, get, acquire (come to have or undergo a change of (physical features and attributes)) "He grew a beard"; "The patient developed abdominal pains"; "I got funny spots all over my body"; "Well-developed breasts"
●
S: (v) contract, take, get (be stricken by an illness, fall victim to an illness) "He got AIDS"; "She came down with pneumonia"; "She took a chill"
●
S: (v) get (communicate with a place or person; establish communication with, as if by telephone) "Bill called this number and he got Mary"; "The operator couldn't get Kobe because of the earthquake"
●
S: (v) make, get (give certain properties to something) "get someone mad"; "She made us look silly"; "He made a fool of himself at the meeting"; "Don't make this into a big deal"; "This invention will make you a millionaire"; "Make yourself clear"
●
S: (v) drive, get, aim (move into a desired direction of discourse) "What are you driving at?"
●
S: (v) catch, get (grasp with the mind or develop an understanding of) "did you catch that allusion?"; "We caught something of his theory in the lecture"; "don't catch your meaning"; "did you get it?"; "She didn't get the joke"; "I just don't get him"
●
S: (v) catch, arrest, get (attract and fix) "His look caught her"; "She caught his eye"; "Catch the attention of the waiter"
●
S: (v) get, catch (reach with a blow or hit in a particular spot) "the rock caught her in the back of the head"; "The blow got him in the back"; "The punch caught him in the stomach"
●
S: (v) get (reach by calculation) "What do you get when you add up these numbers?"
●
S: (v) get (acquire as a result of some effort or action) "You cannot get water out of a stone"; "Where did she get these news?"
●
S: (v) get (purchase) "What did you get at the toy store?"
●
S: (v) catch, get (perceive by hearing) "I didn't catch your name"; "She didn't get his name when they met the first time"
●
S: (v) catch, get (suffer from the receipt of) "She will catch hell for this behavior!"
●
S: (v) get, receive (receive as a retribution or punishment) "He got 5 years in prison"
●
S: (v) scram, buzz off, fuck off, get, bugger off (leave immediately; used usually in the imperative form) "Scram!"
●
S: (v) get (reach and board) "She got the bus just as it was leaving"
●
S: (v) get, get under one's skin (irritate) "Her childish behavior really get to me"; "His lying really gets me"
●
S: (v) get (evoke an emotional response) "Brahms's `Requiem' gets me every time"
●
S: (v) catch, get (apprehend and reproduce accurately) "She really caught the spirit of the place in her drawings"; "She got the mood just right in her photographs"
●
S: (v) draw, get (earn or achieve a base by being walked by the pitcher) "He drew a base on balls"
●
S: (v) get (overcome or destroy) "The ice storm got my hibiscus"; "the cat got the goldfish"
●
S: (v) perplex, vex, stick, get, puzzle, mystify, baffle, beat, pose, bewilder, flummox, stupefy, nonplus, gravel, amaze, dumbfound (be a mystery or bewildering to) "This beats me!"; "Got me--I don't know the answer!"; "a vexing problem"; "
This question really stuck me"
●
S: (v) get down, begin, get, start out, start, set about, set out, commence (take the first step or steps in carrying out an action) "We began working at dawn"; "Who will start?"; "Get working as soon as the sun rises!"; "The first tourists began to arrive
in Cambodia"; "He began early in the day"; "Let's get down to work now"
●
S: (v) suffer, sustain, have, get (undergo (as of injuries and illnesses)) "She suffered a fracture in the accident"; "He had an insulin shock after eating three candy bars"; "She got a bruise on her leg"; "He got his arm broken in the scuffle"
●
S: (v) beget, get, engender, father, mother, sire, generate, bring forth (make (offspring) by reproduction) "Abraham begot Isaac"; "John fathered four daughters"
W
ordN
et3.1

Effect of context: Color naming
Human vision: rods, cones,...
Physical reasons for color
Contextuality of naming
red wine
red skin
red shirt
Hardin
Gärdenfors

User-specific difficulty measure

All users have their own background knowledge and
vocabulary: different texts are difficult for different
people -> need for user modeling
Paukkeri, Ollikainen & Honkela, 2013

User-specific difficulty measure
Paukkeri, Ollikainen &
Honkela, 2013

Tensor-based analysis of
subjectivity

GICA: Grounded Intersubjective
Concept Analysis
Sanat,
fraasit,
tulkinnat tms.
Kontekstit
Yksilöt

The word “health” in
State of the Union Addresses
Subjects on objects in contexts:
Using GICA method to quantify
epistemological subjectivity.
Timo Honkela, Juha Raitio, Krista Lagus,
Ilari T. Nieminen, Nina Honkela, and Mika Pantzar.
Proc. of IJCNN 2012.

Implications of
multiperspective knowledge engineering
● Descriptions of contents need to be
standardized to a lesser degree
because systems learn to create
mappings between different
conceptual systems
● In the future, machines can facilitate
meaning negotiations between, e.g., experts
of different disciplines or between experts and
laypersons

Digital
Humanities
Case: Historical Newspaper Collection
of the National Library of Finland

Digital humanities
● Research within humanities
with the help of computers
– Digital resources
– Computational models
● Basic motivation
– One can already fly to moon and
build sophisticated factory products
– The most important open questions
in the world are related to humanities
and social sciences

Digital Computational
Humanities
Content
storage and
transfer
Content
analysis

Accessing and analyzing
digital resources

Resources
Content and
information
professional
Users of
the contents
(professionals
and lay people)
Machine learning
and
pattern recognition
systems
Formal metadata
Language
technology
resources and
systems
Other forms of description

Resources
Users of
the contents
(professionals
and lay people)
Other forms of description
Crowdsourcing
Importance
of openness

Historical newspaper collection
● The National Library of Finland has digitized a
large proportion of the historical newspapers
published in Finland between 1771 and 1910
(Bremer-Laamanen 2001, 2005).
● This collection contains approximately 1.95
million pages in Finnish and Swedish
● According to Legal Deposit law, the National
Library of Finland receives a copy of each
newspaper and magazine published in Finland.

Search interface
http://digi.kansalliskirjasto.fi

FIN-CLARIN corpus
www.kielipankki.fi

Challenges of pattern recognition

OCR Challenges
● Regardless of recent development of the OCR
software, there are still challenges with it, as
some material is very old, with
– varying paper and print quality,
– varying number of columns and layout patterns,
– different languages (mainly Finnish and Swedish
but also French, German, etc.), and
– varying font types (fraktur and antiqua)

zzhdysvautki Yhdyspankki
v, u, p ? u, n, ll ?

A very long tail of low frequency forms...

taioafliftiutpn tawallisuuden

Similarity diagram of Fraktur letter shapes
(a self-organizing map)

Areas of further analysis
● Multidimensional sentiment analysis
● Analysis of social and
historical context
● Intercultural and
multilingual analysis
● Analysis of points of view
● Comparison with other data sets such as modern
newspapers or social media discussions
● Analysis of subjective
understanding

Brain
Cancer

Einsiedeln,
Switzerland,
May 2014
The right side of
my visual field
disappeared
Helsinki,
May 2014
TIA (transient
ischemic attack)
negative
Somethins white behind,
maybe an old stroke.

Mikkeli,
September
2014
I am completely exhausted
and my right hand
does not move
normally
Helsinki,
September
2014
A sick leave is needed
for your burnout
after working 70 hours
per week

Espoo,
November
2014
My husband, normally
an intelligent professor is
“our of this world”. I take
him to the hospital now!
Espoo,
Jorvi,
Nov 2014
Normally we would force
you to go through
your occupational doctor but
as you don't know which year it is...
You have a large
tumor in your brain

Treatments
● Surgical opetation
● Radiation therapy
● Cytostatic drugs
● Good nursing
● Psychological consultations
● Physiotherapeutic consultations
Supportive action
Other medical actions
● Neuro-oftalmologic consult.
● Neurologic consultations
● Psychiatric consultations
● MRI

Treatments
● Surgical operation
● Radiation therapy
● Cytostatic drugs
● Good nursing
● Psychological consultations
● Physiotherapeutic consultations
Supportive action
Other medical actions
● Neuro-oftalmologic consult.
● Neurologic consultations
● Psychiatric consultations
● MRI

Operation 1st of Dec 2014: A complete
success but unfortunately malignant ...
Operated by world famous prof. Hernesniemi

Expressing
my deepest
gratitude.
Wishing more
funding wherever
needed and
better health care
information systems.

Radiation
therapy 6
weeks plus
low dose of
cytostatic
drugs
Six months of cytostatic
treatment 5 days/month
300 mg/day

Plan for the
radiation therapy
carefully avoiding
the brain stem.

MRI
- Dec 2014 (om the left)
- Spring 2015
- Left visual cortex defected
- Additional problems caused
by inflammation, cortisone helped
quickly

Blood tests to check if treatment can be continued

Visual field
impairment

After the operation:
“Random” top-down simulation
Nowadays:
“Data-driven” simulation

Own actions
● Optimistic attitude
● Enjoying life
excl. dark moments
● Family support
● Social relations
● Peer support
● Physical exercise
● Mental exercises
● Therapeutic effect of
music
● “Ten changes”
regime
● Visualization
exercising experiment
● Nutrition:
healthy & enjoyable
● Light work (opposite
to the old scheme)
● Openness

Colleagues visiting Jorvi two days before operation.
Discussions strictly on Digital Humanities.

Social media
forum
Etelä-Suomen
syöpäyhdistys

Dec 14, 2014
Dec26,2014
Jun, 2015
Sep9,2015
Aug, 2015

Christmas
porridge

Work experiment
● 20 hours per week
● Supported by KEVA
● Started 1st of September
● Basically 4 hours per day,
partly at home

Helsinki Challenge
pitch Sep 3, 2015

Planning next steps

Ideas,
opportunies

Open data volunteers
● People could donate their health and life style
data for research
● This attitude could be promoted as a primary
moral choice that helps other people
● If widely adopted, this kind of practice requires
strongly integrity from the people who use the
data

High-dimensional health research
● It would be useful to study very high-
dimensional data sets with an open minded
attitude
– Gene data
– Life style factors (exercise, nutrition, etc.)
– Emotions
– Environment (e.g. chemical risk factors)
● These should be analyzed with latest data-
driven statistical machine learning methods

“Mind-body problem”
● Philosophers have considered different
options on how mind and body are related or if
they are the same
● Medical doctors often seem to set this issue
aside, refer to the plasebo effect, or send the
patient to a psychiatrist
● However, there seems to be more in this issue
that would require more careful research than
what has been conducted so far

Adaptive neuro-physiological
systems
● An opportunity woud be to conduct interdisciplinary
research in which human health would be studies
concurrently from multiple perspectives
● The data would be gathered at multiple levels of
detail and abstraction including gene data,
immunological data, hormonal process, emotional
factors, etc.
● This data could be collected including both
objective measurements and subjective
assessments

Measuring the subjective
● It is well known that the variation in how people
describe their condition is high even if they seem
to suffer of the same condition
● Medical doctors may have developed some kind
of sensitivity to hypocondria but in general this
modeling is most likely quite superficial
● There are new methods that could be used to
model the subjectivity in a more fine grained
manner, helping both the medical professionals
and the patients

Thank you for
your attention!

Timo Honkela: Linking Cognitive Systems, Digital Humanities and Brain Cancer Experiences

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

More from Timo Honkela

More from Timo Honkela (20)

Recently uploaded

Recently uploaded (20)

Timo Honkela: Linking Cognitive Systems, Digital Humanities and Brain Cancer Experiences