This document discusses conditional random fields (CRFs) and their use for named entity recognition and event extraction tasks. It notes that CRFs address the label bias problem of hidden Markov models (HMMs) and maximum entropy Markov models (MEMMs) by conditioning on observations and considering many feature types. CRFs model transition probabilities between states and state probabilities conditioned on observations, using edge features between labels as well as vertex features of the observations.

1. BIONLP09
and
CRFs
Farzaneh Sarafraz
18 February 2009

2. BioNLP'09
Event rather than entity

Most entities are given

3 tasks

− Event detection and characterization
− Event argument recognition
− Negations and speculations

3. Example
quot;I kappa B/MAD­3 masks the nuclear localization
signal of NF­kappa B p65 and requires the
transactivation domain to inhibit NF­kappa B
p65 DNA binding. quot;
Event: negative regulation
Trigger: masks
Theme1: the first p65
Cause: MAD­3
Site: nuclear localization signal

4. Example
quot;In contrast, NF­kappa B p50 alone fails to
stimulate kappa B­directed transcription, and
based on prior in vitro studies, is not
directly regulated by I kappa B. quot;
Event: regulation
Theme1: this p50
Trigger: regulated
Negation: true for this event
Speculation: none

5. HMM and MEMM
(X1, X2, ...)
Observations
(Y1, Y2, ...)
labels
p(Xi , Yi)

X ranges over observation sequence
Y ranges over and label sequence
Requires independence assumption

i.e. each item is labelled independently

6. Conditional Random Field
p(Y |X)

Y: label sequence
X: observation sequence
Maximise p


7. MMEM Label Bias Problem
Probability given the current state

− Transitions leaving a state compete against
each other

not all states

− Per­state normalization
− Probability bias towards states with few transitions
− Demonstrated experimentall

8. Label Bias Example
Training data:

− A B C D
− A B D D
− A B C E
− A B D C
Model says:

− C > D 50%
− C > E 50%
Why predict E when D is much more common?


9. CRF Solution
Model probability of transitions and probability

of states
CRFs

− Models probability of transition between states
− Probability is conditional on current observation
− Not normalised
− Considers many quot;featuresquot; of observations

10. Features
quot;edge featuresquot; as well as quot;vertex featuresquot;

− Word is capitalized
− Word ends in quot;­ingquot;
− Label is quot;proper nounquot;
Features are important!


11. End.