Xenia miscouridou wi mlds 4

PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices
X Miscouridou 1 / 33

Outline
1 Research overview
From probabilistic modelling to deep learning
2 Deep dive in probabilistic modelling
Understanding social interactions
3 Open questions
Bridging the gap between statistics and computation

Outline
1 PhD and research overview
2 Deep dive in probabilistic modelling for interaction data
3 Open Questions

My research map
Statistical Machine Learning
Probabilistic Modelling
Model a phenomenon using
random variables and
probability distributions
Statistical methods
Expressivity and explainability
Accuracy
Deep Learning
Algorithms inspired by the
structure of neural networks in
the brain
Computation
Artificial neural networks
Prediction
Goals of the two areas may differ but combining their strengths is promising

Outline
Setup
Modelling Temporal interactions
Inference and performance
3 Open Questions

Setup
Modelling temporal interactions
Model temporal interaction data in triplets of the form
(i, j, t)
corresponding to an action from a node i to node j at time t
Which are the factors that underpin these interactions?
Which properties are observed in the underlying network that i, j belong to?
Arising in
social networks
biological data
neural activity
Aiming to
uncover trends
predict future links
assist in decision making

Setup
B

Setup
A
B
2.1

Setup
A
B
2.1, 2.8

Setup
A
B
2.1, 2.8
5.6

Setup
A
B
2.1, 2.8, 7.5
5.6

Setup
A
B
2.1, 2.8, 7.5
5.6, 7.6

Setup
A
B
2.1, 2.8, 7.5
5.6, 7.6, 7.8

Setup
A
B
2.1, 2.8, 7.5, 8.1
5.6, 7.6, 7.8

Setup
A
B
2.1, 2.8, 7.5, 8.1, 8.4
5.6, 7.6, 7.8

Which is the key property and how is it modelled?
Reciprocity: an action from A to B increases the chances of a similar
action being returned in the near future
Modelled by inhomogeneous point processes:
N(t):Counting process representing the number of events up to t
λ(t): Intensity function driving number of events at time t
N(t) ∼ HP (λ (t)) ,
P (N (t + dt) − N (t) = 1|Ht) = λ (t) dt,
where Ht is the subset of events up to time t

Mutually exciting processes
Model mutual excitation between a pair using two point processes
(NAB, NBA) with intensities (λAB, λBA)
NAB(t) =
P
events up to time t with sender A, receiver B
NBA(t) =
P
events up to time t with sender B, receiver A
Mathematically,
λAB(t) = µ +
Z t
0
gAB(t − u) dNBA(u)
λBA(t) = µ +
Z t
0
gBA(t − u) dNAB(u)
where µ = λAB(0) = λBA(0) > 0 are symmetric and
gAB, gBA are non-negative functions

Recall the (A,B) pair example
A
B
2.1, 2.8, 7.5, 8.1, 8.4
5.6, 7.6, 7.8

Intensity and counting process graphically
A −→ B B −→ A

Model all pairs in the network
B
A C
D
E
2.1, 2.8, 7.5, 8.1, 8.4
5.6, 7.6, 7.8 2
1.3
2
2.4, 5.6, 9.3
7.8
0.5, 1.5
1.2
1.0, 2.0, 2.3

Model all the pairs in a network
For each directed pair (i, j) let Nij(t) be the counting process for
the events from i towards j, with intensity process λij(t)
Nij, Nji are mutually exciting via their exponential intensities
λij(t) = µij +
Z t
0
ηe−δ(t−u)
dNji(u)
λji(t) = µji +
Z t
0
ηe−δ(t−u)
dNij(u)
µij = µji

Global network structure
Through the base intensities µij we model the network structure.
Sparsity/Density
Heterogeneous behaviour
Communities
a. Dense b. Slightly Sparse c. Sparse d. Very Sparse

How is this model family useful and novel?
1 Provides accurate prediction of number of future links
2 Can be used to measure effects of interventions
3 Allows for personalization
4 Provides a good trade-off in explainability - scalability

References
Miscouridou X., Caron F. and Teh W.Y.
Modelling sparsity, heterogeneity, reciprocity and community structure in temporal interaction data.
Neural Information Processing Systems, 2018
Miscouridou X., Caron F. and Teh Y.W.
Code for Hawkes Processes on Dynamic Graphs
https: // github. com/ OxCSML-BayesNP/ HawkesNetOC , 2018
Todeschini, A., Miscouridou X.,and Caron, F.
Exchangeable random measures for sparse and modular graphs with overlapping communities.
Journal of the Royal Statistical Society, 2020.
Todeschini, A., Miscouridou X.,and Caron, F.
NetOC : Matlab package for Sparse Stochastic Blockmodels
https: // github. com/ OxCSML-BayesNP/ SNetOC , 2019.

Outline
3 Open Questions
Bridging the gap between Statistics and Computation

Bridging the gap between Statistics & Computation
Combining the strengths of both contributes to effective and richer AI
Statistical methods
Can we scale them up and make
them efficient?
Deep Learning
Can we make it more
probabilistically principled?

Recap
1 Research overview
Probabilistic modelling to deep learning
2 Deep dive in probabilistic modelling
A bayesian probabilistic model for social interactions
3 Open questions
Bridging the gap between probabilistic and deep learning

Q&A
Thank You

Bibliography I
Lee J., Miscouridou X., Caron F.
A unified construction for series representations and finite approximations of completely random measures
submitted to Bernoulli 2019.
Miscouridou X., Caron F. and Teh W.Y. (2018).
modelling sparsity, reciprocity, heterogeneity and community structure in temporal interaction data.
Neural Information Processing Systems 2018 .
Miscouridou X., Perotte A., Elhadad N., Ranganath R.
Deep Survival Analysis: Nonparametrics and Missingness
Proceedings of Machine Learning Research.
Todeschini, A., Miscouridou X.,and Caron, F. (2019).
Exchangeable random measures for sparse and modular graphs with overlapping communities.
Journal of the Royal Statistical Society, 2019.
Willetts M., Miscouridou X., Teh Y.W. , Holmes C, Roberts S.
Relaxed Responsibility Vector Quantization, Under Review
.

Kernel parameters
The kernel is responsible for interactions from i to j that respond to
previous interactions from j to i.
Assumptions:
Among all nodes in the network there is global behavior on the
reciprocation
Hyperparameters
η ≥ 0 : size of the excitation jump
δ > 0 : constant rate of exponential decay
The stationarity condition for the processes is η < δ

Base parameters
The base intensity µij is responsible for the interactions from i to j that
arise from similar affiliations and link to sparsity and heterogeneity
Assumptions
A set of positive latent parameters (wi1, . . . , wip) ∈ Rp
+, ∀i
wik: level of its affiliation to each latent community k = 1, . . . , p.
The base rate is given by
µij = µji =
p
X
k=1
wikwjk (1)
Assortativity: Nodes with high levels of affiliation to the same
communities are more likely to interact

Datasets
• Email : emails sent within a research institution.
T = 803 days, N = 986, E = 24929, I = 332334
• College: private messages on an online social network
T = 193 days, N = 1899, E = 20296, I = 59835
.
• Math: stack exchange website Math Overflow
T = 2350 days, N = 24818, E = 239978, I = 506550
.
• The Ubuntu: stack exchange website Ask Ubuntu
T = 2613 days, N = 159316, E = 596933, I = 964437

Link prediction performance across different models
MSE between true and predicted number of links Properties of the six different models
email college math ubuntu
Hawkes-CCRM 10.95 1.88 20.07 29.1
CCRM 12.08 2.90 89.0 36.5
Hawkes-IRM 14.2 3.56 96.9 59.5
Poisson-IRM 31.7 15.7 204.7 79.3
Hawkes 154.8 153.29 220.10 191.39
Poisson ∼ 103 ∼ 104 ∼ 104 ∼ 104
sparsity/ community reciprocity
heterogeneity structure
X X X
X X
X X
X
X

Xenia miscouridou wi mlds 4

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Similar to Xenia miscouridou wi mlds 4

Similar to Xenia miscouridou wi mlds 4 (20)

More from Paris Women in Machine Learning and Data Science

More from Paris Women in Machine Learning and Data Science (20)

Recently uploaded

Recently uploaded (20)

Xenia miscouridou wi mlds 4