1. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
Bayesian Nonparametric Modelling for
Space-Time Emission Tomography
Éric Barat1
joint work with Mame Diarra Fall
1
Laboratory of Modelling, Simulation and Systems
CEA–LIST
October 25, 2010
Séminaire Parisien de Statistiques
NPB Modelling for Space-Time Emission Tomography October 25, 2010 1 / 46

2. Éric Barat
Positron Emission
tomography
Physics
Idealized PET
Usual approach
Limitations
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
Positron Emission Tomography (PET)
Exploiting physical properties of positron annihilation.
PET: a molecular imaging modality.
Physical basis:
Radioactively tagged
molecule injected to the
patient (glucose + 18
F).
(β+
, β−
) annihilation →
two-photons emission in
opposite directions.
Event: detectors record
photons pair coincidence.
Mathematical modelling.
Image reconstruction
(2D/3D).
3D + time: space-time
activity distribution →
metabolic imaging.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 2 / 46

3. Éric Barat
Positron Emission
tomography
Physics
Idealized PET
Usual approach
Limitations
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
Positron Emission Tomography (PET)
An idealized 2D problem.
u
φ
D2 l
D1
ρu = D1D2
2
A
+
Figure: Patient, detector circle and parameterized line of response.
Ideal detector circle and Radon transform
Denote by g(x1, x2) the activity pdf in brain space.
All (φ, u) are exactly observable → pdf in the detector space
f (φ, u) = 1
2ρu
ρu
−ρu
g(u cos φ − t sin φ, u sin φ + t cos φ)dt.
The integral above is the so-called Radon transform of g.
Reconstruction = inversion of the mapping of densities.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 3 / 46

4. Éric Barat
Positron Emission
tomography
Physics
Idealized PET
Usual approach
Limitations
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
Positron Emission Tomography (PET)
Optimality of linear reconstruction in idealized system.
Linear estimators for inversion of the Radon transform (2D)
Known as filtered backprojection (FPB), see Natterer (1986).
ˆgn(x) =
1
n
n
i=1
Kη( φi , x − ui )
with n = # events, φ = (cos(φ), sin(φ)) , x = (x1, x2) and
Kη(t) ∝
1
η
0
r cos(t r)dr
Fourier
−→ FKη
(ν) ∝ |ν| 1|ν|≤ 1
η
Kη is a band-limited filter (≈ kernel with window η).
See Johnstone and Silverman (1990), Cavalier (2000) for
rates of convergence and efficiency of linear estimators in
idealized situation (mildly ill-posed problem).
Smoothness p: direct data: rD ≈ (log n
n
)
p
p+1 ; indirect: rI ≈ (1
n
)
p
p+2 .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 4 / 46

5. Éric Barat
Positron Emission
tomography
Physics
Idealized PET
Usual approach
Limitations
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
Positron Emission Tomography (PET)
Real life systems.
u
φ
D2 l
D1
ρu = D1D2
2
A
+
Figure: Finite size detectors ring.
Physical limitations → data more incomplete (wrt Radon)
Detectors geometry → only a limited number of distinct
detectors pair coordinates values y = {D1, D2} are observed.
Finite pmf in detector space → more strongly ill-posed.
Results on rates may not hold for linear estimation of a
continuous distribution from a finite number of projections.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 5 / 46

6. Éric Barat
Positron Emission
tomography
Physics
Idealized PET
Usual approach
Limitations
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
Positron Emission Tomography (PET)
Usual algorithms: parametric models and discrete-discrete reconstruction.
Brain space discretization: voxels (pixels) basis function
g(x) =
K
k=1 gk
1(x∈vk )
|v| where |v| = volume of voxels vk .
fy = X
p(y|x)g(x)dx =
K
k=1 ay,k gk .
e.g. Radon: p(y|x) ∝ δ( φy, x − uy).
Denote ny=# events recorded in y: ny|g ∼ Poisson(fy)
→ Parametric Poisson inverse problem framework.
Penalized likelihood estimators of g = {g1, . . . , gK }
ˆg = argmax
g>0
(log L(g|{ny}) + λΨ(g)).
Expectation-Maximization algorithm.
ML estimator (λ = 0): Shepp and Vardi (1982).
MAP estimator: exp(λΨ(g)) = prior on g, e.g. Gibbs field,
see Green (1990).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 6 / 46

7. Éric Barat
Positron Emission
tomography
Physics
Idealized PET
Usual approach
Limitations
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
Positron Emission Tomography (PET)
Parametric approach limitations.
Voxel size impact on reconstruction
Regularization parameter.
Trade-off between reconstruction noise and resolution.
Finite model limitations
Do we really trust in a discretized brain structure ?
How do we choose the voxel size ?
Can we give an interpretation to models with several millions
(3D) or billions (4D) of parameters ?
Do Gibbs fields correspond to biological structures prior ?
→ model selection and averaging ?
Nonparametric discrete to continuous reconstruction ?
Solutions in the whole space of probability measures M(X) ?
Regularization ?
NPB Modelling for Space-Time Emission Tomography October 25, 2010 7 / 46

8. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
Nonparametric Bayesian Model for 4D PET
A more general framework
Nonparametric Bayesian Poisson inverse problem framework
G ∼ G
F (·, t) =
X
P (·|x) G (dx, t)
Yi , Ti |F
iid
∼ F, for i = 1, . . . , n
(1)
G(·): G-distributed random probability measure (RPM), defined on
(X × T , σ(X) ⊗ σ(T )).
Objective: estimate the posterior distribution of G(·) from the observed
F-distributed dataset (Y, T) = {(Y1, T1), . . . , (Yn, Tn)}.
P(·|x): given probability distribution, indexed by x, defined on (Y, σ(Y)).
Emission Tomography context X ⊆ R3
, T ⊆ R+
.
Yi : index of the tube of response (TOR) and Ti : arrival time of the ith
observed event.
Radon: P (y = l|x) ∝ δ( φl , x − ul )
NPB Modelling for Space-Time Emission Tomography October 25, 2010 8 / 46

9. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
Nonparametric Bayesian Model for 4D PET
Pro & contras.
Relevant points
Does not require any adhoc discretization.
Classical dynamic PET: collection of spatial reconstructions
on separate time frames.
EM with 4D discretization (≈ 109
parameters).
Smoothness prior on continuous distribution → regularization.
Gives access to posterior uncertainty.
e.g. highest probability density (HPD) interval of activity
concentration for any region of interest (R ⊂ X).
Key point in dose reduction (few events).
Difficulties
How to elicit prior for G(·) (over M(X)) ?
How to infer on infinite dimensional objects ?
NPB Modelling for Space-Time Emission Tomography October 25, 2010 9 / 46

10. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Dirichlet Process
Ferguson (1973)
Definition
G0 be a probability measure over (X, σ(X)) and α ∈ R+
.
A Dirichlet process is the distribution of a random measure G
over (X, σ(X)) s.t., for any finite partition (B1, . . . , Br ) of X,
(G (B1) , . . . , G (Br )) ∼ Dir (αG0 (B1) , . . . , αG0 (Br ))
G0 is the mean distribution, α the concentration parameter.
We write G ∼ DP (α, G0).
Representations of Dirichlet processes
Chinese restaurant (prior over partitions).
Pólya urns (DP arises here as the De Finetti measure of the
exchangeable sequence).
Stick-breaking representation (constructive).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 10 / 46

11. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Dirichlet Process
Some properties
Properties
Sample discrete random distributions.
Expectation and variance :
∀B ⊆ X, E [G(B)] = G0 (B)
V (G(B)) =
G0 (B) (1 − G0 (B))
1 + α
Therefore, if α is large G is concentrated around G0.
Conjugacy : Consider G ∼ DP (α, G0) and X1, . . . , Xn
i.i.d.
∼ G.
G (·) |X1, . . . , Xn ∼ DP α + n,
αG0 (·) +
n
i=1 δXi
(·)
α + n
Predictive distribution: Xn+1|X1, . . . , Xn ∼
αG0(·)+
n
i=1
δXi
(·)
α+n
NPB Modelling for Space-Time Emission Tomography October 25, 2010 11 / 46

12. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .
1 P(X1)
Figure: Assignment probability for customer 1.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

13. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .θ1
Figure: Table draw for customer 1.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

14. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .
1
1+α
α
1+α
P(X2|X1)
θ1
Figure: Assignment probability for customer 2.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

15. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .
1
1+α
α
1+α
θ1
P(X2|X1)
Figure: Table draw for customer 2.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

16. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .
2
2+α
α
2+α
θ1
P(X3|X2)
Figure: Assignment probability for customer 3.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

17. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .
2
2+α
α
2+α
θ1
P(X3|X2)
θ2
Figure: Table draw for customer 3.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

18. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .
2
3+α
1
3+α
θ1
P(X4|X3)
θ2
α
3+α
Figure: Assignment probability for customer 4.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

19. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .
2
3+α
1
3+α
θ1
P(X4|X3)
θ2
α
3+α
θ3
Figure: Table draw for customer 4.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

20. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .
2
4+α
1
4+α
θ1
P(X5|X4)
θ2
1
4+α
θ3
α
4+α
Figure: Assignment probability for customer 5.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

21. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .
2
4+α
1
4+α
θ1
P(X5|X4)
θ2 θ3
α
4+α
1
4+α
Figure: Table draw for customer 5.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

22. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
A worthy allegory for partition prior construction.
. . .
3
5+α
1
5+α
θ1
P(X6|X5)
θ2 θ3
α
5+α
1
5+α
Figure: Assignment probability for customer 6.
Sequentially generating from a CRP
First customer sits at table 1 and order θ1 ∼ G0.
Customer n + 1 sits at:
Table k with probability
nk
n+α
with nk the number of
customers at table k.
A new table K + 1 with probability α
n+α
and order θK+1 ∼ G0
Xn = X1, . . . , Xn take on K ≤ n distinct values θ1, . . . , θK .
This defines a partition of {1, . . . , n} into K clusters, s.t. i
belongs to cluster k iff Xi = θk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 12 / 46

23. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Chinese Restaurant Process
Clustering behaviour (α = 30).
The CRP exhibits the clustering property of the DP.
Expected number of clusters K = O(α log n).
Rich-gets-richer effect → Reinforcement (small number of
large clusters).
E.g.: Ewens sampling formula, species sampling.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 13 / 46

24. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Blackwell-MacQueen Urn
Exchangeabiliy and de Finetti measure
Blackwell-MacQueen Urn
Xn|X1, . . . , Xn−1 ∼
αG0 (·) +
n−1
i=1 δXi (·)
α + n − 1
→ P(X1, . . . , Xn) =
n
i=1 P(Xi |X1, . . . , Xi−1)
Infinitely exchangeable sequence
For any n and permutation σ, P(X1, . . . , Xn) = P(Xσ(1), . . . , Xσ(n))
de Finetti theorem
For any exchangeable X1, X2 . . ., it exists RPM G s.t. Xn|G
iid
∼ G and
P(X1, . . . , Xn) =
M(X)
n
i=1
G(Xi )dP(G)
For exchangeable Blackwell-MacQueen urn sequences,
G ∼ DP(α, G0)
NPB Modelling for Space-Time Emission Tomography October 25, 2010 14 / 46

25. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Exchangeable Random Partitions
Kingman (1975), Pitman (1995,1996,2006).
Exchangeable Partition Probability Function
Xn exchangeable sequence from CRP. (A1, . . . , AK ) partition
of {1, . . . , n} in the order of appearance, nj = #Aj for all j.
P ∩K
j=1 (Xl = θj for all l ∈ Aj ) = pα(n1, . . . , nK |K)
and pα is a symmetric function s.t.
pα(n1, . . . , nK |K) =
αK K
j=1(nj − 1)!
[1 + α]n−1
with [x]m =
m
j=1(x + j − 1)
Ewens sampling formula: ml = #{j : nj = l} (
n
l=1 l ml = n)
pα(m1, . . . , mn) =
n!
[1 + α]n−1
n
l=1
αml
lml ml !
NPB Modelling for Space-Time Emission Tomography October 25, 2010 15 / 46

26. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Stick-Breaking Representation
Constructive definition, Sethuraman (1994)
Stick-breaking representation.
θ = (θ1, θ2, . . .)
iid
∼ G0
V = (V1, V2, . . .)
iid
∼ Beta (1, α)
p = (p1, p2, . . .), s.t. p1 = V1 and pk = Vk
k−1
i=1 (1 − Vi ).
Then,
G (·) =
∞
k=1
pk δθk
(·)
is a DP (α, G0)-distributed random probability distribution.
We say that: p ∼ GEM(α).
Almost sure truncation, Ishwaran and James (2001):
PN (·) =
N
k=1 pk δθk
(·) with VN = 1 converges a.s. to a
DP (αG0) random probability measure.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 16 / 46

27. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Stick-Breaking Representation
Example of construction
0 0.5 1
Stick-breaking
0
0.25
0.5
0.75
1
DPweights
−3 −2 −1 0 1 2 3
k = 0
Figure: Dirichlet process GEM construction (α = 3, G0 = N(0, 1)).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 17 / 46

28. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Stick-Breaking Representation
Example of construction
0 0.5 1
Stick-breaking
0
0.25
0.5
0.75
1
DPweights
−3 −2 −1 0 1 2 3
k = 1
Figure: Dirichlet process GEM construction (α = 3, G0 = N(0, 1)).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 17 / 46

29. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Stick-Breaking Representation
Example of construction
0 0.5 1
Stick-breaking
0
0.25
0.5
0.75
1
DPweights
−3 −2 −1 0 1 2 3
k = 2
Figure: Dirichlet process GEM construction (α = 3, G0 = N(0, 1)).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 17 / 46

30. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Stick-Breaking Representation
Example of construction
0 0.5 1
Stick-breaking
0
0.25
0.5
0.75
1
DPweights
−3 −2 −1 0 1 2 3
k = 3
Figure: Dirichlet process GEM construction (α = 3, G0 = N(0, 1)).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 17 / 46

31. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Stick-Breaking Representation
Example of construction
0 0.5 1
Stick-breaking
0
0.25
0.5
0.75
1
DPweights
−3 −2 −1 0 1 2 3
k = 4
Figure: Dirichlet process GEM construction (α = 3, G0 = N(0, 1)).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 17 / 46

32. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Stick-Breaking Representation
Example of construction
0 0.5 1
Stick-breaking
0
0.25
0.5
0.75
1
DPweights
−3 −2 −1 0 1 2 3
k = 5
Figure: Dirichlet process GEM construction (α = 3, G0 = N(0, 1)).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 17 / 46

33. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Stick-Breaking Representation
Example of construction
0 0.5 1
Stick-breaking
0
0.25
0.5
0.75
1
DPweights
−3 −2 −1 0 1 2 3
k = 6
Figure: Dirichlet process GEM construction (α = 3, G0 = N(0, 1)).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 17 / 46

34. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Stick-Breaking Representation
Example of construction
0 0.5 1
Stick-breaking
0
0.25
0.5
0.75
1
DPweights
−3 −2 −1 0 1 2 3
k → ∞
Figure: Dirichlet process GEM construction (α = 3, G0 = N(0, 1)).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 17 / 46

35. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Stick-Breaking Representation
Relation to CRP and exchangeable partitions
Invariance by Size-Biased Permutation (ISBP)
Let p ∼ GEM(α). Generate ˜p = (˜p1, ˜p2, . . .) as follows:
i P( ˜p1 = pk |p) = pk and for j ≥ 1,
ii P(˜pj+1 = pk |˜p1, . . . , ˜pj , p) = pk
1−˜p1−...−˜pj
1(pk = ˜p1, . . . , ˜pj )
Then, ˜p ∼ GEM(α).
The limiting relative frequencies of clusters of a CRP random
partition are GEM(α)-distributed (ISBP ↔ EPPF).
General stick-breaking models
Remark: if Vk ∼ Beta(ak , bk ), → p is not ISBP in general.
Relaxing exchangeability (Sethuraman (1994); Ishwaran and
James (2001)): clusters labels are explicitly defined
independently from any sequence sampling.
= CRP: X1 = θ1 by construction (cluster: equivalence class).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 18 / 46

36. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Posterior Distribution and Exchangeability
= different behaviours between representations
Posterior distribution for stick-breaking weights
Vk |Xn ∼ Beta(1 + nk , α +
∞
l=k+1 nl ) (nk = #{i : Xi = θk }).
→ Symmetry of active clusters is lost in G(·)|Xn !
Posterior distribution for limiting frequencies from CRP
Update Vk
i Vk |Xn ∼ Beta(nk , α +
∞
l=k+1
nl ) for k ≤ K
ii Vk |Xn ∼ Beta(1, α) for k > K
→ exchangeable posterior for G(·)|Xn:
G(·)|Xn =
K
k=1
pk δθk
(·) + pK+1G (·)
where
p ∼ Dirichlet(n1, . . . , nK , α) and G (·) ∼ DP(αG0)
Consequences on mixing properties of posterior sampling ?
NPB Modelling for Space-Time Emission Tomography October 25, 2010 19 / 46

37. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Pitman-Yor Process
A two-parameters extension of DP, Pitman and Yor (1997)
Constructive representation
θ = (θ1, θ2, . . .)
iid
∼ G0
for all k, Vk ∼ Beta (1 − d, α + k d)
p = (p1, p2, . . .), s.t. p1 = V1 and pk = Vk
k−1
i=1 (1 − Vi ).
Then,
G (·) =
∞
k=1
pk δθk
(·)
is a PY (d, α, G0)-distributed RPM where d ∈ [0, 1[ and
α > −d. We note: p ∼ GEM(d, α)
Extended CRP representation
P(Xn+1 = θk |Xn) = nk −d
α+n for k ≤ K.
P(Xn+1 = θK+1|Xn) = α+K d
α+n .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 20 / 46

38. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Pitman-Yor Process
Clustering properties
α = 30, d = 0.3
α = 30, d = 0
Properties
Expected number of clusters K = O(αnd
) (Zipf’s law).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 21 / 46

39. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Pitman-Yor Process
Posterior distribution.
Posterior distribution based on extended CRP weights
Update Vk
i Vk |Xn ∼ Beta(nk − d, α + k d +
∞
l=k+1
nl ) for k ≤ K
ii Vk |Xn ∼ Beta(1 − d, α + k d) for k > K
→ exchangeable posterior for G(·)|Xn:
G(·)|Xn =
K
k=1
pk δθk
(·) + pK+1G (·)
where
p ∼ Dirichlet(n1 − d, . . . , nK − d, α + K d)
G (·) ∼ PY(d, α + d K, G0)
GEM(d, α) is the maximal family of ISBP distributions.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 22 / 46

40. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Pitman-Yor Process Mixtures (PYM)
Continuous data modelling
Discreteness of PY(d, α, G0) generated measures
Cannot be used for probability density functions estimation !
→ Hierarchical mixture model with continuous distribution φ.
Hierarchical data model
Yi |Xi ∼ φ(Yi |Xi )
Xi ∼ G(·)
G ∼ PY(d, α, G0)
Data distribution
y|G ∼
Θ
φ(y|θ)G(dθ) =
∞
k=1
pk φ(y|θk )
E.g.: PYM of Normals with G0 taken as Normal-Inverse
Wishart (NIW), s.t. θk = (µk , Σk ).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 23 / 46

41. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Posterior Sampling of PYM
Specific random schemes
How to infer on infinite dimensional objects in a real world
(and in a decent time) ?
Sampling from the posterior: specific MCMC techniques.
Integrate out the random distribution: Escobar (1995),
Mac-Eachern (1998), Neal (2000).
side-step infiniteness by marginalization, only the allocation
to occupied clusters (finite number) is sampled (Pólya Urn
scheme).
Collapsing → good mixing properties.
Gives only access to sequences generated from the RPM.
Almost sure truncation: Ishwaran and James (2001).
Easy implementation.
Slice sampling: Walker (2007), Kalli (2009).
Conditional approach: inference retains whole distribution.
Use of auxiliary variables: only a finite pool of atoms are
involved at each iteration, without truncation.
Gives access to posterior of any functional of the RPM
(mean, variance, credible intervals, etc.).
Variational techniques: Blei (2006).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 24 / 46

42. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Posterior Sampling of PYM
Sampler profiling
Drawback of marginalization approach
Inference for the RPM posterior is based only on posterior
sampled values of Xi : ok for posterior means but cumbersome
for distribution of RPM functionals (credible intervals).
Not easy in non conjugate case (G0 vs. φ).
Computational considerations
Allocation of data to clusters when sampling from mixtures.
Huge datasets (n ≈ 107
) → allocation time turns out to
severely dominate the computation cost.
Need for a parallelizable allocation → blocking.
Which PY representation ?
Retaining whole RPM samples while maintaining
exchangeability and avoiding truncation ?
NPB Modelling for Space-Time Emission Tomography October 25, 2010 25 / 46

43. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Posterior Sampling of PYM
Proposed Gibbs sampler
Combination of:
Use of auxiliary variables.
Use of a dependent (thresholded) slicing function.
Use of exchangeable PY posterior from weighted CRP.
Joint density
Let u = u1, u2, . . . , un uniform auxiliary variables. Joint density for
any (Yi , ui ), for some positive sequence (ξk ):
f (Yi , ui |p, θ) =
∞
k=1
U (ui |0, ξk ) pk φ (Yi |θk ) (2)
where U (·|a, b) is the uniform distribution over ]a, b].
We propose a dependent (ξk ), s.t. for all k,
ξk = min (pk , ζ)
where ζ ∈ ]0, 1], independent of pk .
NPB Modelling for Space-Time Emission Tomography October 25, 2010 26 / 46

44. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Posterior Sampling of PYM
Proposed Gibbs sampler (2)
Joint density (cont.)
f (Yi , ui |p, θ) =
1 (ζ > ui )
ζ
pk >ζ
pk φ (Yi |θk ) +
pk ≤ζ
1 (pk > ui ) φ (Yi |θk )
where both sums are finite since #{j : pj > ε} < ∞ for any ε > 0.
Sampling from the posterior
Let C s.t for i ≤ n, Ci = k iff Yi = θk . Jointly sample (u, p|C):
(p1, p2, . . . , pK , rK |C) ∼ Dirichlet (n1 − d, n2 − d, . . . , nK − d, α + K d)
i ≤ n, (ui |p1, p2, . . . , pK , C) ∼ U ui |0, min pCi
, ζ , set u = min(u)
For K < k ≤ k = min k : 1 −
k
l=1
pl < u , pk ∼ GEM(d, α + k d)
Then, for k ≤ k , sample (θk |C, Y), and
(Ci |p, θ, Y, u) ∼
k
j=1
wj,i , δj and wj,i ∝ 1(pj > ui ) max (pj , ζ) φ (Yi |θj )
Re-label (p, θ) in the order of appearance of clusters in allocation.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 27 / 46

45. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Mixing Properties of Gibbs Sampler
Integrated Autocorrelation Times on galaxy data
1 Proposed Gibbs sampler
2 Efficient Slice sampler, Kalli (2009)
3 Neal (2000), algorithm 8, m = 2 (collapsed sampler).
4 Ishwaran and James (2001).
5 “Pitman posterior” with IJ truncation for G .
-0.2
0
0.2
0.4
0.6
0.8
1
0 50 100 150 200 250 300
# Clusters: autocorrelation time
Exc.+thres: 3.983 (0.928) iat = 14.322 (0.503) T = 2.56e-04
Slice: 3.980 (0.914) iat = 60.286 (2.117) T = 2.45e-04
Trunc.: 3.994 (0.927) iat = 35.785 (1.256) T = 2.96e-04
Neal8 (2): 3.987 (0.927) iat = 8.750 (0.307) T = 2.85e-04
Trunc Unlab.: 3.980 (0.925) iat = 14.175 (0.498) T = 2.87e-04
Figure: IAT for K (active clusters).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 28 / 46

46. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Mixing Properties of Gibbs Sampler
Integrated Autocorrelation Times on galaxy data
1 Proposed Gibbs sampler
2 Efficient Slice sampler, Kalli (2009)
3 Neal (2000), algorithm 8, m = 2 (collapsed sampler).
4 Ishwaran and James (2001).
5 “Pitman posterior” with IJ truncation for G .
-0.2
0
0.2
0.4
0.6
0.8
1
0 10 20 30 40 50 60 70 80
Deviance: autocorrelation time
Exc.+thres: 1561.143 (21.543) iat = 2.926 (0.053) T = 2.56e-04
Slice: 1561.135 (21.531) iat = 5.024 (0.091) T = 2.45e-04
Trunc.: 1561.146 (21.530) iat = 3.265 (0.059) T = 2.96e-04
Neal8 (2): 1561.158 (21.672) iat = 2.534 (0.046) T = 2.85e-04
Trunc Unlab.: 1561.141 (21.605) iat = 2.915 (0.053) T = 2.87e-04
Figure: IAT for deviance.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 28 / 46

47. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Mixing Properties of Gibbs Sampler
Integrated Autocorrelation Times on leptokurtic data, n = 1000
1 Proposed Gibbs sampler
2 Efficient Slice sampler, Kalli (2009)
3 Neal (2000), algorithm 8, m = 2 (collapsed sampler).
4 Ishwaran and James (2001).
5 “Pitman posterior” with IJ truncation for G .
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 50 100 150 200 250 300
# Clusters: autocorrelation time
Exc.+thres: 4.171 (2.073) iat = 144.022 (5.057) T = 7.76e-04
Slice: 4.216 (2.074) iat = 250.609 (8.799) T = 6.98e-04
Trunc.: 4.236 (2.145) iat = 170.813 (5.998) T = 1.62e-03
Neal8 (2): 4.171 (2.089) iat = 105.555 (3.706) T = 1.23e-03
Trunc Unlab.: 4.215 (2.103) iat = 145.651 (5.114) T = 1.41e-03
Figure: IAT for K (active clusters).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 28 / 46

48. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Mixing Properties of Gibbs Sampler
Integrated Autocorrelation Times on leptokurtic data, n = 1000
1 Proposed Gibbs sampler
2 Efficient Slice sampler, Kalli (2009)
3 Neal (2000), algorithm 8, m = 2 (collapsed sampler).
4 Ishwaran and James (2001).
5 “Pitman posterior” with IJ truncation for G .
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 10 20 30 40 50 60 70 80 90 100
Deviance: autocorrelation time
Exc.+thres: 2341.660 (12.175) iat = 8.743 (0.178) T = 7.76e-04
Slice: 2341.660 (12.341) iat = 13.659 (0.277) T = 6.98e-04
Trunc.: 2341.636 (12.231) iat = 9.188 (0.187) T = 1.62e-03
Neal8 (2): 2341.674 (12.309) iat = 8.025 (0.163) T = 1.23e-03
Trunc Unlab.: 2341.666 (12.197) iat = 8.578 (0.174) T = 1.41e-03
Figure: IAT for deviance.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 28 / 46

49. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Dirichlet process
Chinese restaurant
Stick-breaking
Pitman-Yor process
Mixtures of PY
Posterior sampling
Consistency of DPM
Spatial PET model
Space-time PET
model
Conclusion
Consistency Results on Dirichlet Mixtures
Frequentist validation of Bayesian estimates.
An issue in Bayesian nonparametrics: Diaconis and
Freedman, 1986
Depend on mixing RPM and kernel density.
Ghosal et al. (1999): consistency (weak, strong) of DPM of
normals (1D).
Ghosal and Van der Vaart (2007): convergence rates for
DPM of normals ≈ n− 2
5 (log n)
4
5 (twice differentiable pdf,
equivalent as kernel estimators).
Wu and Ghosal (2010): L1-consistency of DPM of
multivariate normals (with general covariance matrix).
Density deconvolution ?
Tokdar et al. (2009): consistency of (not Bayesian) recursive
estimator (Newton, 2002) in density deconvolution (in
relation with NPB).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 29 / 46

50. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
PYM of latent emission locations
Spatial hierarchical model
Yi |Xi
ind
∼ P (Yi |Xi )
Xi |Zi
ind
∼ N (Xi |Zi )
Zi |H
iid
∼ H
H ∼ PY (d, α, NIW)
(3)
Remarks
Tomography: Only Yi is observed, thus Xi (the emission location)
is introduced as latent variable.
In EM approach, latent variables are the number of emissions from
voxel v which are recorded in line of response l.
Compared to NPB density estimation, PET reconstruction mainly
involves a sampling step from conditional (Xi |Yi , p, θ).
Spatial distribution: G(·) = Θ
N(·|θ)H(dθ) =
∞
k=1
pk N(·|θk ).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 30 / 46

51. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Inference by Gibbs sampling
Sampling from the posterior
Let introduce C = C1, C2, . . . , Cn, the classification of emissions
to PY components s.t. Zi = θCi for all i < n.
Let u = u1, u2, . . . , un uniform auxiliary variables, cf. (2).
Successively draw samples from the following conditionals
Annihilation location : (X|Y, p, θ, u)
PYM component parameters : (θ|C, X)
Emission allocation to PY atoms : (C|p, θ, X, u)
PY weights & auxiliary variables : (p, u|C)
Sampling X|Y, p, θ, u: Metropolis (independent MH) within Gibbs
(Xi |Yi , p, θ, u)
∝
∼ P(Yi |Xi ) G(Xi |p, θ, u)
P(Yi |Xi ) accounts for physical and geometrical properties of PET
system → no hope for conjugacy...
Candidate: Xi |Yi , p, θ, u
∝
∼ N(Xi |µYi , ΣYi ) G(Xi |p, θ, u)
NPB Modelling for Space-Time Emission Tomography October 25, 2010 31 / 46

52. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Gibbs sampler in action
Iteration k, (p, u|C), (θ|C, X)
NPB Modelling for Space-Time Emission Tomography October 25, 2010 32 / 46

53. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Gibbs sampler in action
Event Yi
NPB Modelling for Space-Time Emission Tomography October 25, 2010 32 / 46

54. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Gibbs sampler in action
Back-projection Xi |Yi , p, u, θ
NPB Modelling for Space-Time Emission Tomography October 25, 2010 32 / 46

55. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Gibbs sampler in action
Event Yi+1
NPB Modelling for Space-Time Emission Tomography October 25, 2010 32 / 46

56. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Gibbs sampler in action
Back-projection Xi+1|Yi+1, p, u, θ
NPB Modelling for Space-Time Emission Tomography October 25, 2010 32 / 46

57. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Gibbs sampler in action
Back-projections X|Y, p, u, θ
NPB Modelling for Space-Time Emission Tomography October 25, 2010 32 / 46

58. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Gibbs sampler in action
Cluster allocations C|θ, p, u, X
NPB Modelling for Space-Time Emission Tomography October 25, 2010 32 / 46

59. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Gibbs sampler in action
Iteration k + 1, (p, u|C), (θ|C, X)
NPB Modelling for Space-Time Emission Tomography October 25, 2010 32 / 46

60. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Application
Data generation
Realistic digital 3D brain phantom.
n = 107
events.
Geometrical and physical model of system.
Truncated data in axial dimension.
Algorithm parametrization
Dirichlet case: α = 1000, d = 0
NIW: Wishart centred on isotropic 3D normal with
σ = 2.5mm and dof = 4.
→ K ≈ 4000, k ≈ 10000 during iterations at equilibrium.
Comparison with EM approach: MAP using Gibbs prior and
“log cosh” energy function (Green, 1990) with parameters
β = .25 and δ = 10.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 33 / 46

61. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Results
phant
EM
NPB
NPB Modelling for Space-Time Emission Tomography October 25, 2010 34 / 46

62. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Model
Inference
Application
Space-time PET
model
Conclusion
Spatial Model for PET Data
Results: reconstruction uncertainty
1) 2) 0
2
4
6
8
NPB95%CIActivity
−100 −50 0 50 100
x-axis (mm)
Figure: 1) NPB conditional standard deviation, 3D isosurfaces; 2) 95%
HPD on a profile: 97.5% (red), 2.5% (blue), median (green) and
phantom profile (black).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 35 / 46

63. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Space-Time Model for PET Data
Tissue kinetics: time dependency
Modelling metabolic activity
Biokinetic: tissue
dependent.
Functional volume (FV):
spatial region characterized
by a particular kinetic.
Radioactive decay.
Separable space-time activity distribution
G (x, t) =
∞
k=1
pk N (x|θk ) Qk (t)
Kinetics RPM
Each event Yi is time stamped (Ti ).
Continuous measure with compact support (right truncation).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 36 / 46

64. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Pólya Tree Process
Definition
Definition
Let E = {0, 1}, Em
= E × · · · × E and E = ∪∞
m=0Em
.
Let πm = {B : ∈ Em
} be a partition of T and Π = ∪∞
m=0πm.
A probability distribution Q on T has a Pólya tree distribution
PT(Π, A) if there are nonnegative numbers A = {α : ∈ E }
and r.v. W = {W : ∈ E } s.t.
W is a sequence of independent random variables,
for all in E , W ∼ Beta(α 0, α 1), and
for all integer m and = 1 · · · m in Em
,
Q(B 1··· m ) =
m
j=1
j =0
W 1··· j−1 ×
m
j=1
j =1
(1 − W 1··· j−1 )
Note that for ∈ E , W 0 = Q (B 0|B )
NPB Modelling for Space-Time Emission Tomography October 25, 2010 37 / 46

65. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Some Properties of Pólya Tree Processes
Lavine (1992), Mauldin and Sudderth (1992)
Properties
Pólya trees are tail free processes.
Dirichlet processes are Pólya trees s.t. α 0 = α 1 = α /2
PT(Π, A) can generate absolutely continuous distributions.
Conjugacy : posterior of PT(Π, A) after observations
X = (X1, . . . , Xn) is the Pólya tree PT(Π, AX
) with
αT
= α + n
n = # {i ∈ {1, . . . , n} : Ti ∈ B }
Predictive density (conditional mean)
Pr (Tn+1 ∈ B 1··· m |T) =
m
k=1
α 1··· k
+ n 1··· k
α 1··· k−10 + α 1··· k−11 + n 1··· k−1
NPB Modelling for Space-Time Emission Tomography October 25, 2010 38 / 46

66. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Pólya Tree Construction
Dyadic partition
0
0.25
0.5
0.75
1
Density
−3 −2 −1 0 1 2 3
m = 0
Figure: Pólya tree sequence construction (normal mean).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 39 / 46

67. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Pólya Tree Construction
Dyadic partition
0
0.25
0.5
0.75
1
Density
−3 −2 −1 0 1 2 3
m = 1
Figure: Pólya tree sequence construction (A = {αm = 3m
}).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 39 / 46

68. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Pólya Tree Construction
Dyadic partition
0
0.25
0.5
0.75
1
Density
−3 −2 −1 0 1 2 3
m = 2
Figure: Pólya tree sequence construction (A = {αm = 3m
}).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 39 / 46

69. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Pólya Tree Construction
Dyadic partition
0
0.25
0.5
0.75
1
Density
−3 −2 −1 0 1 2 3
m = 3
Figure: Pólya tree sequence construction (A = {αm = 3m
}).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 39 / 46

70. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Pólya Tree Construction
Dyadic partition
0
0.25
0.5
0.75
1
Density
−3 −2 −1 0 1 2 3
m = 4
Figure: Pólya tree sequence construction (A = {αm = 3m
}).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 39 / 46

71. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Pólya Tree Construction
Dyadic partition
0
0.25
0.5
0.75
1
Density
−3 −2 −1 0 1 2 3
m = 5
Figure: Pólya tree sequence construction (A = {αm = 3m
}).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 39 / 46

72. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Pólya Tree Construction
Dyadic partition
0
0.25
0.5
0.75
1
Density
−3 −2 −1 0 1 2 3
m = 6
Figure: Pólya tree sequence construction (A = {αm = 3m
}).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 39 / 46

73. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Pólya Tree Construction
Dyadic partition
0
0.25
0.5
0.75
1
Density
−3 −2 −1 0 1 2 3
m → ∞
Figure: Pólya tree sequence construction (A = {αm = 3m
}).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 39 / 46

74. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Pólya Tree Mixtures (PTM)
Discontinuities mitigation
Discontinuities
Continuous RPM but discontinuities at partition endpoints.
Mitigation of partitions endpoints discontinuities ?
Pólya tree mixtures
Partitions and parameters depend on an r.v. Ψ
(G|Ψ) ∼ PT ΠΨ
, AΨ
Ψ ∼ µ (Ψ)
E.g. Shifted PT.
Uniform partition Πu.
ΠΨ
random shift of Πu.
Adapt AΨ
s.t. mean distribution given Ψ remains uniform
and invariant for all Ψ.
Easy and efficient with finite PT (αm = ∞ for m > M).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 40 / 46

75. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Space-Time Model for PET Data
Dependent PYM of Pólya Trees
Space-Time hierarchical model
Yi |Xi
ind
∼ P (Yi |Xi )
Xi , Ti |Zi , Qi
ind
∼ N (Xi |Zi ) × Qi (Ti )
Zi , Qi |H
iid
∼ H
H|K0 ∼ PY (d, α, NIW × K0)
K0 ∼ PY (h, β, PT (A, Q0))
(4)
With H =
∞
k=1
pk δθk ,Qk
where Q are i.i.d. K0.
K0 =
∞
j=1
πj δQj
with π ∼ GEM(h, β), Q are i.i.d. PT(A, Q0),
a Pólya tree with parameters A and mean Q0.
K0: PY process with PT process as base distribution → nested
RPM (cf. nested DP, Rodriguez et al., 2008).
Distinct θk may share the same Qj (K0 is discrete) → partial
Hierarchical PY (Teh, 2006); (diffuse NIW × K0).
NPB Modelling for Space-Time Emission Tomography October 25, 2010 41 / 46

76. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Space-Time Model for PET Data
Inference
Additional latent variables
Allocation variable: Dk = j iff Qk = Qj (kinetics clustering).
Auxiliary variables v for slice sampling of K0.
Posterior computations
Gibbs sampling of additional conditionals is straightforward.
Functional volumes distribution
For all j (label of K0 atoms),
FVj (x) =
k: Qk =Qj
pk N (x|θk )
Nonparametrics issue: labels are permanently re-ordered →
only the FV-distribution is accessible → need for post
identification of classes.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 42 / 46

77. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Space-Time Model for PET Data
Application and results
Data generation
5 functional volumes: blood pool, gray matter, white matter,
cerebellum, tumors.
Blood fraction in tissues (between 5% and 10%).
n = 107
events (≈ 1
10 usual dose for 4D PET).
Spatial model unchanged.
Results
Point-wise PT kinetics value distribution (temporal marginal).
Space-time distribution.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 43 / 46

78. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Pólya Tree
Model
Inference
Application
Conclusion
Space-Time Model for PET Data
Functional volumes estimation
Clinical interpretation (kinetics discovery)
Construct metabolic parameter from kinetics distribution ?
Post selection of groups → coming back to parametric...
NPB Modelling for Space-Time Emission Tomography October 25, 2010 44 / 46

79. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
Conclusion and Perspectives
Some observations...
Suitable framework for 4D PET: really nonparametric
(K ≈ 4000...).
Alternative approach for Poisson inverse problems (Antoniadis
(2006)).
Flexible nonparametric data modelling (hierarchies,
dependencies, etc.).
Posterior distribution of any RPM functional is accessible
(clinical requirements).
Efficient sampling schemes.
...and perspectives
Consistency and rates results ?
Prior refinement: fragmentation/coagulation, kernel choice ?
General indirect regression problems.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 45 / 46

80. Éric Barat
Positron Emission
tomography
NPB Model for 4D
PET
Random probability
measure
Spatial PET model
Space-time PET
model
Conclusion
For Further Reading.
T. Ferguson.
Ann. Statist. 1, 209–230 (1973).
C. Antoniak.
Ann. Statist. 2, 1152–1174 (1974).
H. Ishwaran and L. F. James.
J. Am. Stat. Assoc. 96, 161–173 (2001).
P. Müller and F. A. Quintana.
Statist. Sci. 19, 95–110 (2004).
Y. W. Teh et al..
J. Am. Stat. Assoc. 101, 1566–1581 (2006).
J. Pitman.
Combinatorial Stochastic Processes, Springer, 2006.
N. Hjort et al..
Bayesian Nonparametrics, Cambridge, 2010.
NPB Modelling for Space-Time Emission Tomography October 25, 2010 46 / 46