SlideShare a Scribd company logo
1 of 54
Download to read offline
Component-wise approximate Bayesian
computation via Gibbs-like steps
Christian P. Robert(1,2)
arXiv:1905.13599 with Grégoire Clarté(1)
, Robin Ryder(1)
, Julien Stoehr(1)
(1) Université Paris-Dauphine, (2) University of Warwick
Université Paris-Dauphine
February 28, 2020
Approximate Bayesian computation (ABC)
ABC is a computational method which stemmed from population ge-
netics models about 20 years ago to deal with generative intractable
distribution.
[Tavaré et al., 1997; Beaumont et al., 2002]
Settings of interest: the likelihood function f(x | θ) does not admit a
closed form as a function of θ and/or is computationally too costly.
1. Model relying on a latent process z ∈ Z
f(x | θ) =
Z
f(y, z | θ)µ(dz).
2. Model with intractable normalising constant
f(x | θ) =
1
Z(θ)
q(x | θ), where Z(θ) =
X
q(x | θ)µ(dx).
Approximate Bayesian computation (ABC)
Bayesian settings: the target is π(θ | xobs
) ∝ π(θ)f(xobs
| θ).
Algorithm: Vanilla ABC
Input: observed dataset xobs
,
number of iterations N,
threshold ε, summary
statistic s.
for i = 1, . . . , N do
θi ∼ π(·)
xi ∼ f(· | θi)
end
return θi d(s(xobs
), s(xi)) ≤ ε
s(xobs)
ε
(θi, S(xi))
Approximate Bayesian computation (ABC)
Bayesian settings: the target is π(θ | xobs
) ∝ π(θ)f(xobs
| θ).
Algorithm: Vanilla ABC
Input: observed dataset xobs
,
number of iterations N,
threshold ε, summary
statistic s.
for i = 1, . . . , N do
θi ∼ π(·)
xi ∼ f(· | θi)
end
return θi d(s(xobs
), s(xi)) ≤ ε
s(xobs)
ε
(θi, S(xi))
Ouput: distributed according to πε(θ | s, xobs
)
π(θ)Pθ d(S(xobs
), S(x)) < ε ∝ π(θ | d(S(xobs
), S(x)) < ε)
Approximate Bayesian computation (ABC)
Two limiting situations:
π∞(θ | s, xobs
) ∝ π(θ) and π0(θ | s, xobs
) ∝ π(θ | s(xobs
))= π(θ | xobs
)
Approximate Bayesian computation (ABC)
Two limiting situations:
π∞(θ | s, xobs
) ∝ π(θ) and π0(θ | s, xobs
) ∝ π(θ | s(xobs
))= π(θ | xobs
)
Some difficulties raised by the vanilla version:
Calibration of the threshold ε: by regression or k-nearest neigh-
bour postprocessing
[Beaumont et al., 2002; Wilkinson, 2013; Biau et al., 2013]
Selection of the summary statistic s: advances with semi-automatic
procedures like pilot-run ABC or random forests
[Fearnhead and Prangle, 2012; Prangle et al., 2014; Raynal et al.,
2018]
Simulating from the prior is often poor in efficiency: versions
modifying proposal distribution on θ to increase density of x’s near
y (MCMC, SMC, PMC)
[Marjoram et al., 2003; Toni et al., 2008]
A first example : hierarchical moving average model
α
µ1 µ2 µn. . .
x1 x2 xn. . .
σ
σ1 σ2 σn. . .
First parameter hierarchy:
α = (α1, α2, α3) ∼ E(1)⊗3
independently for each i ∈ {1, . . . , n},
(βi,1, βi,2, βi,3) ∼ Dir(α1, α2, α3)
µi = (βi,1 − βi,2, 2(βi,1 + βi,2) − 1)
Second parameter hierarchy:
σ = (σ1, σ2) ∼ C+
(1)⊗2
.
independently for each i ∈ {1, . . . , n},
σi ∼ IG(σ1, σ2)
Model for xi: independently for each i ∈ {1, . . . , n}, xi ∼ MA2(µi, σi),
i.e., for all j in N
xi,j = yj + µi,1yj−1 + µi,2yj−2 , with yj ∼ N(0, σ2
i )
A first example : toy dataset
Settings: n = 5 times series of length T = 100 hierarchical model
with 13 parameters.
Figure: Histogram of ABC sam-
ple for µ1,1 along prior distribution
(black line)
Size of ABC reference table:
N = 5.5 · 106
.
ABC posterior sample size:
1000.
A first example : toy dataset
Settings: n = 5 times series of length T = 100 hierarchical model
with 13 parameters.
Figure: Histogram of ABC sam-
ple for µ1,1 along prior distribution
(black line)
Size of ABC reference table:
N = 5.5 · 106
.
ABC posterior sample size:
1000.
Not enough simulations to reach a decent threshold
Not enough time to produce enough simulations
The Gibbs Sampler
Our idea: combining ABC with Gibbs sampler in order to improve
ability to (more) efficiently explore Θ ⊂ Rn
when the number n of
parameters increases.
The Gibbs Sampler
Our idea: combining ABC with Gibbs sampler in order to improve
ability to (more) efficiently explore Θ ⊂ Rn
when the number n of
parameters increases.
The Gibbs Sampler produces a Markov chain with a target joint dis-
tribution π by alternatively sampling from each of its conditionals.
[Geman and Geman, 1984]
Algorithm: Gibbs sampler
Input: observed dataset xobs
, number of iterations N, starting point
θ(0)
= (θ
(0)
1 , . . . , θ
(0)
n ).
for i = 1, . . . , N do
for k = 1, . . . , n do
θ
(i)
k ∼ π · | θ
(i)
1 , . . . , θ
(i)
k−1, θ
(i−1)
k+1 , . . . , θ
(i−1)
n , xobs
end
end
return θ(0)
, . . . , θ(N)
Component-wise ABC [aka ABCG]
ABC Gibbs Sampler produces a Markov chain with each proposal an
ABC posterior conditional on different summaries sj and tolerance εj
Algorithm: Component-wise ABC
Input: observed dataset xobs
, number of iterations N, starting point
θ(0)
= (θ
(0)
1 , . . . , θ
(0)
n ), threshold ε = (ε1, . . . , εn), statistics
s1, . . . , sn.
for i = 1, . . . , N do
for j = 1, . . . , n do
θ
(i)
j ∼ πεj
(· | xobs
, sj, θ
(i)
1 , . . . , θ
(i)
j−1, θ
(i−1)
j+1 , . . . , θ
(i−1)
n )
end
end
return θ(0)
, . . . , θ(N)
Component-wise ABC [aka ABCG]
Algorithm: Component-wise ABC
Input: observed dataset xobs
, number of iterations N, starting point
θ(0)
= (θ
(0)
1 , . . . , θ
(0)
n ), threshold ε = (ε1, . . . , εn), statistics
s1, . . . , sn.
for i = 1, . . . , N do
for j = 1, . . . , n do
θ
(i)
j ∼ πεj
(· | xobs
, sj, θ
(i)
1 , . . . , θ
(i)
j−1, θ
(i−1)
j+1 , . . . , θ
(i−1)
n )
end
end
return θ(0)
, . . . , θ(N)
Questions:
Is there a limiting distribution ν∞
ε to the algorithm?
What is the nature of this limiting distribution?
Gibbs with ABC approximations
Souza Rodrigues et al. (arxiv:1906.04347) alternative ABC-ed Gibbs:
Further references to earlier occurrences of Gibbs versions of ABC
ABC version of Gibbs sampling with approximations to the con-
ditionals with no further corrections
Related to regression post-processing à la Beaumont et al. (2002)
used to designing approximate full conditional, possibly involv-
ing neural networks
Requires preliminary ABC step
Drawing from approximate full conditionals done exactly, possi-
bly via a bootstrapped version.
ABC within Gibbs: Hierarchical models
α
µ1 µ2 µn. . .
x1 x2 xn. . .
Hierarchical Bayes models: often allow for
simplified conditional distributions thanks to
partial independence properties, e.g.,
xj | µj ∼ π(xj | µj), µj | α
i.i.d.
∼ π(µj | α), α ∼ π(α).
Algorithm: Component-wise ABC sampler for hierarchical model
Input: observed dataset xobs
, number of iterations N, thresholds εα
and εµ, summary statistics sα and sµ.
for i = 1, . . . , N do
for j = 1, . . . , n do
µ
(i)
j ∼ πεµ
(· | xobs
j , sµ, α(i−1)
)
end
α(i)
∼ πεα
(· | µ(i)
, sα)
end
ABC within Gibbs: Hierarchical models
Assumption: n = 1.
Theorem (Clarté et al. [2019])
Assume there exists a non-empty convex set C with positive prior measure
such that
κ1 = inf
sα(µ)∈C
π(Bsα(µ), α/4) > 0 ,
κ2 = inf
α
inf
sα(µ)∈C
πεµ (Bsα(µ),3 α/2 | xobs
, sµ, α) > 0 ,
κ3 = inf
α
πεµ (sα(µ) ∈ C | xobs
, sµ, α) > 0 ,
Then the Markov chain converges geometrically in total variation distance to
a stationary distribution ν∞
ε , with geometric rate 1 − κ1κ2κ2
3.
If the prior on α is defined on a compact set, then the assumptions
are satisfied.
ABC within Gibbs: Hierarchical models
Theorem (Clarté et al. [2019])
Assume that,
L0 = sup
εα
sup
µ, ˜µ
πεα
(· | sα, µ) − π0(· | sα, ˜µ) TV < 1/2 ,
L1(εα) = sup
µ
πεα
(· | sα, µ) − π0(· | sα, µ) TV −−−−→
εα→0
0
L2(εµ) = sup
α
πεµ (· | xobs
, sµ, α) − π0(· | xobs
, sµ, α) TV −−−−→
εµ→0
0 .
Then,
ν∞
ε − ν∞
0 TV ≤
L1(εα) + L2(εµ)
1 − 2L0
−−−→
ε→0
0.
ABC within Gibbs: Hierarchical models
Compatibility issue: ν∞
0 is the limiting distribution associated to
Gibbs conditionals with different acceptance events, e.g., different statis-
tics:
π(α)π(sα(µ) | α) and π(µ)f(sµ(xobs
) | α, µ)
Conditionals may then be incompatible and limiting distribution
not genuine posterior [incoherent use of data]
unknown [except for a specific version]
possibly far from a genuine posterior
Proposition (Clarté et al. [2019])
If sα is jointly sufficient, when the precision ε goes to zero, ABC within
Gibbs and ABC have the same limiting distribution.
Hierarchical models: toy example
Model:
α ∼ U([0 ; 20]),
(µ1, . . . , µn) | α ∼ N(α, 1)⊗n
,
(xi,1, . . . , xi,K) | µi ∼ N (µi, 0.1)
⊗K
.
Numerical experiment:
n = 20, K = 10,
Pseudo observation generated for α = 1.7,
Algorithms runs for a constant budget: Ntot = N × Nε = 21000.
We look at the estimates for µ1 whose value for the pseudo obser-
vations is 3.04.
Hierarchical models: toy example
Illustration:
Assumptions of convergence theorem hold
Tolerance as quantile of distances at each call, i..e, selection of sim-
ulation with smallest distance out of Nα = Nµ = 30
Summary statistic as empirical mean (sufficient):
Setting when SMC-ABC fails as well
Hierarchical models: toy example
Figure: comparison of the sampled densities of µ1 (left) and α (right)
[dot-dashed line as true posterior]
0
1
2
3
4
0 2 4 6
0.0
0.5
1.0
1.5
2.0
−4 −2 0 2 4
Method ABC Gibbs Simple ABC
Hierarchical models: toy example
Figure: posterior densities of µ1 and α for 100 replicas of 3 ABC al-
gorithms [true marginal posterior as dashed line]
ABC−GibbsSimpleABCSMC−ABC
0 2 4 6
0
1
2
3
4
0
1
2
3
4
0
10
20
30
40
mu1
density
ABC−GibbsSimpleABCSMC−ABC
0 1 2 3 4 5
0.0
0.5
1.0
1.5
2.0
2.5
0.0
0.5
1.0
1.5
2.0
0
5
10
15
20
hyperparameter
density
Hierarchical models: moving average example
Pseudo observations: xobs
1 generated for µ1 = (−0.06, −0.22).
0
1
2
3
−1.0 −0.5 0.0 0.5 1.0
value
density
type
ABCGibbs
ABCsimple
prior
1st parameter, 1st coordinate
−1.0
−0.5
0.0
0.5
1.0
−1.0 −0.5 0.0 0.5 1.0
b1
b2
0.2
0.4
0.6
0.8
level
1st parameter simple
−1.0
−0.5
0.0
0.5
1.0
−1.0 −0.5 0.0 0.5 1.0
b1
b2
2.5
5.0
7.5
10.0
level
1st parameter gibbs
Separation from the prior for identical number of simulations.
Hierarchical models: moving average example
Real dataset: measures of 8GHz daily flux intensity emitted by 7
stellar objects from the NRL GBI website: http://ese.nrl.navy.
mil/.
[Lazio et al., 2008]
0
1
2
3
−1.0 −0.5 0.0 0.5 1.0
value
density
type
ABCGibbs
ABCsimple
prior
1st parameter, 1st coordinate
−1.0
−0.5
0.0
0.5
1.0
−1.0 −0.5 0.0 0.5 1.0
b1
b2
0.2
0.4
0.6
level
1st parameter simple
−1.0
−0.5
0.0
0.5
1.0
−1.0 −0.5 0.0 0.5 1.0
b1
b2
2
4
6
8
level
1st parameter gibbs
Separation from the prior for identical number of simulations.
Hierarchical models: g&k example
Model: g-and-k distribution (due to Tukey) defined through inverse
cdf: easy to simulate but with no closed-form pdf:
A + B 1 + 0.8
1 − exp(−gΦ−1
(r)
1 + exp(−gΦ−1(r)
1 + Φ−1
(r)2 k
Φ−1
(r)
Note:: MCMC feasible in this setting (Pierre Jacob)
α
A1
A2
...
An
x1
x2
xn
...
B
g
k
Hierarchical models: g&k example
Assumption: B, g and k known, inference on α and Ai solely.
1 2 3 4 Hyperparameter
−7.5 −7.0 −6.5 −6.0 −5.5 −5.0−7.5 −7.0 −6.5 −6.0 −5.5 −5.0−7.5 −7.0 −6.5 −6.0 −5.5 −5.0−7.5 −7.0 −6.5 −6.0 −5.5 −5.0−7.5 −7.0 −6.5 −6.0 −5.5 −5.0
0
2
4
6
8
value
density
Method ABC Gibbs ABC−SMC vanilla ABC
ABC within Gibbs: general case
A general two-parameter model:
(θ1, θ2)
x
Algorithm: ABC within Gibbs
for i = 1, . . . , N do
θ
(i)
2 ∼ πε2
(· | θ
(i−1)
1 , s2, xobs
)
θ
(i)
1 ∼ πε1
(· | θ
(i)
2 , s1, xobs
)
end
return (θ
(i)
1 , θ
(i)
2 )i=2,...,N
ABC within Gibbs: general case
A general two-parameter model:
(θ1, θ2)
x
Algorithm: ABC within Gibbs
for i = 1, . . . , N do
θ
(i)
2 ∼ πε2
(· | θ
(i−1)
1 , s2, xobs
)
θ
(i)
1 ∼ πε1
(· | θ
(i)
2 , s1, xobs
)
end
return (θ
(i)
1 , θ
(i)
2 )i=2,...,N
Theorem (Clarté et al. [2019])
Assume that there exists 0 < κ < 1/2 such that
sup
θ1, ˜θ1
πε2
(· | xobs
, s2, θ1) − πε2
(· | xobs
, s2, ˜θ1) TV = κ.
The Markov chain then converges geometrically in total variation distance
to a stationary distribution ν∞
ε , with geometric rate 1 − 2κ.
ABC within Gibbs: general case
Additional assumption: θ1 and θ2 are a priori independent
Theorem (Clarté et al. [2019])
Assume that
κ1 = inf
θ1,θ2
π(Bs1(xobs),ε1
| θ1, θ2) > 0 ,
κ2 = inf
θ1,θ2
π(Bs2(xobs), 2
| θ1, θ2) > 0 ,
κ3 = sup
θ1, ˜θ1,θ2
π(· | θ1, θ2) − π(· | ˜θ1, θ2) TV < 1/2 .
Then the Markov chain converges in total variation distance to a stationary
distribution ν∞
ε with geometric rate 1 − κ1κ2(1 − 2κ3).
ABC within Gibbs: general case
For both situations, a limiting distribution exists when the thresholds
go to 0.
Theorem (Clarté et al. [2019])
Assume that
L0 = sup
ε2
sup
θ1, ˜θ1
πε2
(· | xobs
, s2, θ1) − π0(· | xobs
, s2, ˜θ1) TV < 1/2 ,
L1(ε1) = sup
θ2
πε1
(· | xobs
, s1, θ2) − π0(· | xobs
, s1, θ2) TV −−−−→
ε1→0
0 ,
L2(ε2) = sup
θ1
πε2
(· | xobs
, s2, θ1) − π0(· | xobs
, s2, θ1) TV −−−−→
ε2→0
0 .
Then
ν∞
ε − ν∞
0 TV ≤
L1(ε1) + L2(ε2)
1 − 2L0
−−−→
ε→0
0.
ABC within Gibbs: general case
Compatibility issue: the general case inherits the compatibility issue
already noticed in the hierarchical setting.
Lemma (Clarté et al. [2019])
1. If sθ1
and sθ2
are conditionally sufficient, the conditionals are compatible
and , when the precision goes to zero, ABC within Gibbs and ABC have
the same limiting distribution.
2. If π(θ1, θ2) = π(θ1)π(θ2) and sθ1
= sθ2
, when the precision goes to
zero, ABC within Gibbs and ABC have the same limiting distribution.
General case: g&k example
Figure: posterior densities for parameters µ1, . . . , µ4 in the doubly
hierarchical g & k model
µ1 µ2 µ3 µ4
−4 −3 −2 −1 0 −4 −3 −2 −1 0 −4 −3 −2 −1 0 −4 −3 −2 −1 0
0
1
2
3
4
5
Method ABC Gibbs ABC−SMC vanilla ABC
General case: g&k example
Figure: posterior densities for α, B, g and k
α B g k
−3 −2 −1 0 0.00 0.25 0.50 0.75 1.000.00 0.25 0.50 0.75 1.000.00 0.25 0.50 0.75 1.00
0.0
2.5
5.0
7.5
10.0
Method ABC Gibbs ABC−SMC vanilla ABC
General case: inverse problem
Example inspired by deterministic inverse problems:
difficult to tackle with traditional methods
pseudo-likelihood extremely expensive to compute
requiring the use of surrogate models
Let y be solution of heat equation on a circle defined for (τ, z) ∈]0, T]×
[0, 1[ by
∂τy(z, τ) = ∂z (θ(z)∂zy(z, τ)) ,
with
θ(z) =
n
j=1
θj1[(j−1)/n,j/n](z)
and boundary conditions y(z, 0) = y0(z) and y(0, τ) = y(1, τ)
[Kaipio and Fox, 2011]
General case: inverse problem
Assume y0 known and parameter θ = (θ1, . . . , θn), with discretized
equation via first order finite elements of size 1/n for z and ∆ for τ.
Stepwise approximation of solution
^y(z, t) =
n
j=1
yj,tφj(z)
where, for j < n,
φj(z) = (1 − |nz − j|)1|nz−j|<1
and
φn(z) = (1 − nz)10<z<1/n + (nz − n + 1)11−1/n<z<1
and with yj,t defined by
yj,t+1 − yj,t
3∆
+
yj+1,t+1 − yj+1,t
6∆
+
yj−1,t+1 − yj−1,t
6∆
= yj,t+1(θj+1 + θj) − yj−1,t+1θj − yj+1,t+1θj+1.
Noisy version of this process, chosen as xj,t = N(^yj,t, σ2
).
General case: inverse problem
In ABC-Gibbs, each parameter θm updated with summary statistics
observations at locations m−2, m−1, m, m+1 while ABC-SMC relies
on the whole data as statistic. In experiment, n = 20 and ∆ = 0.1, with
a prior θj ∼ U[0, 1], independently. Convergence theorem applies to
this setting.
Comparison with ABC, using same simulation budget, keeping the
total number of simulations constant at N · N = 8 · 106
.
Choice of ABC reference table size critical: for a fixed computational
budget reach balance between quality of the approximations of the
conditionals (improved by increasing N ), and Monte-Carlo error and
convergence of the algorithm, (improved by increasing N).
General case: inverse problem
Figure: mean and variance of the ABC and ABC-Gibbs estimators of
θ1 as N increases [horizontal line shows true value]
0.5
0.6
0.7
0.8
0 10 20 30 40
0.000
0.002
0.004
0 10 20 30 40
Method ABC Gibbs Simple ABC
General case: inverse problem
Figure: histograms of ABC-Gibbs and ABC outputs compared with
uniform prior
ABC Gibbs Simple ABC
0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00
0
1
2
3
Explicit limiting distribution
For hierarchical model
xj | µj ∼ π(xj | µj) , µj | α
i.i.d.
∼ π(µj | α) , α ∼ π(α)
alternative ABC based on:
˜π(α, µ | xobs
) ∝ π(α)q(µ)
generate a new µ
π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα
d˜µ
× f(˜x | µ)π(xobs
| µ)
with q arbitrary distribution on µ
Explicit limiting distribution
For hierarchical model
xj | µj ∼ π(xj | µj) , µj | α
i.i.d.
∼ π(µj | α) , α ∼ π(α)
induces full conditionals
˜π(α | µ) ∝ π(α) π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα
d˜x
and
˜π(µ | α, xobs
) ∝ q(µ) π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα
d˜µ
× f(˜x | µ)π(xobs
| µ)1d(sµ(xobs),sµ( ˜x))<εµ
d˜x
now compatible with new artificial joint
Explicit limiting distribution
For hierarchical model
xj | µj ∼ π(xj | µj) , µj | α
i.i.d.
∼ π(µj | α) , α ∼ π(α)
that is,
prior simulations of α ∼ π(α) and of ˜µ ∼ π(˜µ | α) until
d(sα(µ), sα(˜µ)) < εα
simulation of µ from instrumental q(µ) and of auxiliary variables
˜µ and ˜x until both constraints satisfied
Explicit limiting distribution
For hierarchical model
xj | µj ∼ π(xj | µj) , µj | α
i.i.d.
∼ π(µj | α) , α ∼ π(α)
Resulting Gibbs sampler stationary for posterior proportional to
π(α, µ) q(sα(µ))
projection
f(sµ(xobs
) | µ)
projection
that is, for likelihood associated with sµ(xobs
) and prior distribution
proportional to π(α, µ)q(sα(µ)) [exact!]
Explicit limiting distribution
Figure: histograms of ABC-Gibbs and ABC outputs compared with
uniform prior
ABC Gibbs Simple ABC
0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00
0
1
2
3
Explicit limiting distribution
For hierarchical model
xj | µj ∼ π(xj | µj) , µj | α
i.i.d.
∼ π(µj | α) , α ∼ π(α)
alternative ABC based on:
˜π(α, µ | xobs
) ∝ π(α)q(µ)
generate a new µ
π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα
d˜µ
× f(˜x | µ)π(xobs
| µ)
with q arbitrary distribution on µ
Explicit limiting distribution
For hierarchical model
xj | µj ∼ π(xj | µj) , µj | α
i.i.d.
∼ π(µj | α) , α ∼ π(α)
induces full conditionals
˜π(α | µ) ∝ π(α) π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα
d˜x
and
˜π(µ | α, xobs
) ∝ q(µ) π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα
d˜µ
× f(˜x | µ)π(xobs
| µ)1d(sµ(xobs),sµ( ˜x))<εµ
d˜x
now compatible with new artificial joint
Explicit limiting distribution
For hierarchical model
xj | µj ∼ π(xj | µj) , µj | α
i.i.d.
∼ π(µj | α) , α ∼ π(α)
that is,
prior simulations of α ∼ π(α) and of ˜µ ∼ π(˜µ | α) until
d(sα(µ), sα(˜µ)) < εα
simulation of µ from instrumental q(µ) and of auxiliary variables
˜µ and ˜x until both constraints satisfied
Explicit limiting distribution
For hierarchical model
xj | µj ∼ π(xj | µj) , µj | α
i.i.d.
∼ π(µj | α) , α ∼ π(α)
Resulting Gibbs sampler stationary for posterior proportional to
π(α, µ) q(sα(µ))
projection
f(sµ(xobs
) | µ)
projection
that is, for likelihood associated with sµ(xobs
) and prior distribution
proportional to π(α, µ)q(sα(µ)) [exact!]
Take home messages
Under certain conditions to specify,
Take home messages
We provide theoretical guarantee on the convergence of ABC within
Gibbs.
• Result n°1: a limiting distribution ν∞
ε exists when the sample
size grows
• Result n°2: a limiting distribution ν∞
0 exists when the thresh-
old goes to 0
• Result n°3: ν∞
0 is the posterior distribution π(θ | s(xobs
)).
The method inherits issues from vanilla ABC, namely the
choice of the statistics [plus compatibility of the condition-
als].
In practice, ABC within Gibbs exhibits better performances than
vanilla ABC and SMC-ABC [even when conditions not satisfied]
ABC postdoc positions
2 post-doc positions open with the ABSint ANR research grant:
Focus on approximate Bayesian techniques like ABC, variational
Bayes, PAC-Bayes, Bayesian non-parametrics, scalable MCMC,
and related topics. A potential direction of research would be the
derivation of new Bayesian tools for model checking in such c
omplex environments.
Terms: up to 24 months, no teaching duty attached, primarily
located in Université Paris-Dauphine, with supported periods in
Oxford (J. Rousseau) [barring no-deal Brexit!] and visits to Mont-
pellier (J.-M. Marin).
No hard deadline.
If interested, send application to me: bayesianstatistics@gmail.com
ABC workshops
ABC in Grenoble, France, March 18-19 2020 [mirrored in War-
wick]
ABC in Longyearbyen, Svalbard/Spitzberg, April 12-13 2021
Bibliography I
M. A. Beaumont, W. Zhang, and D. J. Balding. Approximate Bayesian
Computation in Population Genetics. Genetics, 162(4):2025–2035,
2002.
G. Biau, F. Cérou, and A. Guyader. New insights into Approximate
Bayesian Computation. Annales de l’Institut Henri Poincaré (B) Prob-
abilités et Statistiques, in press, 2013.
G. Clarté, C. P. Robert, R. Ryder, and J. Stoehr. Component-wise ap-
proximate Bayesian computation via Gibbs-like steps. arXiv preprint
arXiv:1905.13599, 2019.
P. Fearnhead and D. Prangle. Constructing summary statistics for
approximate Bayesian computation: semi-automatic approximate
Bayesian computation. Journal of the Royal Statistical Society. Series
B (Statistical Methodology), 74(3):419–474, 2012.
S. Geman and D. Geman. Stochastic Relaxation, Gibbs Distributions,
and the Bayesian Restoration of Images. IEEE Transactions on Pattern
Analysis and Machine Intelligence, 6(6):721–741, 1984.
Bibliography II
J. P. Kaipio and C. Fox. The Bayesian Framework for Inverse Problems
in Heat Transfer. Heat Transfer Engineering, 32(9):718–753, 2011.
T. J. W. Lazio, E. B. Waltman, F. D. Ghigo, R. Fiedler, R. S. Foster, and
a. K. J. Johnston. A Dual-Frequency, Multiyear Monitoring Program
of Compact Radio Sources. The Astrophysical Journal Supplement Se-
ries, 136:265, December 2008. doi: 10.1086/322531.
P. Marjoram, J. Molitor, V. Plagnol, and S. Tavaré. Markov chain Monte
Carlo without likelihoods. Proceedings of the National Academy of Sci-
ences, 100(26):15324–15328, 2003.
D. Prangle, P. Fearnhead, M. P. Cox, P. J. Biggs, and N. P. French. Semi-
automatic selection of summary statistics for ABC model choice.
Statistical applications in genetics and molecular biology, 13(1):67–82,
2014.
L. Raynal, J.-M. Marin, P. Pudlo, M. Ribatet, C. P. Robert, and A. Es-
toup. ABC random forests for Bayesian parameter inference. Bioin-
formatics, 2018. doi: 10.1093/bioinformatics/bty867.
Bibliography III
S. Tavaré, D. J. Balding, R. C. Griffiths, and P. Donnelly. Inferring Coa-
lescence Times From DNA Sequence Data. Genetics, 145(2):505–518,
1997.
T. Toni, D. Welch, N. Strelkowa, A. Ipsen, and M. P. H. Stumpf. Ap-
proximate Bayesian computation scheme for parameter inference
and model selection in dynamical systems. Journal of the Royal Soci-
ety Interface, 6(31):187–202, 2008.
R. D. Wilkinson. Approximate Bayesian computation (ABC) gives ex-
act results under the assumption of model error. Statistical Applica-
tions in Genetics and Molecular Biology, 12(2):129–141, 2013.

More Related Content

What's hot

Multiple estimators for Monte Carlo approximations
Multiple estimators for Monte Carlo approximationsMultiple estimators for Monte Carlo approximations
Multiple estimators for Monte Carlo approximationsChristian Robert
 
Convergence of ABC methods
Convergence of ABC methodsConvergence of ABC methods
Convergence of ABC methodsChristian Robert
 
ABC based on Wasserstein distances
ABC based on Wasserstein distancesABC based on Wasserstein distances
ABC based on Wasserstein distancesChristian Robert
 
ABC convergence under well- and mis-specified models
ABC convergence under well- and mis-specified modelsABC convergence under well- and mis-specified models
ABC convergence under well- and mis-specified modelsChristian Robert
 
ABC short course: survey chapter
ABC short course: survey chapterABC short course: survey chapter
ABC short course: survey chapterChristian Robert
 
Coordinate sampler : A non-reversible Gibbs-like sampler
Coordinate sampler : A non-reversible Gibbs-like samplerCoordinate sampler : A non-reversible Gibbs-like sampler
Coordinate sampler : A non-reversible Gibbs-like samplerChristian Robert
 
Approximate Bayesian model choice via random forests
Approximate Bayesian model choice via random forestsApproximate Bayesian model choice via random forests
Approximate Bayesian model choice via random forestsChristian Robert
 
NCE, GANs & VAEs (and maybe BAC)
NCE, GANs & VAEs (and maybe BAC)NCE, GANs & VAEs (and maybe BAC)
NCE, GANs & VAEs (and maybe BAC)Christian Robert
 
Likelihood-free Design: a discussion
Likelihood-free Design: a discussionLikelihood-free Design: a discussion
Likelihood-free Design: a discussionChristian Robert
 
ABC short course: final chapters
ABC short course: final chaptersABC short course: final chapters
ABC short course: final chaptersChristian Robert
 
random forests for ABC model choice and parameter estimation
random forests for ABC model choice and parameter estimationrandom forests for ABC model choice and parameter estimation
random forests for ABC model choice and parameter estimationChristian Robert
 
ABC short course: model choice chapter
ABC short course: model choice chapterABC short course: model choice chapter
ABC short course: model choice chapterChristian Robert
 
accurate ABC Oliver Ratmann
accurate ABC Oliver Ratmannaccurate ABC Oliver Ratmann
accurate ABC Oliver Ratmannolli0601
 
An overview of Bayesian testing
An overview of Bayesian testingAn overview of Bayesian testing
An overview of Bayesian testingChristian Robert
 
Bayesian inference on mixtures
Bayesian inference on mixturesBayesian inference on mixtures
Bayesian inference on mixturesChristian Robert
 
Poster for Bayesian Statistics in the Big Data Era conference
Poster for Bayesian Statistics in the Big Data Era conferencePoster for Bayesian Statistics in the Big Data Era conference
Poster for Bayesian Statistics in the Big Data Era conferenceChristian Robert
 
better together? statistical learning in models made of modules
better together? statistical learning in models made of modulesbetter together? statistical learning in models made of modules
better together? statistical learning in models made of modulesChristian Robert
 

What's hot (20)

Multiple estimators for Monte Carlo approximations
Multiple estimators for Monte Carlo approximationsMultiple estimators for Monte Carlo approximations
Multiple estimators for Monte Carlo approximations
 
Convergence of ABC methods
Convergence of ABC methodsConvergence of ABC methods
Convergence of ABC methods
 
ABC based on Wasserstein distances
ABC based on Wasserstein distancesABC based on Wasserstein distances
ABC based on Wasserstein distances
 
ABC workshop: 17w5025
ABC workshop: 17w5025ABC workshop: 17w5025
ABC workshop: 17w5025
 
ABC convergence under well- and mis-specified models
ABC convergence under well- and mis-specified modelsABC convergence under well- and mis-specified models
ABC convergence under well- and mis-specified models
 
ABC short course: survey chapter
ABC short course: survey chapterABC short course: survey chapter
ABC short course: survey chapter
 
Coordinate sampler : A non-reversible Gibbs-like sampler
Coordinate sampler : A non-reversible Gibbs-like samplerCoordinate sampler : A non-reversible Gibbs-like sampler
Coordinate sampler : A non-reversible Gibbs-like sampler
 
ABC-Gibbs
ABC-GibbsABC-Gibbs
ABC-Gibbs
 
Approximate Bayesian model choice via random forests
Approximate Bayesian model choice via random forestsApproximate Bayesian model choice via random forests
Approximate Bayesian model choice via random forests
 
NCE, GANs & VAEs (and maybe BAC)
NCE, GANs & VAEs (and maybe BAC)NCE, GANs & VAEs (and maybe BAC)
NCE, GANs & VAEs (and maybe BAC)
 
Likelihood-free Design: a discussion
Likelihood-free Design: a discussionLikelihood-free Design: a discussion
Likelihood-free Design: a discussion
 
ABC short course: final chapters
ABC short course: final chaptersABC short course: final chapters
ABC short course: final chapters
 
random forests for ABC model choice and parameter estimation
random forests for ABC model choice and parameter estimationrandom forests for ABC model choice and parameter estimation
random forests for ABC model choice and parameter estimation
 
ABC short course: model choice chapter
ABC short course: model choice chapterABC short course: model choice chapter
ABC short course: model choice chapter
 
accurate ABC Oliver Ratmann
accurate ABC Oliver Ratmannaccurate ABC Oliver Ratmann
accurate ABC Oliver Ratmann
 
An overview of Bayesian testing
An overview of Bayesian testingAn overview of Bayesian testing
An overview of Bayesian testing
 
Bayesian inference on mixtures
Bayesian inference on mixturesBayesian inference on mixtures
Bayesian inference on mixtures
 
Poster for Bayesian Statistics in the Big Data Era conference
Poster for Bayesian Statistics in the Big Data Era conferencePoster for Bayesian Statistics in the Big Data Era conference
Poster for Bayesian Statistics in the Big Data Era conference
 
Intro to ABC
Intro to ABCIntro to ABC
Intro to ABC
 
better together? statistical learning in models made of modules
better together? statistical learning in models made of modulesbetter together? statistical learning in models made of modules
better together? statistical learning in models made of modules
 

Similar to ABC-Gibbs

Asymptotics of ABC, lecture, Collège de France
Asymptotics of ABC, lecture, Collège de FranceAsymptotics of ABC, lecture, Collège de France
Asymptotics of ABC, lecture, Collège de FranceChristian Robert
 
Workshop in honour of Don Poskitt and Gael Martin
Workshop in honour of Don Poskitt and Gael MartinWorkshop in honour of Don Poskitt and Gael Martin
Workshop in honour of Don Poskitt and Gael MartinChristian Robert
 
ABC with Wasserstein distances
ABC with Wasserstein distancesABC with Wasserstein distances
ABC with Wasserstein distancesChristian Robert
 
Approximate Bayesian computation for the Ising/Potts model
Approximate Bayesian computation for the Ising/Potts modelApproximate Bayesian computation for the Ising/Potts model
Approximate Bayesian computation for the Ising/Potts modelMatt Moores
 
Stratified sampling and resampling for approximate Bayesian computation
Stratified sampling and resampling for approximate Bayesian computationStratified sampling and resampling for approximate Bayesian computation
Stratified sampling and resampling for approximate Bayesian computationUmberto Picchini
 
Columbia workshop [ABC model choice]
Columbia workshop [ABC model choice]Columbia workshop [ABC model choice]
Columbia workshop [ABC model choice]Christian Robert
 
Workshop on Bayesian Inference for Latent Gaussian Models with Applications
Workshop on Bayesian Inference for Latent Gaussian Models with ApplicationsWorkshop on Bayesian Inference for Latent Gaussian Models with Applications
Workshop on Bayesian Inference for Latent Gaussian Models with ApplicationsChristian Robert
 
Automatic Bayesian method for Numerical Integration
Automatic Bayesian method for Numerical Integration Automatic Bayesian method for Numerical Integration
Automatic Bayesian method for Numerical Integration Jagadeeswaran Rathinavel
 
ABC with data cloning for MLE in state space models
ABC with data cloning for MLE in state space modelsABC with data cloning for MLE in state space models
ABC with data cloning for MLE in state space modelsUmberto Picchini
 
Colloquium in honor of Hans Ruedi Künsch
Colloquium in honor of Hans Ruedi KünschColloquium in honor of Hans Ruedi Künsch
Colloquium in honor of Hans Ruedi KünschChristian Robert
 
Semi-automatic ABC: a discussion
Semi-automatic ABC: a discussionSemi-automatic ABC: a discussion
Semi-automatic ABC: a discussionChristian Robert
 
SPDE presentation 2012
SPDE presentation 2012SPDE presentation 2012
SPDE presentation 2012Zheng Mengdi
 
Markov chain Monte Carlo methods and some attempts at parallelizing them
Markov chain Monte Carlo methods and some attempts at parallelizing themMarkov chain Monte Carlo methods and some attempts at parallelizing them
Markov chain Monte Carlo methods and some attempts at parallelizing themPierre Jacob
 
ABC in London, May 5, 2011
ABC in London, May 5, 2011ABC in London, May 5, 2011
ABC in London, May 5, 2011Christian Robert
 
Stratified Monte Carlo and bootstrapping for approximate Bayesian computation
Stratified Monte Carlo and bootstrapping for approximate Bayesian computationStratified Monte Carlo and bootstrapping for approximate Bayesian computation
Stratified Monte Carlo and bootstrapping for approximate Bayesian computationUmberto Picchini
 

Similar to ABC-Gibbs (20)

Asymptotics of ABC, lecture, Collège de France
Asymptotics of ABC, lecture, Collège de FranceAsymptotics of ABC, lecture, Collège de France
Asymptotics of ABC, lecture, Collège de France
 
Workshop in honour of Don Poskitt and Gael Martin
Workshop in honour of Don Poskitt and Gael MartinWorkshop in honour of Don Poskitt and Gael Martin
Workshop in honour of Don Poskitt and Gael Martin
 
ABC with Wasserstein distances
ABC with Wasserstein distancesABC with Wasserstein distances
ABC with Wasserstein distances
 
Automatic bayesian cubature
Automatic bayesian cubatureAutomatic bayesian cubature
Automatic bayesian cubature
 
Approximate Bayesian computation for the Ising/Potts model
Approximate Bayesian computation for the Ising/Potts modelApproximate Bayesian computation for the Ising/Potts model
Approximate Bayesian computation for the Ising/Potts model
 
Stratified sampling and resampling for approximate Bayesian computation
Stratified sampling and resampling for approximate Bayesian computationStratified sampling and resampling for approximate Bayesian computation
Stratified sampling and resampling for approximate Bayesian computation
 
Columbia workshop [ABC model choice]
Columbia workshop [ABC model choice]Columbia workshop [ABC model choice]
Columbia workshop [ABC model choice]
 
Workshop on Bayesian Inference for Latent Gaussian Models with Applications
Workshop on Bayesian Inference for Latent Gaussian Models with ApplicationsWorkshop on Bayesian Inference for Latent Gaussian Models with Applications
Workshop on Bayesian Inference for Latent Gaussian Models with Applications
 
Edinburgh, Bayes-250
Edinburgh, Bayes-250Edinburgh, Bayes-250
Edinburgh, Bayes-250
 
CDT 22 slides.pdf
CDT 22 slides.pdfCDT 22 slides.pdf
CDT 22 slides.pdf
 
Automatic Bayesian method for Numerical Integration
Automatic Bayesian method for Numerical Integration Automatic Bayesian method for Numerical Integration
Automatic Bayesian method for Numerical Integration
 
ABC with data cloning for MLE in state space models
ABC with data cloning for MLE in state space modelsABC with data cloning for MLE in state space models
ABC with data cloning for MLE in state space models
 
QMC: Transition Workshop - Density Estimation by Randomized Quasi-Monte Carlo...
QMC: Transition Workshop - Density Estimation by Randomized Quasi-Monte Carlo...QMC: Transition Workshop - Density Estimation by Randomized Quasi-Monte Carlo...
QMC: Transition Workshop - Density Estimation by Randomized Quasi-Monte Carlo...
 
Colloquium in honor of Hans Ruedi Künsch
Colloquium in honor of Hans Ruedi KünschColloquium in honor of Hans Ruedi Künsch
Colloquium in honor of Hans Ruedi Künsch
 
Semi-automatic ABC: a discussion
Semi-automatic ABC: a discussionSemi-automatic ABC: a discussion
Semi-automatic ABC: a discussion
 
SPDE presentation 2012
SPDE presentation 2012SPDE presentation 2012
SPDE presentation 2012
 
BIRS 12w5105 meeting
BIRS 12w5105 meetingBIRS 12w5105 meeting
BIRS 12w5105 meeting
 
Markov chain Monte Carlo methods and some attempts at parallelizing them
Markov chain Monte Carlo methods and some attempts at parallelizing themMarkov chain Monte Carlo methods and some attempts at parallelizing them
Markov chain Monte Carlo methods and some attempts at parallelizing them
 
ABC in London, May 5, 2011
ABC in London, May 5, 2011ABC in London, May 5, 2011
ABC in London, May 5, 2011
 
Stratified Monte Carlo and bootstrapping for approximate Bayesian computation
Stratified Monte Carlo and bootstrapping for approximate Bayesian computationStratified Monte Carlo and bootstrapping for approximate Bayesian computation
Stratified Monte Carlo and bootstrapping for approximate Bayesian computation
 

More from Christian Robert

How many components in a mixture?
How many components in a mixture?How many components in a mixture?
How many components in a mixture?Christian Robert
 
Testing for mixtures at BNP 13
Testing for mixtures at BNP 13Testing for mixtures at BNP 13
Testing for mixtures at BNP 13Christian Robert
 
Inferring the number of components: dream or reality?
Inferring the number of components: dream or reality?Inferring the number of components: dream or reality?
Inferring the number of components: dream or reality?Christian Robert
 
Testing for mixtures by seeking components
Testing for mixtures by seeking componentsTesting for mixtures by seeking components
Testing for mixtures by seeking componentsChristian Robert
 
discussion on Bayesian restricted likelihood
discussion on Bayesian restricted likelihooddiscussion on Bayesian restricted likelihood
discussion on Bayesian restricted likelihoodChristian Robert
 
a discussion of Chib, Shin, and Simoni (2017-8) Bayesian moment models
a discussion of Chib, Shin, and Simoni (2017-8) Bayesian moment modelsa discussion of Chib, Shin, and Simoni (2017-8) Bayesian moment models
a discussion of Chib, Shin, and Simoni (2017-8) Bayesian moment modelsChristian Robert
 
short course at CIRM, Bayesian Masterclass, October 2018
short course at CIRM, Bayesian Masterclass, October 2018short course at CIRM, Bayesian Masterclass, October 2018
short course at CIRM, Bayesian Masterclass, October 2018Christian Robert
 
prior selection for mixture estimation
prior selection for mixture estimationprior selection for mixture estimation
prior selection for mixture estimationChristian Robert
 
Coordinate sampler: A non-reversible Gibbs-like sampler
Coordinate sampler: A non-reversible Gibbs-like samplerCoordinate sampler: A non-reversible Gibbs-like sampler
Coordinate sampler: A non-reversible Gibbs-like samplerChristian Robert
 
comments on exponential ergodicity of the bouncy particle sampler
comments on exponential ergodicity of the bouncy particle samplercomments on exponential ergodicity of the bouncy particle sampler
comments on exponential ergodicity of the bouncy particle samplerChristian Robert
 

More from Christian Robert (13)

discussion of ICML23.pdf
discussion of ICML23.pdfdiscussion of ICML23.pdf
discussion of ICML23.pdf
 
How many components in a mixture?
How many components in a mixture?How many components in a mixture?
How many components in a mixture?
 
restore.pdf
restore.pdfrestore.pdf
restore.pdf
 
Testing for mixtures at BNP 13
Testing for mixtures at BNP 13Testing for mixtures at BNP 13
Testing for mixtures at BNP 13
 
Inferring the number of components: dream or reality?
Inferring the number of components: dream or reality?Inferring the number of components: dream or reality?
Inferring the number of components: dream or reality?
 
Testing for mixtures by seeking components
Testing for mixtures by seeking componentsTesting for mixtures by seeking components
Testing for mixtures by seeking components
 
discussion on Bayesian restricted likelihood
discussion on Bayesian restricted likelihooddiscussion on Bayesian restricted likelihood
discussion on Bayesian restricted likelihood
 
eugenics and statistics
eugenics and statisticseugenics and statistics
eugenics and statistics
 
a discussion of Chib, Shin, and Simoni (2017-8) Bayesian moment models
a discussion of Chib, Shin, and Simoni (2017-8) Bayesian moment modelsa discussion of Chib, Shin, and Simoni (2017-8) Bayesian moment models
a discussion of Chib, Shin, and Simoni (2017-8) Bayesian moment models
 
short course at CIRM, Bayesian Masterclass, October 2018
short course at CIRM, Bayesian Masterclass, October 2018short course at CIRM, Bayesian Masterclass, October 2018
short course at CIRM, Bayesian Masterclass, October 2018
 
prior selection for mixture estimation
prior selection for mixture estimationprior selection for mixture estimation
prior selection for mixture estimation
 
Coordinate sampler: A non-reversible Gibbs-like sampler
Coordinate sampler: A non-reversible Gibbs-like samplerCoordinate sampler: A non-reversible Gibbs-like sampler
Coordinate sampler: A non-reversible Gibbs-like sampler
 
comments on exponential ergodicity of the bouncy particle sampler
comments on exponential ergodicity of the bouncy particle samplercomments on exponential ergodicity of the bouncy particle sampler
comments on exponential ergodicity of the bouncy particle sampler
 

Recently uploaded

Biopesticide (2).pptx .This slides helps to know the different types of biop...
Biopesticide (2).pptx  .This slides helps to know the different types of biop...Biopesticide (2).pptx  .This slides helps to know the different types of biop...
Biopesticide (2).pptx .This slides helps to know the different types of biop...RohitNehra6
 
Forensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdfForensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdfrohankumarsinghrore1
 
Chemistry 4th semester series (krishna).pdf
Chemistry 4th semester series (krishna).pdfChemistry 4th semester series (krishna).pdf
Chemistry 4th semester series (krishna).pdfSumit Kumar yadav
 
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service 🪡
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service  🪡CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service  🪡
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service 🪡anilsa9823
 
VIRUSES structure and classification ppt by Dr.Prince C P
VIRUSES structure and classification ppt by Dr.Prince C PVIRUSES structure and classification ppt by Dr.Prince C P
VIRUSES structure and classification ppt by Dr.Prince C PPRINCE C P
 
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...anilsa9823
 
Botany 4th semester file By Sumit Kumar yadav.pdf
Botany 4th semester file By Sumit Kumar yadav.pdfBotany 4th semester file By Sumit Kumar yadav.pdf
Botany 4th semester file By Sumit Kumar yadav.pdfSumit Kumar yadav
 
Nanoparticles synthesis and characterization​ ​
Nanoparticles synthesis and characterization​  ​Nanoparticles synthesis and characterization​  ​
Nanoparticles synthesis and characterization​ ​kaibalyasahoo82800
 
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdfPests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdfPirithiRaju
 
DIFFERENCE IN BACK CROSS AND TEST CROSS
DIFFERENCE IN  BACK CROSS AND TEST CROSSDIFFERENCE IN  BACK CROSS AND TEST CROSS
DIFFERENCE IN BACK CROSS AND TEST CROSSLeenakshiTyagi
 
Presentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptxPresentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptxgindu3009
 
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 60009654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000Sapana Sha
 
fundamental of entomology all in one topics of entomology
fundamental of entomology all in one topics of entomologyfundamental of entomology all in one topics of entomology
fundamental of entomology all in one topics of entomologyDrAnita Sharma
 
Formation of low mass protostars and their circumstellar disks
Formation of low mass protostars and their circumstellar disksFormation of low mass protostars and their circumstellar disks
Formation of low mass protostars and their circumstellar disksSérgio Sacani
 
Botany 4th semester series (krishna).pdf
Botany 4th semester series (krishna).pdfBotany 4th semester series (krishna).pdf
Botany 4th semester series (krishna).pdfSumit Kumar yadav
 
Stunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCR
Stunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCRStunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCR
Stunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCRDelhi Call girls
 
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticsPulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticssakshisoni2385
 
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptxPhysiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptxAArockiyaNisha
 
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptx
Unlocking  the Potential: Deep dive into ocean of Ceramic Magnets.pptxUnlocking  the Potential: Deep dive into ocean of Ceramic Magnets.pptx
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptxanandsmhk
 
Animal Communication- Auditory and Visual.pptx
Animal Communication- Auditory and Visual.pptxAnimal Communication- Auditory and Visual.pptx
Animal Communication- Auditory and Visual.pptxUmerFayaz5
 

Recently uploaded (20)

Biopesticide (2).pptx .This slides helps to know the different types of biop...
Biopesticide (2).pptx  .This slides helps to know the different types of biop...Biopesticide (2).pptx  .This slides helps to know the different types of biop...
Biopesticide (2).pptx .This slides helps to know the different types of biop...
 
Forensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdfForensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdf
 
Chemistry 4th semester series (krishna).pdf
Chemistry 4th semester series (krishna).pdfChemistry 4th semester series (krishna).pdf
Chemistry 4th semester series (krishna).pdf
 
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service 🪡
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service  🪡CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service  🪡
CALL ON ➥8923113531 🔝Call Girls Kesar Bagh Lucknow best Night Fun service 🪡
 
VIRUSES structure and classification ppt by Dr.Prince C P
VIRUSES structure and classification ppt by Dr.Prince C PVIRUSES structure and classification ppt by Dr.Prince C P
VIRUSES structure and classification ppt by Dr.Prince C P
 
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...
 
Botany 4th semester file By Sumit Kumar yadav.pdf
Botany 4th semester file By Sumit Kumar yadav.pdfBotany 4th semester file By Sumit Kumar yadav.pdf
Botany 4th semester file By Sumit Kumar yadav.pdf
 
Nanoparticles synthesis and characterization​ ​
Nanoparticles synthesis and characterization​  ​Nanoparticles synthesis and characterization​  ​
Nanoparticles synthesis and characterization​ ​
 
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdfPests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
Pests of cotton_Borer_Pests_Binomics_Dr.UPR.pdf
 
DIFFERENCE IN BACK CROSS AND TEST CROSS
DIFFERENCE IN  BACK CROSS AND TEST CROSSDIFFERENCE IN  BACK CROSS AND TEST CROSS
DIFFERENCE IN BACK CROSS AND TEST CROSS
 
Presentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptxPresentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptx
 
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 60009654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000
 
fundamental of entomology all in one topics of entomology
fundamental of entomology all in one topics of entomologyfundamental of entomology all in one topics of entomology
fundamental of entomology all in one topics of entomology
 
Formation of low mass protostars and their circumstellar disks
Formation of low mass protostars and their circumstellar disksFormation of low mass protostars and their circumstellar disks
Formation of low mass protostars and their circumstellar disks
 
Botany 4th semester series (krishna).pdf
Botany 4th semester series (krishna).pdfBotany 4th semester series (krishna).pdf
Botany 4th semester series (krishna).pdf
 
Stunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCR
Stunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCRStunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCR
Stunning ➥8448380779▻ Call Girls In Panchshil Enclave Delhi NCR
 
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticsPulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
 
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptxPhysiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
 
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptx
Unlocking  the Potential: Deep dive into ocean of Ceramic Magnets.pptxUnlocking  the Potential: Deep dive into ocean of Ceramic Magnets.pptx
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptx
 
Animal Communication- Auditory and Visual.pptx
Animal Communication- Auditory and Visual.pptxAnimal Communication- Auditory and Visual.pptx
Animal Communication- Auditory and Visual.pptx
 

ABC-Gibbs

  • 1. Component-wise approximate Bayesian computation via Gibbs-like steps Christian P. Robert(1,2) arXiv:1905.13599 with Grégoire Clarté(1) , Robin Ryder(1) , Julien Stoehr(1) (1) Université Paris-Dauphine, (2) University of Warwick Université Paris-Dauphine February 28, 2020
  • 2. Approximate Bayesian computation (ABC) ABC is a computational method which stemmed from population ge- netics models about 20 years ago to deal with generative intractable distribution. [Tavaré et al., 1997; Beaumont et al., 2002] Settings of interest: the likelihood function f(x | θ) does not admit a closed form as a function of θ and/or is computationally too costly. 1. Model relying on a latent process z ∈ Z f(x | θ) = Z f(y, z | θ)µ(dz). 2. Model with intractable normalising constant f(x | θ) = 1 Z(θ) q(x | θ), where Z(θ) = X q(x | θ)µ(dx).
  • 3. Approximate Bayesian computation (ABC) Bayesian settings: the target is π(θ | xobs ) ∝ π(θ)f(xobs | θ). Algorithm: Vanilla ABC Input: observed dataset xobs , number of iterations N, threshold ε, summary statistic s. for i = 1, . . . , N do θi ∼ π(·) xi ∼ f(· | θi) end return θi d(s(xobs ), s(xi)) ≤ ε s(xobs) ε (θi, S(xi))
  • 4. Approximate Bayesian computation (ABC) Bayesian settings: the target is π(θ | xobs ) ∝ π(θ)f(xobs | θ). Algorithm: Vanilla ABC Input: observed dataset xobs , number of iterations N, threshold ε, summary statistic s. for i = 1, . . . , N do θi ∼ π(·) xi ∼ f(· | θi) end return θi d(s(xobs ), s(xi)) ≤ ε s(xobs) ε (θi, S(xi)) Ouput: distributed according to πε(θ | s, xobs ) π(θ)Pθ d(S(xobs ), S(x)) < ε ∝ π(θ | d(S(xobs ), S(x)) < ε)
  • 5. Approximate Bayesian computation (ABC) Two limiting situations: π∞(θ | s, xobs ) ∝ π(θ) and π0(θ | s, xobs ) ∝ π(θ | s(xobs ))= π(θ | xobs )
  • 6. Approximate Bayesian computation (ABC) Two limiting situations: π∞(θ | s, xobs ) ∝ π(θ) and π0(θ | s, xobs ) ∝ π(θ | s(xobs ))= π(θ | xobs ) Some difficulties raised by the vanilla version: Calibration of the threshold ε: by regression or k-nearest neigh- bour postprocessing [Beaumont et al., 2002; Wilkinson, 2013; Biau et al., 2013] Selection of the summary statistic s: advances with semi-automatic procedures like pilot-run ABC or random forests [Fearnhead and Prangle, 2012; Prangle et al., 2014; Raynal et al., 2018] Simulating from the prior is often poor in efficiency: versions modifying proposal distribution on θ to increase density of x’s near y (MCMC, SMC, PMC) [Marjoram et al., 2003; Toni et al., 2008]
  • 7. A first example : hierarchical moving average model α µ1 µ2 µn. . . x1 x2 xn. . . σ σ1 σ2 σn. . . First parameter hierarchy: α = (α1, α2, α3) ∼ E(1)⊗3 independently for each i ∈ {1, . . . , n}, (βi,1, βi,2, βi,3) ∼ Dir(α1, α2, α3) µi = (βi,1 − βi,2, 2(βi,1 + βi,2) − 1) Second parameter hierarchy: σ = (σ1, σ2) ∼ C+ (1)⊗2 . independently for each i ∈ {1, . . . , n}, σi ∼ IG(σ1, σ2) Model for xi: independently for each i ∈ {1, . . . , n}, xi ∼ MA2(µi, σi), i.e., for all j in N xi,j = yj + µi,1yj−1 + µi,2yj−2 , with yj ∼ N(0, σ2 i )
  • 8. A first example : toy dataset Settings: n = 5 times series of length T = 100 hierarchical model with 13 parameters. Figure: Histogram of ABC sam- ple for µ1,1 along prior distribution (black line) Size of ABC reference table: N = 5.5 · 106 . ABC posterior sample size: 1000.
  • 9. A first example : toy dataset Settings: n = 5 times series of length T = 100 hierarchical model with 13 parameters. Figure: Histogram of ABC sam- ple for µ1,1 along prior distribution (black line) Size of ABC reference table: N = 5.5 · 106 . ABC posterior sample size: 1000. Not enough simulations to reach a decent threshold Not enough time to produce enough simulations
  • 10. The Gibbs Sampler Our idea: combining ABC with Gibbs sampler in order to improve ability to (more) efficiently explore Θ ⊂ Rn when the number n of parameters increases.
  • 11. The Gibbs Sampler Our idea: combining ABC with Gibbs sampler in order to improve ability to (more) efficiently explore Θ ⊂ Rn when the number n of parameters increases. The Gibbs Sampler produces a Markov chain with a target joint dis- tribution π by alternatively sampling from each of its conditionals. [Geman and Geman, 1984] Algorithm: Gibbs sampler Input: observed dataset xobs , number of iterations N, starting point θ(0) = (θ (0) 1 , . . . , θ (0) n ). for i = 1, . . . , N do for k = 1, . . . , n do θ (i) k ∼ π · | θ (i) 1 , . . . , θ (i) k−1, θ (i−1) k+1 , . . . , θ (i−1) n , xobs end end return θ(0) , . . . , θ(N)
  • 12. Component-wise ABC [aka ABCG] ABC Gibbs Sampler produces a Markov chain with each proposal an ABC posterior conditional on different summaries sj and tolerance εj Algorithm: Component-wise ABC Input: observed dataset xobs , number of iterations N, starting point θ(0) = (θ (0) 1 , . . . , θ (0) n ), threshold ε = (ε1, . . . , εn), statistics s1, . . . , sn. for i = 1, . . . , N do for j = 1, . . . , n do θ (i) j ∼ πεj (· | xobs , sj, θ (i) 1 , . . . , θ (i) j−1, θ (i−1) j+1 , . . . , θ (i−1) n ) end end return θ(0) , . . . , θ(N)
  • 13. Component-wise ABC [aka ABCG] Algorithm: Component-wise ABC Input: observed dataset xobs , number of iterations N, starting point θ(0) = (θ (0) 1 , . . . , θ (0) n ), threshold ε = (ε1, . . . , εn), statistics s1, . . . , sn. for i = 1, . . . , N do for j = 1, . . . , n do θ (i) j ∼ πεj (· | xobs , sj, θ (i) 1 , . . . , θ (i) j−1, θ (i−1) j+1 , . . . , θ (i−1) n ) end end return θ(0) , . . . , θ(N) Questions: Is there a limiting distribution ν∞ ε to the algorithm? What is the nature of this limiting distribution?
  • 14. Gibbs with ABC approximations Souza Rodrigues et al. (arxiv:1906.04347) alternative ABC-ed Gibbs: Further references to earlier occurrences of Gibbs versions of ABC ABC version of Gibbs sampling with approximations to the con- ditionals with no further corrections Related to regression post-processing à la Beaumont et al. (2002) used to designing approximate full conditional, possibly involv- ing neural networks Requires preliminary ABC step Drawing from approximate full conditionals done exactly, possi- bly via a bootstrapped version.
  • 15. ABC within Gibbs: Hierarchical models α µ1 µ2 µn. . . x1 x2 xn. . . Hierarchical Bayes models: often allow for simplified conditional distributions thanks to partial independence properties, e.g., xj | µj ∼ π(xj | µj), µj | α i.i.d. ∼ π(µj | α), α ∼ π(α). Algorithm: Component-wise ABC sampler for hierarchical model Input: observed dataset xobs , number of iterations N, thresholds εα and εµ, summary statistics sα and sµ. for i = 1, . . . , N do for j = 1, . . . , n do µ (i) j ∼ πεµ (· | xobs j , sµ, α(i−1) ) end α(i) ∼ πεα (· | µ(i) , sα) end
  • 16. ABC within Gibbs: Hierarchical models Assumption: n = 1. Theorem (Clarté et al. [2019]) Assume there exists a non-empty convex set C with positive prior measure such that κ1 = inf sα(µ)∈C π(Bsα(µ), α/4) > 0 , κ2 = inf α inf sα(µ)∈C πεµ (Bsα(µ),3 α/2 | xobs , sµ, α) > 0 , κ3 = inf α πεµ (sα(µ) ∈ C | xobs , sµ, α) > 0 , Then the Markov chain converges geometrically in total variation distance to a stationary distribution ν∞ ε , with geometric rate 1 − κ1κ2κ2 3. If the prior on α is defined on a compact set, then the assumptions are satisfied.
  • 17. ABC within Gibbs: Hierarchical models Theorem (Clarté et al. [2019]) Assume that, L0 = sup εα sup µ, ˜µ πεα (· | sα, µ) − π0(· | sα, ˜µ) TV < 1/2 , L1(εα) = sup µ πεα (· | sα, µ) − π0(· | sα, µ) TV −−−−→ εα→0 0 L2(εµ) = sup α πεµ (· | xobs , sµ, α) − π0(· | xobs , sµ, α) TV −−−−→ εµ→0 0 . Then, ν∞ ε − ν∞ 0 TV ≤ L1(εα) + L2(εµ) 1 − 2L0 −−−→ ε→0 0.
  • 18. ABC within Gibbs: Hierarchical models Compatibility issue: ν∞ 0 is the limiting distribution associated to Gibbs conditionals with different acceptance events, e.g., different statis- tics: π(α)π(sα(µ) | α) and π(µ)f(sµ(xobs ) | α, µ) Conditionals may then be incompatible and limiting distribution not genuine posterior [incoherent use of data] unknown [except for a specific version] possibly far from a genuine posterior Proposition (Clarté et al. [2019]) If sα is jointly sufficient, when the precision ε goes to zero, ABC within Gibbs and ABC have the same limiting distribution.
  • 19. Hierarchical models: toy example Model: α ∼ U([0 ; 20]), (µ1, . . . , µn) | α ∼ N(α, 1)⊗n , (xi,1, . . . , xi,K) | µi ∼ N (µi, 0.1) ⊗K . Numerical experiment: n = 20, K = 10, Pseudo observation generated for α = 1.7, Algorithms runs for a constant budget: Ntot = N × Nε = 21000. We look at the estimates for µ1 whose value for the pseudo obser- vations is 3.04.
  • 20. Hierarchical models: toy example Illustration: Assumptions of convergence theorem hold Tolerance as quantile of distances at each call, i..e, selection of sim- ulation with smallest distance out of Nα = Nµ = 30 Summary statistic as empirical mean (sufficient): Setting when SMC-ABC fails as well
  • 21. Hierarchical models: toy example Figure: comparison of the sampled densities of µ1 (left) and α (right) [dot-dashed line as true posterior] 0 1 2 3 4 0 2 4 6 0.0 0.5 1.0 1.5 2.0 −4 −2 0 2 4 Method ABC Gibbs Simple ABC
  • 22. Hierarchical models: toy example Figure: posterior densities of µ1 and α for 100 replicas of 3 ABC al- gorithms [true marginal posterior as dashed line] ABC−GibbsSimpleABCSMC−ABC 0 2 4 6 0 1 2 3 4 0 1 2 3 4 0 10 20 30 40 mu1 density ABC−GibbsSimpleABCSMC−ABC 0 1 2 3 4 5 0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.5 1.0 1.5 2.0 0 5 10 15 20 hyperparameter density
  • 23. Hierarchical models: moving average example Pseudo observations: xobs 1 generated for µ1 = (−0.06, −0.22). 0 1 2 3 −1.0 −0.5 0.0 0.5 1.0 value density type ABCGibbs ABCsimple prior 1st parameter, 1st coordinate −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 b1 b2 0.2 0.4 0.6 0.8 level 1st parameter simple −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 b1 b2 2.5 5.0 7.5 10.0 level 1st parameter gibbs Separation from the prior for identical number of simulations.
  • 24. Hierarchical models: moving average example Real dataset: measures of 8GHz daily flux intensity emitted by 7 stellar objects from the NRL GBI website: http://ese.nrl.navy. mil/. [Lazio et al., 2008] 0 1 2 3 −1.0 −0.5 0.0 0.5 1.0 value density type ABCGibbs ABCsimple prior 1st parameter, 1st coordinate −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 b1 b2 0.2 0.4 0.6 level 1st parameter simple −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 b1 b2 2 4 6 8 level 1st parameter gibbs Separation from the prior for identical number of simulations.
  • 25. Hierarchical models: g&k example Model: g-and-k distribution (due to Tukey) defined through inverse cdf: easy to simulate but with no closed-form pdf: A + B 1 + 0.8 1 − exp(−gΦ−1 (r) 1 + exp(−gΦ−1(r) 1 + Φ−1 (r)2 k Φ−1 (r) Note:: MCMC feasible in this setting (Pierre Jacob) α A1 A2 ... An x1 x2 xn ... B g k
  • 26. Hierarchical models: g&k example Assumption: B, g and k known, inference on α and Ai solely. 1 2 3 4 Hyperparameter −7.5 −7.0 −6.5 −6.0 −5.5 −5.0−7.5 −7.0 −6.5 −6.0 −5.5 −5.0−7.5 −7.0 −6.5 −6.0 −5.5 −5.0−7.5 −7.0 −6.5 −6.0 −5.5 −5.0−7.5 −7.0 −6.5 −6.0 −5.5 −5.0 0 2 4 6 8 value density Method ABC Gibbs ABC−SMC vanilla ABC
  • 27. ABC within Gibbs: general case A general two-parameter model: (θ1, θ2) x Algorithm: ABC within Gibbs for i = 1, . . . , N do θ (i) 2 ∼ πε2 (· | θ (i−1) 1 , s2, xobs ) θ (i) 1 ∼ πε1 (· | θ (i) 2 , s1, xobs ) end return (θ (i) 1 , θ (i) 2 )i=2,...,N
  • 28. ABC within Gibbs: general case A general two-parameter model: (θ1, θ2) x Algorithm: ABC within Gibbs for i = 1, . . . , N do θ (i) 2 ∼ πε2 (· | θ (i−1) 1 , s2, xobs ) θ (i) 1 ∼ πε1 (· | θ (i) 2 , s1, xobs ) end return (θ (i) 1 , θ (i) 2 )i=2,...,N Theorem (Clarté et al. [2019]) Assume that there exists 0 < κ < 1/2 such that sup θ1, ˜θ1 πε2 (· | xobs , s2, θ1) − πε2 (· | xobs , s2, ˜θ1) TV = κ. The Markov chain then converges geometrically in total variation distance to a stationary distribution ν∞ ε , with geometric rate 1 − 2κ.
  • 29. ABC within Gibbs: general case Additional assumption: θ1 and θ2 are a priori independent Theorem (Clarté et al. [2019]) Assume that κ1 = inf θ1,θ2 π(Bs1(xobs),ε1 | θ1, θ2) > 0 , κ2 = inf θ1,θ2 π(Bs2(xobs), 2 | θ1, θ2) > 0 , κ3 = sup θ1, ˜θ1,θ2 π(· | θ1, θ2) − π(· | ˜θ1, θ2) TV < 1/2 . Then the Markov chain converges in total variation distance to a stationary distribution ν∞ ε with geometric rate 1 − κ1κ2(1 − 2κ3).
  • 30. ABC within Gibbs: general case For both situations, a limiting distribution exists when the thresholds go to 0. Theorem (Clarté et al. [2019]) Assume that L0 = sup ε2 sup θ1, ˜θ1 πε2 (· | xobs , s2, θ1) − π0(· | xobs , s2, ˜θ1) TV < 1/2 , L1(ε1) = sup θ2 πε1 (· | xobs , s1, θ2) − π0(· | xobs , s1, θ2) TV −−−−→ ε1→0 0 , L2(ε2) = sup θ1 πε2 (· | xobs , s2, θ1) − π0(· | xobs , s2, θ1) TV −−−−→ ε2→0 0 . Then ν∞ ε − ν∞ 0 TV ≤ L1(ε1) + L2(ε2) 1 − 2L0 −−−→ ε→0 0.
  • 31. ABC within Gibbs: general case Compatibility issue: the general case inherits the compatibility issue already noticed in the hierarchical setting. Lemma (Clarté et al. [2019]) 1. If sθ1 and sθ2 are conditionally sufficient, the conditionals are compatible and , when the precision goes to zero, ABC within Gibbs and ABC have the same limiting distribution. 2. If π(θ1, θ2) = π(θ1)π(θ2) and sθ1 = sθ2 , when the precision goes to zero, ABC within Gibbs and ABC have the same limiting distribution.
  • 32. General case: g&k example Figure: posterior densities for parameters µ1, . . . , µ4 in the doubly hierarchical g & k model µ1 µ2 µ3 µ4 −4 −3 −2 −1 0 −4 −3 −2 −1 0 −4 −3 −2 −1 0 −4 −3 −2 −1 0 0 1 2 3 4 5 Method ABC Gibbs ABC−SMC vanilla ABC
  • 33. General case: g&k example Figure: posterior densities for α, B, g and k α B g k −3 −2 −1 0 0.00 0.25 0.50 0.75 1.000.00 0.25 0.50 0.75 1.000.00 0.25 0.50 0.75 1.00 0.0 2.5 5.0 7.5 10.0 Method ABC Gibbs ABC−SMC vanilla ABC
  • 34. General case: inverse problem Example inspired by deterministic inverse problems: difficult to tackle with traditional methods pseudo-likelihood extremely expensive to compute requiring the use of surrogate models Let y be solution of heat equation on a circle defined for (τ, z) ∈]0, T]× [0, 1[ by ∂τy(z, τ) = ∂z (θ(z)∂zy(z, τ)) , with θ(z) = n j=1 θj1[(j−1)/n,j/n](z) and boundary conditions y(z, 0) = y0(z) and y(0, τ) = y(1, τ) [Kaipio and Fox, 2011]
  • 35. General case: inverse problem Assume y0 known and parameter θ = (θ1, . . . , θn), with discretized equation via first order finite elements of size 1/n for z and ∆ for τ. Stepwise approximation of solution ^y(z, t) = n j=1 yj,tφj(z) where, for j < n, φj(z) = (1 − |nz − j|)1|nz−j|<1 and φn(z) = (1 − nz)10<z<1/n + (nz − n + 1)11−1/n<z<1 and with yj,t defined by yj,t+1 − yj,t 3∆ + yj+1,t+1 − yj+1,t 6∆ + yj−1,t+1 − yj−1,t 6∆ = yj,t+1(θj+1 + θj) − yj−1,t+1θj − yj+1,t+1θj+1. Noisy version of this process, chosen as xj,t = N(^yj,t, σ2 ).
  • 36. General case: inverse problem In ABC-Gibbs, each parameter θm updated with summary statistics observations at locations m−2, m−1, m, m+1 while ABC-SMC relies on the whole data as statistic. In experiment, n = 20 and ∆ = 0.1, with a prior θj ∼ U[0, 1], independently. Convergence theorem applies to this setting. Comparison with ABC, using same simulation budget, keeping the total number of simulations constant at N · N = 8 · 106 . Choice of ABC reference table size critical: for a fixed computational budget reach balance between quality of the approximations of the conditionals (improved by increasing N ), and Monte-Carlo error and convergence of the algorithm, (improved by increasing N).
  • 37. General case: inverse problem Figure: mean and variance of the ABC and ABC-Gibbs estimators of θ1 as N increases [horizontal line shows true value] 0.5 0.6 0.7 0.8 0 10 20 30 40 0.000 0.002 0.004 0 10 20 30 40 Method ABC Gibbs Simple ABC
  • 38. General case: inverse problem Figure: histograms of ABC-Gibbs and ABC outputs compared with uniform prior ABC Gibbs Simple ABC 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0 1 2 3
  • 39. Explicit limiting distribution For hierarchical model xj | µj ∼ π(xj | µj) , µj | α i.i.d. ∼ π(µj | α) , α ∼ π(α) alternative ABC based on: ˜π(α, µ | xobs ) ∝ π(α)q(µ) generate a new µ π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα d˜µ × f(˜x | µ)π(xobs | µ) with q arbitrary distribution on µ
  • 40. Explicit limiting distribution For hierarchical model xj | µj ∼ π(xj | µj) , µj | α i.i.d. ∼ π(µj | α) , α ∼ π(α) induces full conditionals ˜π(α | µ) ∝ π(α) π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα d˜x and ˜π(µ | α, xobs ) ∝ q(µ) π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα d˜µ × f(˜x | µ)π(xobs | µ)1d(sµ(xobs),sµ( ˜x))<εµ d˜x now compatible with new artificial joint
  • 41. Explicit limiting distribution For hierarchical model xj | µj ∼ π(xj | µj) , µj | α i.i.d. ∼ π(µj | α) , α ∼ π(α) that is, prior simulations of α ∼ π(α) and of ˜µ ∼ π(˜µ | α) until d(sα(µ), sα(˜µ)) < εα simulation of µ from instrumental q(µ) and of auxiliary variables ˜µ and ˜x until both constraints satisfied
  • 42. Explicit limiting distribution For hierarchical model xj | µj ∼ π(xj | µj) , µj | α i.i.d. ∼ π(µj | α) , α ∼ π(α) Resulting Gibbs sampler stationary for posterior proportional to π(α, µ) q(sα(µ)) projection f(sµ(xobs ) | µ) projection that is, for likelihood associated with sµ(xobs ) and prior distribution proportional to π(α, µ)q(sα(µ)) [exact!]
  • 43. Explicit limiting distribution Figure: histograms of ABC-Gibbs and ABC outputs compared with uniform prior ABC Gibbs Simple ABC 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0 1 2 3
  • 44. Explicit limiting distribution For hierarchical model xj | µj ∼ π(xj | µj) , µj | α i.i.d. ∼ π(µj | α) , α ∼ π(α) alternative ABC based on: ˜π(α, µ | xobs ) ∝ π(α)q(µ) generate a new µ π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα d˜µ × f(˜x | µ)π(xobs | µ) with q arbitrary distribution on µ
  • 45. Explicit limiting distribution For hierarchical model xj | µj ∼ π(xj | µj) , µj | α i.i.d. ∼ π(µj | α) , α ∼ π(α) induces full conditionals ˜π(α | µ) ∝ π(α) π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα d˜x and ˜π(µ | α, xobs ) ∝ q(µ) π(˜µ | α)1d(sα(µ),sα( ˜µ))<εα d˜µ × f(˜x | µ)π(xobs | µ)1d(sµ(xobs),sµ( ˜x))<εµ d˜x now compatible with new artificial joint
  • 46. Explicit limiting distribution For hierarchical model xj | µj ∼ π(xj | µj) , µj | α i.i.d. ∼ π(µj | α) , α ∼ π(α) that is, prior simulations of α ∼ π(α) and of ˜µ ∼ π(˜µ | α) until d(sα(µ), sα(˜µ)) < εα simulation of µ from instrumental q(µ) and of auxiliary variables ˜µ and ˜x until both constraints satisfied
  • 47. Explicit limiting distribution For hierarchical model xj | µj ∼ π(xj | µj) , µj | α i.i.d. ∼ π(µj | α) , α ∼ π(α) Resulting Gibbs sampler stationary for posterior proportional to π(α, µ) q(sα(µ)) projection f(sµ(xobs ) | µ) projection that is, for likelihood associated with sµ(xobs ) and prior distribution proportional to π(α, µ)q(sα(µ)) [exact!]
  • 48. Take home messages Under certain conditions to specify,
  • 49. Take home messages We provide theoretical guarantee on the convergence of ABC within Gibbs. • Result n°1: a limiting distribution ν∞ ε exists when the sample size grows • Result n°2: a limiting distribution ν∞ 0 exists when the thresh- old goes to 0 • Result n°3: ν∞ 0 is the posterior distribution π(θ | s(xobs )). The method inherits issues from vanilla ABC, namely the choice of the statistics [plus compatibility of the condition- als]. In practice, ABC within Gibbs exhibits better performances than vanilla ABC and SMC-ABC [even when conditions not satisfied]
  • 50. ABC postdoc positions 2 post-doc positions open with the ABSint ANR research grant: Focus on approximate Bayesian techniques like ABC, variational Bayes, PAC-Bayes, Bayesian non-parametrics, scalable MCMC, and related topics. A potential direction of research would be the derivation of new Bayesian tools for model checking in such c omplex environments. Terms: up to 24 months, no teaching duty attached, primarily located in Université Paris-Dauphine, with supported periods in Oxford (J. Rousseau) [barring no-deal Brexit!] and visits to Mont- pellier (J.-M. Marin). No hard deadline. If interested, send application to me: bayesianstatistics@gmail.com
  • 51. ABC workshops ABC in Grenoble, France, March 18-19 2020 [mirrored in War- wick] ABC in Longyearbyen, Svalbard/Spitzberg, April 12-13 2021
  • 52. Bibliography I M. A. Beaumont, W. Zhang, and D. J. Balding. Approximate Bayesian Computation in Population Genetics. Genetics, 162(4):2025–2035, 2002. G. Biau, F. Cérou, and A. Guyader. New insights into Approximate Bayesian Computation. Annales de l’Institut Henri Poincaré (B) Prob- abilités et Statistiques, in press, 2013. G. Clarté, C. P. Robert, R. Ryder, and J. Stoehr. Component-wise ap- proximate Bayesian computation via Gibbs-like steps. arXiv preprint arXiv:1905.13599, 2019. P. Fearnhead and D. Prangle. Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 74(3):419–474, 2012. S. Geman and D. Geman. Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6):721–741, 1984.
  • 53. Bibliography II J. P. Kaipio and C. Fox. The Bayesian Framework for Inverse Problems in Heat Transfer. Heat Transfer Engineering, 32(9):718–753, 2011. T. J. W. Lazio, E. B. Waltman, F. D. Ghigo, R. Fiedler, R. S. Foster, and a. K. J. Johnston. A Dual-Frequency, Multiyear Monitoring Program of Compact Radio Sources. The Astrophysical Journal Supplement Se- ries, 136:265, December 2008. doi: 10.1086/322531. P. Marjoram, J. Molitor, V. Plagnol, and S. Tavaré. Markov chain Monte Carlo without likelihoods. Proceedings of the National Academy of Sci- ences, 100(26):15324–15328, 2003. D. Prangle, P. Fearnhead, M. P. Cox, P. J. Biggs, and N. P. French. Semi- automatic selection of summary statistics for ABC model choice. Statistical applications in genetics and molecular biology, 13(1):67–82, 2014. L. Raynal, J.-M. Marin, P. Pudlo, M. Ribatet, C. P. Robert, and A. Es- toup. ABC random forests for Bayesian parameter inference. Bioin- formatics, 2018. doi: 10.1093/bioinformatics/bty867.
  • 54. Bibliography III S. Tavaré, D. J. Balding, R. C. Griffiths, and P. Donnelly. Inferring Coa- lescence Times From DNA Sequence Data. Genetics, 145(2):505–518, 1997. T. Toni, D. Welch, N. Strelkowa, A. Ipsen, and M. P. H. Stumpf. Ap- proximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. Journal of the Royal Soci- ety Interface, 6(31):187–202, 2008. R. D. Wilkinson. Approximate Bayesian computation (ABC) gives ex- act results under the assumption of model error. Statistical Applica- tions in Genetics and Molecular Biology, 12(2):129–141, 2013.