03/AUG/2013 @Matsuo lab, the University of Tokyo.

1. www.***.com
Part2:
Chapter 5
Inferring a Binominal
Proportion via Exact
Mathematical Analysis
Haru Negami
03/08/2013

2. summary
! binomial proportion
" the likelihood function
! the Bernoulli likelihood function
" the prior/posterior distribution
! beta distribution
" estimation ---- uncertainty <-HDI
" prediction ---- p(y)<-(z+a)/(N+a+z)
" comparison ---- best model <-p(D|M)

3. Binomial Proportion
!
! binomial or dichotomous
!
"

4. p(θ| ) = p(D|θ,M)p(θ|M)/p(D|M)
posterior likelihood prior evidence
prior
probability
observation
posterior
probability

5. Likelihood function
! Example : coin flipping
" y = {0,1} ex) (0: , 1: )
" p(y=1|θ)=f(θ)=θ θ [0,1]
" p(y=0|θ) = 1-θ
" the Bernoulli distribution
" p(y|θ) = θy(1-θ)(1-y)
! θ y
! (Σy p(y|θ) = 1)

6. Likelihood function
! Bernoulli likelihood function
" p(y|θ) = θy(1-θ)(1-y)
" y θ
! θ
! y
"
! p(y|θ) = θy(1-θ)(1-y)(y=i) θ

7. p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M)
posterior likelihood prior evidence
prior
probability
observation
posterior
probability
done

8. belief (to make a model)
!
" p(θ|y) = p(y|θ)p(θ)/Σyp(y|θ)
" p(θ) p(y|θ)p(θ)
!
!
" denominator Σyp(y|θ)
" p(θ)
a conjugate prior for p(y|θ)

9. belief (to make a model)
! p(θ) = θa(1-θ)b
p(y|θ)×p(θ) = θy(1-θ)(1-y) × θa(1-θ)b
= θy+a(1-θ)(1-y+b)
! beta
distribution

10. belief (to make a model)
! beta distribution
" a, b 2 (a,b > 0)
" p(θ|a,b) = beta(θ|a,b)
= θ(a-1)(1-θ)(b-1)/B(a,b)
! B(a,b) beta function
" beta distribution normalizer
! B(a,b) = ∫0
1dθ θ(a-1)(1-θ)(b-1)

11. belief (to make a model)
! Beta Distribution
b
a

12. p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M)
posterior likelihood prior evidence
prior
probability
observation
posterior
probability
done

13. belief in detail (prior)
! beta distribution : beta(θ|a,b)
" two parameters
! mean :
! standard deviation :

14. belief in detail (prior)
! beta distribution : beta(θ|a,b)
" guess the values of a and b
! from (observed) data
" ex) a=b=1, a=b=4, etc…
! m = a/(a+b), n=(a+b)
a = mn, b = (1-m)n
! from mean and standard deviation
a 1 & b 1 a<1 &/or b<1

15. p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M)
posterior likelihood prior evidence
prior
probability
observation
posterior
probability
done done

16. belief in
detail(posterior)
! supposition : N flips, z heads
! prior distribution : beta(θ|a,b)
! posterior distribution :
beta(θ|z+a, N-z+b)

17. belief in
detail(posterior)
! supposition : N flips, z heads
! prior distribution : beta(θ|a,b)
! posterior distribution :
beta(θ|z+a, N-z+b)
one of the beauties of
mathematical approach to
Bayesian inference!

18. belief in detail
(updated parameters)
! probability distribution :
" prior : beta(θ|a,b)
N flips, z heads
" posterior : beta(θ|z+a, N-z+b)
! mean :
" prior : a/(a+b)
" posterior : (z+a)/[(z+a)+(N-z+b)]
= (z+a)/(N+a+b) !!!

19. belief in detail
(updated parameters)
! probability distribution :
" prior : beta(θ|a,b)
N flips, z heads
" posterior : beta(θ|z+a, N-z+b)
! mean :
" prior : a/(a+b)
" posterior : (z+a)/[(z+a)+(N-z+b)]
= (z+a)/(N+a+b) !!!

20. belief in detail
(updated parameters)
! probability distribution :
" prior : beta(θ|a,b)
N flips, z heads
" posterior : beta(θ|z+a, N-z+b)
! mean :
" prior : a/(a+b)
" posterior : (z+a)/[(z+a)+(N-z+b)]
= (z+a)/(N+a+b) !!!
0 z/N a/(a+b)
1-α α
α = N/(N+a+b)

21. Discussion(?)
Three inferential goals
! from chapter 4
" estimating the binominal proportion
" predicting Data
" comparing models

22. estimation
! uncertainty of the prior distribution
" From the posterior distribution
! HDI : the highest density interval (chapter 3)
HDI
L : broad
R : narrow
prior dist.
L : more uncertain

23. estimation
! reasonable credibility of a value concerned
" From the posterior distribution
! ROPE : region of practical equivalence
coin flipping
θ = 0.5
credible?
ROPE = [0.48,0.52]
if
95% HDI ∩ ROPE =
then
θ is incredible

24. estimation
! reasonable credibility of a value concerned
" From the posterior distribution
! ROPE : region of practical equivalence
coin flipping
θ = 0.5
credible?
ROPE = [0.48,0.52]
if
95% HDI ∩ ROPE =
then
θ is incredible
includes many extra
assumptions!

25. prediction
! p(y) = ∫dθp(y|θ)p(θ) <-posterior

26. prediction
! p(y) = ∫dθp(y|θ)p(θ) <-posterior
0 z/N a/(a+b)
1-α α
α = N/(N+a+b)

27. prediction (example 1)
! 1st beta(θ|1,1) mean 1/2 (= p(y))
observation : head (N=1,z=1)
! 2nd beta(θ|2,1) mean 2/3 (= p(y))
observation : head (N=1,z=1)
! 3rd beta(θ|3,1) mean 3/4 (= p(y))

28. prediction (example 2)
! 1st beta(θ|100,100) : 1/2 (= p(y))
observation : head (N=1,z=1)
observation : head (N=1,z=1)
! 3rd beta(θ|102,100) : 102/202( 50%)

29. comparison
to compare the models,
p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M)
posterior likelihood prior evidence
prior
probability
observation
posterior
probability

30. comparison
! Calculation of evidence
" p(D|M) = p(z,N)

31. comparison
uniform strongly peaked uniform strongly peaked
N = 14, z = 11 N = 14, z = 7
p(D|M)=0.000183>p(D|M)=6.86×10-5 p(D|M)=1.94×10-5<p(D|M)=5.9×10-5

32. comparison
! both are important
" the prior distribution
" the likelihood function
" in detail, see chapter 4
The best model (so far)
is not a good model.

33. summary
! binomial proportion
" the likelihood function
! the Bernoulli likelihood function
" the prior/posterior distribution
! beta distribution
" estimation ---- uncertainty <-HDI
" prediction ---- p(y)<-(z+a)/(N+a+b)
" comparison ---- best model <-p(D|M)