CMA-ES with local meta-models

Investigating the Local-Meta-Model CMA-ES for
Large Population Sizes

Zyed Bouzarkouna1,2 Anne Auger2 Didier Yu Ding1

1 IFP (Institut Fran¸ais du P´trole)
c e
2 TAO Team, INRIA Saclay-Ile-de-France, LRI

April 07, 2010

Statement of the Problem

Objective
To solve a real-world optimization problem formulated in a
black-box scenario with an objective function f : Rn → R.

multimodal non-smooth
noisy non-convex
f may be:
non-separable computationally expensive
...

Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 2 of 15

A Real-World Problem in Petroleum Engineering

History Matching
The act of adjusting a reservoir model until it closely reproduces
the past behavior of a production history.

A ﬂuid ﬂow simulation takes several minutes to several hours !!


Statement of the Problem (Cont’d)

Diﬃculties
Evolutionary Algorithms (EAs) are usually able to cope with
noise, multiple optima . . .

Computational cost
build a model of f , based on true evaluations ;
use this model during the optimization to save evaluations.

⇒ How to decide whether:
the quality of the model is good enough to continue
exploiting this model ?
or
new evaluations on the “true” objective function should be
performed ?


Table of Contents

1 CMA-ES with Local-Meta-Models
Covariance Matrix Adaptation-ES
Locally Weighted Regression
Approximate Ranking Procedure

2 A New Variant of lmm-CMA
A New Meta-Model Acceptance Criterion
nlmm-CMA Performance

3 Conclusions


Covariance Matrix Adaptation-ES

CMA-ES (Hansen & Ostermeier 2001)
Initialize distribution parameters m, σ and C, set population size
λ ∈ N.
while not terminate
Sample xi = m + σNi (0, C), for i = 1 . . . λ according to a
multivariate normal distribution
Evaluate x1 , . . . , xλ on f
Update distribution parameters
(m, σ, C) ← (m, σ, C, x1 , . . . , xλ , f (x1 ), . . . , f (xλ ))
where
m ∈ Rn : the mean of the multivariate normal distribution
σ ∈ R+ : the step-size
C ∈ Rn×n : the covariance matrix.


Covariance Matrix Adaptation-ES (Cont’d)

Moving the mean
µ(= λ )
2
m= ωi xi:λ .
i=1

where xi:λ is the i th ranked individual:
f (x1:λ ) ≤ . . . f (xµ:λ ) ≤ . . . f (xλ:λ ) ,
Pµ
ω1 ≥ . . . ≥ ωµ > 0, ωi = 1.
i=1

Other updates
Adapting the Covariance Matrix
Step-Size Control
⇒ Updates rely on the ranking of individuals according to f and
not on their exact values on f .



q ∈ Rn : A point to evaluate

⇒ ˆ
f (q) : a full quadratic meta-model on q.



A training set containing m points with their objective function
values (xj , yj = f (xj )) , j = 1..m



We select the k nearest neighbor data points to q according to
Mahalanobis distance with respect to the current covariance matrix
C.



h is the bandwidth deﬁned by the distance of the k th nearest
neighbor data point to q.



ˆ
Building the meta-model f on q
k 2 n(n+3)
min ˆ
f (xj , β) − yj ωj , w.r.t β ∈ R 2
+1
.
j=1

ˆ T
f (q) = β T q1 , · · · , qn , · · · , q1 q2 , · · · , qn−1 qn , q1 , · · · , qn , 1
2 2 .


Approximate Ranking Procedure

Every generation g , CMA-ES has λ points to evaluate.
⇒ Which are the points that must be evaluated with:
the true objective function f ?
ˆ
the meta-model f ?

Approximate ranking procedure (Kern et al. 2006)
1 ˆ
approximate f and rank the µ best individuals
2 evaluate f on the ninit best individuals
“ λ−n ”
3 for nic := 1 to n
init do
b
4 ˆ
approximate f and rank the µ best individuals
5 if (the exact ranking of the µ best individuals changes) then
6 evaluate f on the nb best unevaluated individuals
7 else
8 break
9 ﬁ
10 od
11 adapt ninit depending on nic


A New Meta-Model Acceptance Criterion

Requiring the preservation of the exact ranking of the µ best
individuals is a too conservative criterion to measure the quality of
the meta-model.

New acceptance criteria (nlmm-CMA)
The meta-model is accepted if it succeeds in keeping:
the best individual and the ensemble of the µ best individuals
unchanged
or
the best individual unchanged, if more than one fourth of the
population is evaluated.



Success Performance (SP1):
mean (number of function evaluations for successful runs)
SP1 = ratio of successful runs .

SP1(algo)
Speedup (algo) = SP1(CMA−ES) .

8

6
Speedup

4

2

0
(Dimension, Population Size)



nlmm-CMA lmm-CMA
fSchwefel fSchwefel1/4 fNoisySphere
8 8 8

6 6 6
Speedup

Speedup

Speedup
4 4 4

2 2 2

0 0 0
(2, 6) (4, 8) (8, 10) (16, 12) (2, 6) (4, 8) (5, 8) (8, 10) (2, 6) (4, 8) (8, 10) (16, 12)
(Dimension, Population Size) (Dimension, Population Size) (Dimension, Population Size)

fRosenbrock fAckley fRastrigin
8 8 8

6 6 6
Speedup

Speedup

Speedup
4 4 4

2 2 2

0 0 0
(2, 6) (4, 8) (5, 8) (8, 10) (2, 5) (5, 7) (10, 10) (2, 50) (5, 140)

⇒ nlmm-CMA outperforms lmm-CMA, on the test functions investigated
with a speedup between 1.5 and 7.

nlmm-CMA Performance for Increasing Population
Sizes

nlmm-CMA lmm-CMA
Dimension n = 5

fSchwefel1/4 fRosenbrock fRastrigin
5 5 5

4 4 4
Speedup

Speedup

Speedup
3 3 3

2 2 2

1 1 1

0 0 0
8 16 24 32 48 96 8 16 24 32 48 96 70 140 280
Population Size Population Size Population Size

⇒ nlmm-CMA maintains a signiﬁcant speedup,between 2.5 and 4, when
increasing λ while the speedup of lmm-CMA drops to one.


Impact of the Recombination Type

nlmm-CMA
a default weighted recombination type
ln(µ+1)−ln(i)
ωi = µ ln(µ+1)−ln(µ!) , for i = 1 . . . µ.

nlmm-CMAI
an intermediate recombination type
1
ωi = µ , for i = 1 . . . µ.


Impact of the Recombination Type (Cont’d)

nlmm-CMA nlmm-CMAI (with equal RT)
fSchwefel fSchwefel1/4 fNoisySphere
8 8 8

6 6 6
Speedup

Speedup

Speedup
4 4 4

2 2 2

0 0 0
(2, 6) (4, 8) (8, 10) (16, 12) (2, 6) (4, 8) (8, 10) (2, 6) (4, 8) (8, 10) (16, 12)

fRosenbrock fAckley fRastrigin
8 8 8

6 6 6
Speedup

Speedup

Speedup
4 4 4

2 2 2

0 0 0
(2, 6) (4, 8) (8, 10) (2, 5) (5, 7) (10, 10) (2, 50) (5, 140)

⇒ nlmm-CMA outperforms nlmm-CMAI .
⇒ The ranking obtained with the new acceptance criterion still has an amount
of information to guide CMA-ES.

Summary

CMA-ES with meta-models
The speedup of lmm-CMA with respect to CMA-ES drops to one when the population
size λ increases.
⇒ The meta-model acceptance criterion is too conservative.

New variant of CMA-ES with meta-models
A new meta-model acceptance criterion: It must keep:
the best individual and the ensemble of the µ best individuals unchanged
the best individual unchanged, if more than one fourth of the population
is evaluated.
nlmm-CMA outperforms lmm-CMA on the test functions investigated with a
speedup in between 1.5 and 7.

nlmm-CMA maintains a signiﬁcant speedup, between 2.5 and 4, when
increasing the population size on tested functions.


Thank You For Your Attention


CMA-ES with local meta-models

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to CMA-ES with local meta-models

Similar to CMA-ES with local meta-models (20)

Recently uploaded

Recently uploaded (20)

CMA-ES with local meta-models