Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Datamining 4th Adaboost

661 views

Published on

Published in: Education, Technology
  • Login to see the comments

Datamining 4th Adaboost

  1. 1. (AdaBoost) • Naive Bayes Yes, No ○ × • ○ × 1 0 • 3 • (Weighted Voting) • • • • • •
  2. 2. • 1 0 • N • (Xi , ci )(i = 1, . . . , N ) C XC = (No, Yes, Yes, Yes), cC = 1
  3. 3. • R: • • • 1 i wi • N wi = 1/N (i = 1, . . . , N ) 1 • 10 wi = 1/10 1
  4. 4. • t=1,...,R 1. : t i t pt i pt = wi i N t i=1 wi N • p1 = wi = 1/10 1 t=1 t wi =1 i i=1 2. WeakLearner WeakLearner ( t < 1/2 ) ht t N t = pt |ht (Xi ) − ci | < 1/2 i i=1 0, ht (Xi ) ci ht (Xi ) − ci = 1, Step 3
  5. 5. • t=1 WeakLearner ID A F h1(Xi) = 1 E,F h1(Xi)=ci D,J ci=0 ID G,H,I,J h1(Xi) = 0 WeakLearner 10 2 p1 = 1/10 i 1 = 1/10 × 2 = 1/5 < 1/2 T2 WeakLearner
  6. 6. t+1 3. wi βt t βt = 0≤ < 1/2, 0 ≤ βt < 1 1− t t • β 1−|ht (Xi )−ci | wi = wi βt t+1 t • εt βt • εt βt • WeakLearner ht
  7. 7. • t=1 1 = 0.2 1 β1 = = 0.2/0.8 = 0.25 1− 1 • A D, G J ht E, F 2 wA = wB = wC = wD = wG = wH = wI = wJ 2 2 2 2 2 2 2 = 1/10 × β1 = 0.025 1 2 wE = wF = 1/10 × β1 = 0.1 2 0
  8. 8. WeakLearner • t=1 WeakLearner T2 • WeakLearner • WeakLearner • WeakLearner • t=1 • T1 Yes ε1 ε1(T1=Yes)
  9. 9. p1 = 1/10(i ∈ {A, B, ..., J}) i 1 1 (T1 = Yes) = × 4 = 0.4 10 1 1 (T2 = Yes) = × 2 = 0.2 10 1 1 (T3 = Yes) = × 3 = 0.3 10 1 1 (T4 = Yes) = × 2 = 0.2 10 T2=Yes T4=Yes T2
  10. 10. • t=2 wA2 = wB = wC = wD = wG = wH = wI = wJ 2 2 2 2 2 2 2 = 1/10 × β1 = 0.025 1 2 wE = wF = 1/10 × β1 = 0.1 2 0 wi = 0.400 2 i∈{A,...,J} p2 = p2 A B = p2 = p2 = p2 = p2 = p2 = p2 C D G H I J = 0.025/0.400 = 0.0625 p2 = p2 E F = 0.1/0.400 = 0.25 • WeakLearner 2 (T1 = Yes) = 0.0625 × 2 + 0.25 × 2 = 0.625 2 (T1 = No) = 0.375 2 (T2 = Yes) = 0.25 + 0.25 = 0.5 2 (T3 = Yes) = 0.0625 × 2 + 0.25 × 1 = 0.3125 2 (T4 = Yes) = 0.0625 × 2 = 0.125 T4=Yes
  11. 11. β2 = 2 /(1 − 2) = 0.143
  12. 12. R 1 if t=1 (− log10 βt )ht (X) ≥ (− log10 βt ) 1 hf (X) = 2 0 othrewise t βt = 1− t • 1, 0 • Log • εt 0 βt -Log βt WeakLearner ht(X) • -Log βt ht(X)
  13. 13. β1 = 0.25, β2 = 0.143 2 1 (− log βt ) t=1 2 1 1 = − log10 0.25 × − log10 0.143 × 2 2 = 0.723 • 1.
  14. 14. • h1: T2=Yes C=1, h2: T4=Yes C=1 2 • K (− log βt )ht (XK ) t=1 = − log10 0.25 × 0 − log10 0.143 × 1 = 0.845 0.723 ○ 2 • L (− log βt )ht (XL ) t=1 = − log10 0.25 × 1 − log10 0.143 × 0 = 0.602 0.723 ×
  15. 15. AdaBoost • hf ε = D(i) {i|hf (Xi )=yi } • D(i) • R εt t ≤ 2 t (1 − t) t=1 • t εt<1/2 2 t (1 − t) < 1 • WeakLearner ε
  16. 16. • 1 • R R+1 N R 1/2 R+1 wi ≥ βt i=1 t=1 • • N R+1 R+1 wi ≥ wi i=1 {i|hf (Xi )=yi } t 1−|hf (Xi )−yi | t+1 wi = wi βi R 1−|hf (Xi )−yi | t+1 wi = D(i) βt {i|hf (Xi )=yi } {i|hf (Xi )=yi } t=1
  17. 17. R hf (hf (Xi ) = yi ) 1−|hf (Xi )−yi | βt hf (Xi ) = 1 yi = 0 hf (Xi ) = 0 t=1 2 hf 2 hf (Xi ) = 1 yi = 0 5.1 hf (Xi ) = 1 yi = 0, hf(Xi)=1 (hf (Xi ) = R i ) y R 1 Xi ) = 1 yi = 0 h(− log βt )hf (Xi ) y≥ = 1 (− log βt ) f (Xi ) = 0 i t=1 t=1 2 R 0 t=1 (log5.1 βt ) R R R 1 1 − log βt )hf (Xi ) ≥ (− log(log βt )(1 − hf (Xi )) ≥ βt ) (log βt ) 2 t=1 t=1 2 t=1 1 − hf (Xi ) = 1− | hf (Xi ) − yi | R R 1/2 R 1t f (Xi )−yi | ≥ 1−|h g βt )(1 − hf (Xi )) ≥ (log βt ) β βt t=1 2 t=1 t=1
  18. 18. t t t=1 t=1 hf (Xi )hf (Xi ) = 0 yi = i1= 1 =0 y 5.1 R R 1 (− log βt )ht (Xi ) < (− log βt ) t=1 t=1 2 174 −1 hf (Xi ) = 1− | ht (Xi ) 5 yi | − 1 R R 2 1−|ht (Xi )−yi | βt > βt t=1 t=1 hf (Xi ) = 1 yi = 0 hf (Xi ) = yi = 1 1 R R 2 1−|ht (Xi )−yi | βt ≥ βt t=1 t=1
  19. 19. 5.2  1  R R 2 1−|hf (Xi )−yi | D(i)  D(i) βt ≥ βt {i|hf (Xi )=yi } t=1 {i|hf (Xi )=yi } t=1   1 R 2 = D(i) βt {i|hf (Xi )=yi } t=1   1 R 2 R+1 wi ≥ D(i) βt {i|hf (Xi )=yi } {i|hf (Xi )=yi } t=1 1 R 2 = · βt t=1 5.2
  20. 20. = · t=1 βt t=1 • 2 5.2 • 5.2 N N t+1 wi N ≥ t wi N× 2 i i=1 t+1 wi i=1≥ wi × 2 t i • : α≥0 r = {0, 1} i=1 i=1 : α≥0 r = {0, 1} αr ≤ 1 − (1 − α)r αr ≤ 1 − (1 − α)r
  21. 21. 5.6. 175 N N 1−|ht (Xi )−yi | t+1 wi = t wi βt i=1 i=1 N ≤ wi (1 − (1 − βt )(1 − |ht (Xi ) − yi |)) t i=1 N N N = wi − (1 − βt ) t t wi − wi |ht (Xi ) − yi | t i=1 i=1 i=1 N N N = wi − (1 − βt ) t t wi − t t wi i=1 i=1 i=1 N N = wi − (1 − βt ) t t wi (1 − t ) i=1 i=1 N = t wi × (1 − (1 − βt )(1 − t )) i=1 βt = t /(1 − t) N = t wi ×2 t i=1
  22. 22. • 5.3 t WeakLearner • t εt t WeakLearner hf hf ε R ≤ 2 t (1 − t) • 1/2 t=1 R N N N R : 5.1 βt ≤ 5.2 wi R+1 ≤ R wi ×2 t ≤ 1 wi 2 t t=1 t=1 R 1/2 i=1 N i=1 i=1 N βt wi = 1 1 ≤ R+1 wi ( 5.1 ) t=1 i=1 i=1 R N = 2 ≤ t=1 t R wi × 2 t( 5.2 ) i=1 βt = t /(1 − t ) 5.2 t = R − 1, R − 2, . . . , 1 R R −1/2 N R ≤ 2 t× βt = 12 t (1 − t) ≤ wi 2 t t=1 t=1
  23. 23. • 3 • • • (Weighted Voting) • • WeakLearner WeakLearner NaiveBayes K-NN, SVM • Ensemble Learning
  24. 24. 4 5 2 A 9 3 1 C 1 8 6 7 B 2 4 5 2 A 4 4 5 2 A 8 6 7 B 3 8 6 7 B 8 6 5 9 3 1 C 9 7 9 3 1 C 8 6 7 B 4 5 2 A 9 3 1 C
  25. 25. 1 1 2 3 4 5 6 7 8 9 2 4 1 2 3 4 5 6 7 9 8 3 8 6 1 2 3 4 5 6 8 9 7 5 9 7 1 2 3 4 5 7 8 9 6 1 2 3 4 6 7 8 9 5 1 2 3 5 6 7 8 9 4 1 2 4 5 6 7 8 9 3 1 3 4 5 6 7 8 9 2 2 3 4 5 6 7 8 9 1

×