Machine Learning Concepts & Decision Tree Learning Overview
1. Machine Learning: Concept Learning & Decision-Tree Learning Yuval Shahar M.D., Ph.D. Medical Decision Support Systems
2.
3.
4. A Concept-Learning Example Enjoy? Fore-cast Water Wind Humid Air temp Sky # Yes Same Warm Strong Normal Warm Sun 1 Yes Same Warm Strong High Warm Sun 2 No Change Warm Strong High Cold Rain 3 Yes Change Cool Strong High Warm Sun 4
5.
6.
7.
8.
9.
10.
11.
12. Example Decision Tree Outlook? Humidity? Wind? Yes Yes Yes No No Sun Overcast Rain High Normal Strong Weak
13.
14.
15.
16.
17. Entropy Function for a Boolean Classification p 1.0 0.0 0.5 Entropy( S ) 1.0
18.
19.
20. Information Gain Example Humidity? Wind? {3+, 4-} E = 0.985 High Normal Strong Weak S: {9+,5-} E = 0.940 S: {9+,5-} E = 0.940 {6+, 1-} E = 0.592 {6+, 2-} E = 0.811 {3+, 3-} E = 1.0 Gain( S , Humidity ) = 0.940-(7/14)0.985-(7/14)0.592 = 0.151 Gain( S , Wind ) = 0.940-(8/14)0.811-(6/14)1.0 = 0.048
![Machine Learning ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)