2. About me
• Education
• NCU (MIS)、NCCU (CS)
• Work Experience
• Telecom big data Innovation
• AI projects
• Retail marketing technology
• User Group
• TW Spark User Group
• TW Hadoop User Group
• Taiwan Data Engineer Association Director
• Research
• Big Data/ ML/ AIOT/ AI Columnist
2
「How can you not get romantic about baseball ? 」
6. Steps of the Decision-Making Process
• 1. Identify the decision
• 2. Gather relevant information
• 3. Identify the alternatives
• 4. Weight the evidences
• 5. Choose among the alternatives
• 6. Take action
• 7. Review your decision
Quiz: What is the goals defining process and why is it so important?
6
7. IDENTIFY the decision
• To make a decision, you must first identify the problem you need to
solve or the question you need to answer.
• If you need to achieve a specific goal from your decision, make it
measurable and timely so you know for certain that you met the goal
at the end of the process.
7
8. Gather relevant information
• Do an INTERNAL assessment, seeing
where your organization has succeeded
and failed in areas related to your decision.
• Seek information from EXTERNAL sources,
including studies, market research, and, in
some cases, evaluation from paid
consultants.
• You may easily become bogged down by
too much information—facts and statistics
that seem applicable to your situation
might only complicate the process.
8
9. IDENTIFY the alternatives
• There is usually more than one option to consider when trying to
MEET A GOAL.
• For example, if your company is trying to gain more engagement on
social media, your alternatives could include paid social
advertisements, a change in your organic social media strategy, or a
combination of the two.
9
10. WEIGHT the evidences
• What companies have done
in the past to succeed in
these areas, and take a good
hard look at your own
organization’s wins and
losses.
• Identify potential PITFALLS
for each of your alternatives,
and weigh those against the
possible rewards.
10
11. CHOOSE among the alternatives
• You’ve identified and clarified what decision needs to be made.
• Gathered all relevant information, and developed and considered the
POTENTIAL PATHS to take.
• You are perfectly prepared to CHOOSE.
• Considering to AHP, Delphi method, Factor analysis, Literature Review,
Questionnaire.
11
https://blog.mesydel.com/what-is-the-delphi-method-and-
what-is-it-used-for-feb2d26f917a
12. TAKE action
• Once you’ve made your decision, act on it! Develop a plan to make
your decision tangible and achievable.
• Following to PMBOK, AGILE SCRUM MASTER.
https://www.agilealliance.org/
12
13. REVIEW your decision
• Take an honest look back at your
decision.
• Did you solve the problem?
• Did you answer the question?
• Did you meet your goals?
• If so, take NOTE of what worked
for future reference
• if not, learn from your MISTAKES
as you begin the decision and
making process again.
Solve the
question
Learn the
knowledge
G
o
o
d
13
15. Analytic Hierarchy Process Introduction
• Thomas L. Saaty was a distinguished University
Professor at the University of Pittsburgh.He
has made contributions in the fields of
operations research.
• He is the inventor, architect, and primary
theoretician of the Analytic Hierarchy Process
(AHP), a decision-making framework used for
large-scale, multi-party, multi-criteria decision
analysis.
http://www.rafikulislam.com/uploads/resourses/1972455
12559a37aadea6d.pdf
15
16. Analytic Hierarchy Process Introduction
• Procedure of AHP
• Model problem as a HIERACHY
DECOMPOSE the problem into from of
hierarchy.
• Evaluate hierarchy: PAIRWISE comparison
from point of view of importance to the
problem solution.
• Compute priorities: ESTABLISH weight
system.
Problem
Criteria A Criteria B Criteria C
Element 1
Element 2
…
Element 1
Element 2
…
Element 1
Element 2
…
Alternative 1 Alternative 2 Alternative 3
16
Homogeneous criteria are placed on the same level and are
independent of each other
Elements above is the criterion, and pairwise the importance of
the elements.
Literature Review, Questionnaire, Factor
analysis, Delphi
17. Analytic Hierarchy Process Introduction
• A multicriteria decision making
Price or Cost Storage Space Camera Quality Looks
Mobile1 $ 250 16 GB 12 MP 5
Mobile2 $ 200 16 GB 8 MP 3
Mobile3 $ 300 32 GB 16 MP 4
Mobile4 $ 270 32 GB 8 MP 4
Mobile5 $ 225 16 GB 16 MP 2
Alternative
Criteria
17
18. Analytic Hierarchy Process Introduction
• Step1 in AHP
• Developing a hierarchical structure with a GOAL at the top level, the criteria
at the second level and alternatives at the third level.
Buying a Mobile
Price or Cost Storage Space Camera Quality
Mobile 1 Mobile 2 Mobile 3 Mobile 4 Mobile 5
Level 1 (goal)
Level 2
(independent
criteria)
Level 3
Looks
(Thomas Saaty: no more than 7 criteria)
TOP
to
bottom
approach
AHP doesn’t consider the relation between criteria
18
19. Analytic Hierarchy Process Introduction
• Step2 in AHP
• Determine the relative importance of each criteria respect to the GOAL.
• Scale of relative importance.
• Pairwise comparison.
Scale of importance Description
1 Equal importance
3 Moderate importance
5 Strong importance
7 Very strong importance
9 Extreme importance
2, 4, 6, 8 Intermediate values
1/3, 1/5, 1/7, 1/9 Value for inverse comparison
19
20. Analytic Hierarchy Process Introduction
• Step3 in AHP
• Fill-in the data in the matrix.
• How importance is to 「Price or Cost 」with respect to 「 Storage Space」?
• Ex: Price or Cost is of a STRONG importance than Storage Space.
• Storage Space: x Value
• Price or Cost: 5x Value
Price of Cost Storage Space Camera Quality Looks
Price or Cost 1 5x/x = 5 4 7
Storage Space x/5x = 1/5 1 1/2 3
Camera Quality 1/4 2 1 3
Looks 1/7 1/3 1/3 1
Pairwise comparison of matrix
Row element
Column element
20
Reciprocal
21. Analytic Hierarchy Process Introduction
• More than one questionnaire inputs from experts, how to calculate
the value to fill-in the matrix?
• Use GEOMETRIC mean instead of arithmetic mean
Expert A: 1⁄3
Expert B: 3
Example of two experts’
inputs to the same criteria
21
Geometric:
Arithmetic:
22. Analytic Hierarchy Process Introduction
• Step4 in AHP
• Sum of columns and normalize them. (In ML we called feature scaling)
Price of Cost Storage Space Camera Quality Looks
Price or Cost 1/(1.59)=0.6289 5/(8.33)=0.6002 4/(5.83)=0.6861 7/14=0.5
Storage Space 1/5/(1.59)=0.1258 1/(8.33)=0.12 1/2/(5.83)=0.0858 3/14=0.2143
Camera Quality 1/4/(1.59)=0.1572 2/(8.33)=0.2401 1/(5.83)=0.1715 3/14=0.2143
Looks 1/7/(1.59)=0.0898 1/3/(8.33)=0.04 1/3/(5.83)=0.0572 1/14=0.0714
Sum 1.59 8.33 5.83 14
22
1.59 = 1+(1/5)+(1/4)+(1/7)
Normalization: Data will be rescaled so that the data will fall in the range of [0,1]
Standardization: Also called Z-score normalization, they’ll have the properties of a
standard normal distribution with mean(μ=0) and standard deviation(σ=1). This
scales the features in a way that they range between [-1,1]
23. Analytic Hierarchy Process Introduction
• Step5 in AHP
• Calculating Criteria weights in a new column
Price of Cost Storage Space Camera Quality Looks Criteria Weights
Price or Cost 0.6289 0.6002 0.6861 0.5 0.6038
Storage Space 0.1258 0.12 0.0858 0.2143 0.1365
Camera Quality 0.1572 0.2401 0.1715 0.2143 0.1958
Looks 0.0898 0.04 0.0572 0.0714 0.0646
0.6289+0.6002+0.6861+0.5
4 = 0.6038
23
Eigenvector
eig.ipynb
24. Analytic Hierarchy Process Introduction
• Step6 in AHP
• Calculating the consistency.
Criteria weights 0.6038 0.1365 0.1957 0.0646
Weighted
Sum Value
Price or Cost 1*0.6038=0.6038 5*0.1365=0.6825 4*0.1957=0.7832 7*0.0646=0.4522 2.5217
Storage Space 0.2*0.6038=0.1208 1*0.1365=0.1365 0.5*0.1957=0.0979 3*0.0646=0.1938 0.549
Camera Quality 0.25*0.6038=0.151 2*0.1365=0.273 1*0.1957=0.1958 3*0.0646=0.1938 0.8136
Looks 0.140.6038=0.0863 0.33*0.1365=0.045 0.33*0.1957=0.0646 1*0.0646=0.0646 0.2616
0.6038+0.6825+0.7832+0.4522
4
= 2.5217
A * Eigenvector
24
25. Analytic Hierarchy Process Introduction
• Step7 in AHP
• Weighted Sum Value & Criteria weights
Price of Cost Storage Space Camera Quality Looks
Weighted
Sum Value
Criteria
weights
Average λ
Price or Cost 0.6038 0.6825 0.7832 0.4522 2.5217 0.6038 2.5217/0.6038
=4.1762
Storage Space 0.1208 0.1365 0.0979 0.1938 0.549 0.1365 0.549/0.1365
=4.0225
Camera Quality 0.151 0.273 0.1958 0.1938 0.8136 0.1958 0.8136/0.1958
=4.1553
Looks 0.0863 0.045 0.0646 0.0646 0.2616 0.0646 0.2616/0.0646
=4.0488
λmax
4.1762+4.0225+4.1553+4.0488
4
= = 4.1007, Consistency Index (C.I.) =
λ max – n
n-1
4.1007 - 4
4-1
= = 0.03358
A * Eigenvector = λ * Eigenvector
A* Eigenvector =
(eigenvalue) 25
26. Analytic Hierarchy Process Introduction
• Step8 in AHP
• Calculating consistency ratio. What is consistency ratio?
• Consistency Ratio = (C.I.)/(Random Index)
• Threshold of 0.1 is a rule of thumb, 0 is the ideally
• If C.I. < 0.1, we can assume our metrics is reasonably
consistent, we may continue with the process of
decision making using AHP based on
criteria weights.
• If C.I. > 0.1, what can we do?
= 0.03358
0.9 = 0.037311
https://bpmsg.com/ahp-consistency-ratio-cr/
26
27. Analytic Hierarchy Process Introduction
• What is consistency ratio?
1, 2, 5
1/2, 1, 3
1/5, 1/3, 1
Consistency: to is 1 : 2 (A is important)
to is 1 : 5 (A is important)
to should be 2:5 = 2.5 (B is important)
A
A B C
B
C
A B
A C
B C
In principle, consistency ratio can’t vary to much,
the consistency is the way to evaluate
27
How to handling high consistency ratio?
https://bpmsg.com/ahp-high-consistency-ratio/
28. Analytic Hierarchy Process Introduction
• We found the importance is below
Attribute or Criteria Criteria weights Rank of importance (In priority)
Price or Cost 0.6038 1
Storage Space 0.1365 3
Camera Quality 0.1958 2
Looks 0.0646 4
ahp_practice.ipynb
28
29. Analytic Hierarchy Process Introduction
• Some more complex hierarchies.
Homework 1: Choosing an automobile
https://en.wikipedia.org/wiki/Analytic_hierarchy_process_%E2%80%93_car_example 29
33. Analytic Hierarchy Process Introduction
• Step4: Make a table and choose (budget is 25,000)
33
Alternatives
31090-20360=10730 31090/20360=1.53
25000-20360=4640
34. Analytic Hierarchy Process Introduction
• Step5: Make a table and choose (Derive preference based on
evaluation)
34
How can we get those information ?
Ans: Literature Review, Questionnaire, crawler social media…
The source helps us to fill-in the value of matrix
35. Analytic Hierarchy Process Introduction
• Step6: Make a table and choose
• Derive preference based on evaluation
35
How to evaluate the factor 「Passengers」
and 「 Capacity 」 relation?
36. Analytic Hierarchy Process Introduction
• Step7: Purchase price
• Step8: Fuel costs
Get Eigenvectors
Get Eigenvectors
36
37. Analytic Hierarchy Process Introduction
• Step9: Maintenance cost
• Step10: Resale value
Get Eigenvectors
Get Eigenvectors
37
38. Analytic Hierarchy Process Introduction
• Step11: Safety
• Step12: Style
Get Eigenvectors
Get Eigenvectors
38
39. Analytic Hierarchy Process Introduction
• Step13: Cargo Capacity
• Step14: Passenger Capacity
Get Eigenvectors
Get Eigenvectors
39
40. Analytic Hierarchy Process Introduction
• Each alternative has a priority corresponding to its fitness to all
the judgments table about all those aspects of Cost, Safety,
Style and Capacity.
• Here is a summary of the global priorities of the alternatives.
40
Judgments table
V
42. AHP application
• While the math can be done by hand or with
a calculator, it is far more common to use
one of several computerized methods for
entering and synthesizing the judgments.
• The simplest of these involve standard
spreadsheet software, while the most
complex use custom software, often
augmented by special devices for acquiring
the judgments of decision makers gathered
in a meeting room.
A typical device for entering
judgments in an AHP group decision
making session.
42
43. AHP application
• Grouping user by their behavior using AHP
• Assume we collected customer behavior data for a few days which is from
their internet behavior log to our online-shop.
• The goal is to fine out the preference of our customers, significant or
insignificant (we use more than one method to achieve it)
https://www.slideshare.net/orozcohsu/customer-behavior-analysis-240565709
43
45. Homework
• Homework 1: Study of complex hierarchy of choosing an automobile
• https://en.wikipedia.org/wiki/Analytic_hierarchy_process_%E2%80%93_car_
example
• Homework 2: What is eigenvalues & eigenvectors
• https://medium.com/sho-jp/linear-algebra-part-6-eigenvalues-and-
eigenvectors-35365dc4365a
• Homework 3: Study of Analytic Network Process, ANP)
• https://en.wikipedia.org/wiki/Analytic_network_process
45