This is one approach in calculating the return of investment in training activities.
The nets benefits are calculated based on before and after training performance.
Calculating the costs of training is a matter of accumulating costs incurred on the relevant training activities.
2. Why ROI
Identify a feasible ROI Process
Use ROI to measure the effectiveness of
training
2
3. Training budgets are increasing
ROI is the ultimate level of evaluation
Competitive pressures on costs and
productivity
Top executives are requiring ROI information
3
4. Commitment of Training Expenditure as a
Percentage of Payroll:
◦ US – 1% - 4% of Payroll
◦ Europe – 2.5% to 3% of Payroll
◦ Asia – 4% to 8% of Payroll
4
5. Importance : 5 – Critically important and 1 is Unimportant Level of Level of
Agreement Importance
Agreement with Trend: 5 – Very much progress and
1 – No progress
1. Training Costs are monitored more accurately to manage 3.92 4.83
resources and demonstrate accountability
2. Measuring the return on investment in training is growing in 4.02 4.71
use
3. Systematic evaluation processes measure the success of 4.57 4.69
training
4. Needs assessment and analysis is receiving more emphasis 3.76 4.64
5. Training staff and line management are forming partnerships 3.95 4.57
to achieve common goals
6. Training is linked to strategic direction of the organisation 3.96 4.48
7. The learning organisation concept is being adopted. 4.09 4.47
8. Training delivery is changing rapidly 4.26 4.39
9. Training is shifting to a performance improvement role 4.25 4.37
10. The technology of training is developing rapidly. 4.68 4.32
5
6. ROI measures the contribution of a program/
solution designed in relation to the costs or
capital involved
ROI = Value of Benefits – Cost of Training
Cost of Training
6
7. Balanced Scorecard (Drs Kaplan and
Norton),
Kirkpatrick Four-Level Framework
(Kirkpatrick, 1975),
Jack Phillips’s Five Levels of Evaluation
7
8. Balanced Scorecard (BSC) is a management
framework used for implementing
organisational strategy by linking the objectives,
initiatives and measures of a business.
It integrates traditional financial measures with
other key performance indicators including
customer perspectives, internal business
processes and organisational development and
innovation.
8
9. Level 1 Reaction Evaluation
Level 2 Learning Evaluation
Level 3 Transfer of Learning
Evaluation
Level 4 Results Evaluation
This model is also adopted by Singapore Skills
Development Fund (SDF) for companies’ Training
Effectiveness Reports to SDF for funding
purposes.
9
10. Level Measurement Focus
1. Reaction & Measures participant
satisfaction with
Planned Action the program and captures planned
actions.
2. Learning Measures changes in knowledge,
skills and attitudes.
3. Application Measures changes in on-
the-job
behavior.
4. Business Impact Measures changes in business
impact
variables – any measurable
results?
5. Return on Compares program benefits
to the costs
Investment 10
11. Singapore Airlines
Singapore Technology
Motorola
Compaq
Companies who submit to SDF for funding
11
12. Improve program/Process
Discontinue/Expand Programs
Approve projects (If Pilot)
Rank projects
Like a front end analysis – to decide whether
worthwhile to proceed on the training project.
Develop Data Base of Program Results
Inform/Educate Management
Inform/Educate target Groups
Build Skills with Staff
12
13. Evaluation
Purposes Tabulating
Evaluation
Program
Instruments
Costs
Converting
Collecting Isolating the Data to Calculating
Post Program Effects of Monetary the Return on
Data the Program Value Investment
Significant Influences
Evaluating Policy Statement Identifying
Timing Procedures & Guidelines Intangible
Staff Skills Benefits
Management Support
Evaluating Technical Support
Levels Organisational Culture
13
14. Interactive Selling Skills – 3 days course, 48
participants
Retail Merchandise Company
You may want to pay attention – you will be
tested on your understanding! – Level 2
Evaluation
14
15. Surveys,
Performance records
If management don’t
Evaluation believe in it – don’t do it
Instruments Evaluation
Purposes Tabulating
Program
Costs
*Performance Monitoring
*Questionnaire Increased Sales/Profits
*Follow-up Session
Converting
Collecting Isolating the Data to Calculating
Post Program Effects of Monetary the Return on
Data the Program Value Investment
Control Groups
Significant Influences
Evaluating Policy Statement Identifying
Timing Procedures & Guidelines Intangible
Staff Skills Benefits
Questionnaire Management Support
-3 mths later Evaluating Technical Support
Levels Organisational Culture
15
16. Selected Data:
◦ Success with Objectives 4.3
◦ Relevance of Material 4.4
◦ Usefulness of Program 4.5
◦ Exercises/Skill Practices 3.9
◦ Overall Instructor Skill 4.1
16
17. All Participants Demonstrated That They
Could Use The Skills Successfully
17
18. Selected Data:
Strongly Agree Neither Disagree Strongly
Agree Agree/ Disagree
Disagree
I utilise the 78% 22% 0% 0% 0%
Skills taught
In the program
With Each Every Several At Least At Least
customer 3rd Times Once Daily Once
customer each day Weekly
Frequency of
Use of skills 52% 26% 18% 4% 0%
18
19. Post Training Data
Weeks After Training Training Groups Control
Groups
1 $9,723 $9,698
2 $9,978 $9,720
3 $10,424 $9,812
13 $13,690 $11,572
14 $11,491 $9,683
15 $11,044 $10,092
Average for Weeks $12,075 $10,449
13, 14, 15
19
20. 46 participants were still in job after 3 months
Ave Wkly Sales
◦ Trained Groups $12,075
◦ Untrained Groups $10,449
Increase $1,626
Profit Contribution 2% $32.50
Total Weekly Improvement (x46) $1,495
Total Annual benefits (x48wks) $71,760
20
21. 48 participants in 3 courses
Facilitation Fees: 3 courses @ $3750 $11,250
Program Materials: 48 @ $35/ppt $1,680
Meals/Refreshments: 3 days @$28/ppt $4,032
Facilities: 9 days @ $120 $1,080
Participants Salaries Plus Benefits’ (35%) $12,442
Coordination/Evaluation $2,500
Total Costs $32,984
21
22. ROI =(Value of benefits –cost of training)/Cost of
training
Benefit Cost Ratio 71,760 = 2.2:1
32,984
ROI (%) = 38,776 = 118%
32,984
22
23. Evaluation
Purposes Tabulating
Evaluation
Program
Instruments
Costs
Converting
Collecting Isolating the Data to Calculating
Post Program Effects of Monetary the Return on
Data the Program Value Investment
Evaluating Significant Influences Identifying
Timing Policy Statement Intangible
Procedures & Guidelines Benefits
Evaluating Staff Skills
Levels Management Support
Technical Support
Organisational Culture
23
24. Time spent on ROI
Cost of conducting the measurement
Complexity of variables in ROI
Accuracy in measurements
Credibility
Lack of Skills to measure
If staff does not see the need for ROI, it will
usually fail
Without support from management, ROI process
will usually fail.
24
25. No need to evaluate all courses.
Egs of courses to be evaluated:
Level % Courses
4. Participant satisfaction 100%
5. Learning 70%
6. On-the-job Applications 50%
7. Results 10%
8. Return on Investment 5%
25
26. • Coca Cola – 8 half-day on supervisory skills
workshops – 1447% ROI, Benefit/Cost Ratio 15:1
• Yellow Freight System – Performance Appraisal
Course – 1115% ROI, Benefit/Coast ratio 12:1
• Litton Industries (Avionics) – Self Directed Work
Team course – Productivity increased 30%, Scrap
rate reduction 50%, 700% ROI
• Multi-Marques, Inc (Bakery) – 15 hr Supervisory
Skills Training – 215% ROI, Benefit/Cost Ratio 3.2:1
26
28. Management Budget Cuts
Wants to see (No Results) ..And the
results Cycle Continues
Budget
Levels
Off
Minimum
Budget Training
Renewed increase Level
interest Renewed
Interest in
Training
28
29. A Training Executive met up with an Instructional Designer to
discuss about the design of an e-learning programme for
5000 participants worldwide. This is the first time that the
company would be using e-learning. The initial investment in
the training is huge. The CEO is very excited about the
programme and has high expectations of this training. The
Training Executive suggested to forecast/measure the ROI.
As an instructional designer, would you consider the
suggestion by the Training Executive? Why?
As the Training Executive, what would you like to achieve
from measuring ROI?
29
30. Cost of Technology - Initial cost of
implementing e-learning
Newness of e-learning (is it effective &
efficient as F2F learning?)
E-learning is not a proven process in many
organisation – need to show value now than
later
30
31. In many e-learning projects, the client wants to
know the projected payback from the project.
To venture into the expensive development
process without having some sense of the
payback is undesired by many clients.
Consequently, there is tremendous pressure to
forecast ROI even if it is not very accurate.
31
32. A major consulting firm
◦ CBT course for 7000 consultants in 50
countries. Cost of CBT training program =
$106/student. Previous F2F program cost is
$760/student. Life of program was 5 years &
savings = $4.5m
◦ What does this means to U?
When requesting monetary investment for e-learning,
be prepared to evaluate effectiveness
Conduct cost-benefits analysis to assess cost savings
for organisation
32
33. Based on interviews with 300 managers in
UK
◦ Managers discouraged staff from CBT
◦ Managers recognised potential benefits of CBT
◦ 44% were unsure where CBT was less expensive
than classroom training
◦ 66% felt CBT would isolate staff
◦ What does this means to U?
Evaluate e-learning to get management support
Determine whether e-learning meet organisational efforts
& the identified needs
33
34. Based on Yr 2000 ASTD State if the Industry
Report (USA)
◦ Projections for 2001 – significant majority of
organisations expect to be using multimedia
(91%), CD-ROM (87%), CBT (81%) and intranets
(77%)
◦ What does this mean to U?
ROI will provide data to help you decide to continue to
stop existing training programmes
ROI will help you to obtain support from the “bean
counters”
34
35. Isolate the effects by getting before and after
training performance data.
Performance data of with and without training
groups
35
36. Identify key output for the job to be analyzed
Identify the input variable for the output with
cause effect relationship
Compute the KPI by dividing output with the
input. It is a ratio.
Obtain before and targeted KPIs and insert
the actual KPIs in the diagram.
Separate the value of the output based on
volume and efficiency.
36
37. Example of input output
dimension for Sales
• Output is net profits in a week RM20,000
• Input is weekly sales figure RM100,000
• The KPI is 0.20 profit per ringgit sales or return
on sales
• Insert the previous performance KPI which
may be 0.15 profit per ringgit sales
• The performance gap between the old and the
new position.
37
38. After Profit 0.20 x 100,000 = 20,000
Before Profit 0.15 x 100,000 = 15,000
Performance Improvement = 5,000
Efficiency improvement
Volume improvement
38
40. Increase production units
Increase value of sales
Cost savings
◦ Less wastage
◦ Less time
◦ Less labour
◦ Less materials
◦ Less overheads
40
41. Costs of Trainers
Opportunity costs of trainees
Designing and Planning costs of training
program
Administration cost of training
softwares
41
42. Calculate the total cost of training or
intervention
Identify the input and output variable that
determine performance.
The relationship must be meaningful which
has high cause effect relationship
Isolate the impact of training with the
following test, with and without, before and
after.
42