2. Training Evaluation
“Any attempt to obtain information (feedback) on the
effects of training program and to assess the value of
training in the light of that information for improving further
training” – By Hamblin (1970)
3. Need for Evaluation ?
• Cost Benefit Returns from Training Investment
(Major stakeholders like Training Department,
Senior Manager, Sponsors, Clients, Trainees)
• Enable Improvements in the assessment of Training
Needs (check suitability & feasibility of the
objectives set for training)
• Self-correcting feedback (evaluation will help
decide area of improvement)
• Feedback on the performance of the Trainees (aims
at improving knowledge & attitude of the trainee)
4. Trading Cycle
Identification of Training Needs
Develop Training Objectives
Measure
Training
Results
Design Training
Curriculum
Design/Select Training
Methods
Conduct Training
Programme
Design Training
Methodology
5. Models of Evaluation
• Hamblin Model of Evaluation
• Kirkpatrick’s Design of Evaluation
• Warr’s Framework of Evaluation
• Virmani & Premil’a Model of Evaluation
• Peter Bramely’s Model of Evaluation
6. Hamblin Model of Evaluation
• We can evaluate at any of the 4 levels namely:
Reaction, Learning, Job Behavior & Functioning
• Objectives of the training should be set at each of
the 4 levels
• According to Hamblin, to be effective, any scheme
of evaluation has to be tailor made to fit the needs
of the users of training
• These approaches are possible for external training
as well as in company training
7. Approaches to Evaluating
External & In–Company Training
In Company Training
Ultimate Value
Approach
Trainee Centered
Approach
Cost Benefit
Criteria
Organization
Development
criteria
Job Behavior
Criteria
External Training
Client Centered
Approach
Trainee Centered A
Learning
Criteria
Trainee Centered
Approach
Reach criteria
Transfer
Criteria
Trainee Centered A
9. Warr’s Framework of Evaluation
• Context Evaluation (C) : Obtaining & using information
about the current operational context, that is, about
individual differences & organisational deficiencies
• Input Evaluation (I) : determining & using facts & opinion
about the available human & material training resources
in order to choose between alternative training methods
• Process Evaluation (P) : monitoring the training as it is in
progress. This involves continuous examination of
administrative arrangements & feedback from trainees
• Outcome Evaluation (O) : Measuring the consequences
of training
10. Warr’s Framework of Evaluation
contd
3 levels of outcome evaluation has been distinguished
1. Immediate Outcome : the changes in trainees
knowledge, skills & attitudes which can be identified
immediately after the completion of training
2. Intermediate Outcome : the changes in trainee’s actual
work behavior which result from training assessment,
involves monitoring performance on the job
3. Long Term Outcome : the changes in the functioning of
part or all of the organization which have resulted from
changes in the work behavior initiated through training
11. Vermani & Premila’s Model of
Evaluation
3 STAGES
1. Pre-Training Evaluation (Period before training
during which trainees have expectations from the
course)
2. Context & Input Evaluation ( Teaching & Learning
stage)
3. Post-Training Evaluation (the stage after training,
where trainee’s are supposed to integrate training
with their job performance
12. Peter Bramely’s Model of
Evaluation
Evaluation Before Designing Learning Event
• If the intention is to improve performance by
changing behavior, some evaluation should be
carried out before a learning activity is designed &
run
• Knowledge & theory may be necessary but, if
behavioral change is expected, some practice of
the type of behavior expected will be required
13. Peter Bramely’s Model of
Evaluation contd
Evaluation During the Event
• Evaluation during a learning activity can be a useful
source of information with which we can improve
the quality of the event
• A focus on the utility of the learning & action
planning for its application in the workplace will
reinforce the link between objectives of the event &
organizational requirements
14. Peter Bramely’s Model of
Evaluation contd
Evaluation After the Event
• Most difficult stage in the art of evaluation
• After the event, analysis would measure changes in
effectiveness at the
1. Organizational Level
2. Team Level
3. Individual Level
• Change in Behavior (to assess whether change has
taken place)
• Change in Learning
15. David Reay’s Approach to
Evaluation
STAGE 1 – THE TRAIL PHASE
• Early Tentative phase of development.
1. Development Stage – involves an informal trail of a
sample set of materials, exercises or activities
2. Pilot Testing - more refined operation & tells the
evaluation whether the training being evaluated is
suitable for the target population
16. David Reay’s Approach to
Evaluation contd
STAGE 2 – THE ONGOING PHASE
1. Validation is used to measure the effectiveness of a
whole training design
1. Formative evaluation is the monitoring of the
effectiveness of different aspects of training on a
continuous basis with a view to modification
17. David Reay’s Approach to
Evaluation contd
STAGE 3 – THE FINAL PHASE
• Seeks answers to the following questions
1. Were the training needs properly identified
2. Were the learning objectives relevant
3. Were the performance standards identified
4. Were the right priorities established
5. Did the training meet the objectives set for it
6. How did the learners feel about the training
7. Were people prepared for training
8. Was the training pitched at the right level
9. Was the trainee able to transfer the training to do the
job
10. Did the training result in some benefit to the
organization
18. Pre Training Evaluation
•
•
•
•
•
•
IDENTIFICATION OF TRAINING NEEDS
Trainer carries out Training needs Survey
Uses semi structured interviews
Constructs & administers diagnostic questionnaires
EVALUATE PERFORMANCE STANDARDS
Directly linked to training needs
Performance standards identified as KRA’s focus on
Productivity, profit, Achieving deadlines, Machine
Down time, coping with external pressure etc
Method of evaluation would include detailed
description of each behavior & method of
measuring quality of behavior
19. Pre Training Evaluation contd.
EVALUATE TRAINING OBJECTIVES
• To assess whether there is a goal congruence
between the trainee’s needs & training objectives
EVALUATE TRAINEE’S PROFILE
• Trainee’s level of knowledge, attitude & skill prior &
post training
• Not all but some measures of pre training
knowledge, skill & attitudes is desirable to compare
them with post training performance
20. Pre Training Evaluation contd.
•
•
•
•
INPUT EVALUATION
Committee approach, brainstorming session would
help in situation where feedback on the subject is
already available
Helps to establish the validity of the training design
EVALUATION DURING THE TRAINING
Both evaluation & subsequent modification need
careful handling
Some methods are examined below
21. Pre Training Evaluation contd.
1. Observation- trainer’s perceptiveness &
observation skills cannot be discounted
2. Behavior Analysis- relevant for imbibing certain
skills & attitudes such as team orientation & attitudes
towards learners
3. Course Audits- can be conducted at the end of the
day or midway through the course
4. Session Assessment- done by using semantic
differential scale, 3 point or 5 point & trainee is
supposed to only tick mark
22. Post Training Evaluation
REACTION EVALUATION
• Useful for restructuring training programmes, so
long as it is not confused with an evaluation of
learning/job behavior
• Learning’s divided into 3 sections
1. Knowledge learning
2. Skill learning
3. Attitude learning
23. Evaluating training staff
performance
•
•
•
•
EVALUATING THE INDIVIDUAL TRAINER
When responsibilities are grouped under a common
accountability, measurement of performance
against accountability becomes the first means of
evaluation
Impact of trainers activities on solving human
performance provides means of evaluation
Elements of the reputation of the trainer includes
comments by line manager about the trainer or the
results of training
External reputation is a measure of trainer
performance
24. Evaluating training staff
performance
1.
2.
3.
4.
EVALUATING THE TOTAL STAFF
Accomplishment of departmental objectives
Economic accomplishment of the department.
Training should not cost an organization anything.
Results of its activities should be worth more than
the cost of staffing & running it
How effectively the department uses its resources
is a third basis of evaluation. One method of
measuring this is to have a system for establishing
priorities of departmental activities
The department’s reputation & the budget request
trend can be evaluated
25. Evaluation of the effectiveness
of Training
•Modify training to meet learner’s needs by altering
exercises, changing training methods & reorganizing
the sequence of topics
•Document any changes you make & explain these
changes to the instructional designers
•Use the end of course evaluation to generate ideas
for making the training better
•Keep in mind that most forms of training evaluations
are subjective. Hence, it will be important for you to
look at the overall picture & watch trends rather than
get bogged down with feedback from one source
26. Evaluation Reporting Strategy
•It helps to justify the existence of the training
department, showing the unique value added by the
training function to the organization
•Second reason is to make a “go” or “no go” decision
regarding continuation of a training programme in the
future
•To gather information to make programme
improvements
•To meet the unique needs of different audiences
27. Format of the Evaluation Report
If in written form it should be organized as follows
1. Cover Page: this provides the title of the report, the
date, the company name, and your contact information
2. Table of Contents: this provides title of all section & sub
section, with page numbers
3. Executive Summary: this is a one to two page summary
of the report that is aimed at busy senior managers who
want to skim a short summary & get all the key facts
4. Evaluation Report: provides specific & detailed
information about the training & the learner’s
performance. It contains a brief background description
of the training, purpose of the evaluation, key question
answered by the study, any assumptions/limitations on
the study, methods used to carry out the evaluation &
the results
5. Conclusion & Recommendation: draws conclusion from
evaluation & offers recommendation for improving
training
28. Contents of the Evaluation
Report
Be sure to include in any report such information as:
•The needs that the training was intended to meet
•The training objectives for the course
•Results of end of course reaction surveys, if they
were administered
•Results of tests taken by learner’s in aggregate or
individual form, if tests were administered
•Results of on the job follow up studies to assess
transfer of training to the job, if they have been
conducted
•Impact of training outcome study results, if they have
been conducted
29. Evaluating attitude change
•Most difficult evaluating process of all in the review of
training
•Problem lies in the ability to quantify changes before &
after training
•It is essential to have evidence of changes in approach,
style & adaptability as a result of the learning process
•Changes in the affective area of behavior are concerned
with experiences of individual persons learning to act in
their roles in the concern
•A caring company that trains its staff notices the outward
& visible signs & is not afraid to comment on nonquantifiable aspects of performance such as motivation,
attitudes to work, team building & morale as aresult of its
commitment to training
30. Training Evaluation Challenges
•During observation information can be difficult to
interpret
•During surveys & questionnaires they are more
impersonal for participants, & usually not good for
getting quotes in participant’s own words
