The ‘Measuring the Business Impact of Learning’ benchmarking survey, conducted by LEO Learning and Watershed (on behalf of Learning Technologies Group) is entering its fourth year. With the survey launching on November 1st and closing on December 13th, LEO Learning and Watershed are holding a webinar to reflect on the results so far, plus discuss how organizations they’re working with have overcome the barriers in measurement planning and implementation. The insights are drawn from their group experience working with a range of clients in this field and should be valuable for anyone who wants to get going in learning analytics and sustainable business impact assessment.
Join your hosts as they cover the state of the world of measurement, and you’ll receive:
An understanding of how well-known organizations have overcome the barriers in measurement planning and implementation.
Real-world examples of how to get management buy-in, designing for data, building data ecosystems, implementing a learning analytics strategy and more.
The opportunity to take this years ‘Measuring the Business Impact of Learning’ survey, and see the results coming in live!
MEASURING THE BUSINESS IMPACT OF LEARNING: WHAT WE’VE LEARNED
1. #CLOwebinar
The presentation will begin at the top of the hour.
A dial in number will not be provided.
Listen to today’s webinar using your computer’s
speakers or headphones.
Welcome to the webinar!
2. #CLOwebinar
Tools You Can Use
Audio Control
– A dial in number will not be provided.
– The audio will stream through your
headphones or computer speakers.
– Also check your computer’s volume for
external speakers or headsets.
4. #CLOwebinar
1. May I receive a copy of the slides?
YES! Click on the handouts located on the right portion of
your screen.
2. May I review the webinar recording at a later date?
YES! You may log in again using today’s link to review the
presentation on-demand.
3. Is this webinar HRCI or SHRM certified?
YES! The HRCI and SHRM certification codes will be sent in
the follow up email.
Frequently Asked Questions
5. Learning Design Director
LEO Learning
Measuring the business impact of
learning: What we’ve learned
Ella Richardson
Director of Strategy
Watershed Analytics
Tim Dickinson
Chief Strategy Officer
Learning Technologies Group
Piers Lea
15. But First: Remember measurement is ongoing & iterative
Watershed Method for Learning
Analytics
Categories of analytics
+
Complexities of analytics
16. Start with what you can control.
Barrier 1
I don’t know how to get started.
17. Have they
learned it?
What do people
need to learn?
Are people
doing that?
What do people
need to do?
Did they
complete it?
What training
is required?
Was the goal
achieved?
Business
goal
Learning Design
Measurement
Getting Started: Link design to measurement
18. How In Practice: Get your team to understand
Develop a bespoke
measurement model
and processes
Upskill the trainers
to operate as
performance
consultants
Put measurement
first in the design
and delivery
process
Building internal measurement capability
19. Evaluate
Analysis
Consult
Measurement
Process
Action
Design
DevelopImplement
Evaluate
Analyse
Consult
• Clarify the business goals
• Define the business objectives
• Create a business impact action plan
• Identify the business
data sources
• Write the learning
objectives
• Specify the learning outcomes
• Identify the learning data
sources
• Refine the measurement
action plan
• Identify whether the required data can
be captured
• Baseline the data
• Design how the data will be displayed
• Pilot the learning
• Gather the business and learning data
• Check the quality of the data and
move to full roll-out
• Explore the business and
learning data for insights
• Report back on initial findings
and improve training
• Develop a deep
understanding of the
business performance gaps
• Create a draft
improvement plan
• Modify programme /
manage learners as
required
A bespoke
measurement
process
20. Getting Started: Map out your ecosystem
Identify your
current tools and
resources
Question your
current ecosystem
Choose an area to
test your new
measurement
It’s easier than you think
1 2 3
27. Outcome
There was a 2% sales lift during
the four-week period when video and
assessment were used as planned.
28. Show them what it could look like.
Barrier 2
Competing Priorities: How to get budget and time?
29. Getting Buy-In: Define a strategy
Key components of a measurement strategy
No.1
Co-create
measurement
project
No.2
Establish the
business drivers
No.3
Map out
stakeholders
No.4
Document
chain of
evidence
No.5
Identify data
types and
source
No.6
Define, develop
tech: Integration
& tools
No.7
Pilot the
solution
No.8
Document
strategy and
business case
No.9
Define the
change
programme
No.10
Measure impact
of measurement
strategy
30. The chain of evidence
Engagement Learning Behaviour Results
34. Get aligned and don’t be afraid to ask for help.
Barrier 3
How do you get access to the data?
35. Getting Data: Vendor Alignment
Share the vision
and how your
vendor fits in
Build a partnership
based on
collaboration
Validate
interoperability
before taking out
the checkbook
37. Measurement – sources of evidence
Engagement
• Portal/learning visits and
behaviour data (which
courses, by whom, for
how long)
• Star rating
• User Surveys
• Learner voice
• User acceptance
testing
• Interviews
• Annual staff survey
• Individual 360
• Survey of departmental
satisfaction
K, S, A
• Upfront diagnostics
(including 360)
• Post intervention Online
assessments (immediate
& after 3 months)
• Marked work (essays,
reports)
• Peer assessment
• Demonstration
• Confidence measure
taken after every
intervention
Performance
Behaviours
• User surveys
• Manager surveys
• Interviews (in
manageable numbers)
• Surveys of coaches and
mentors
• User observation
• Service user feedback
(net promoter score)
• Activity monitoring
software
• Use of learning journals
Business Change
• Business Metrics & KPIs
• Profitability, Growth,
Productivity etc.
• Improved staff retention
• Faster turnaround times
• Money saved by new
approach
• Money saved by
outcomes
38. Getting Data: Ask for help
Build alliances
within the
organisation, early
Get the best team
round the table
39. Key Takeaways
Get Started
○ Design for data
○ Start small, test and learn
○ DIY: Learn about it yourself
Get Buy in
○ Define a measurement strategy
○ Map out your ecosystem/what’s available to you already
Get Data
○ Align vendors and tools for long term success
○ Ask for help