Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kedar kulkarni ev10 nov09-pmi_india_aninchora_mile

898 views

Published on

Published in: Business
  • My brother found Custom Writing Service ⇒ www.WritePaper.info ⇐ and ordered a couple of works. Their customer service is outstanding, never left a query unanswered.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • You can try to use this service ⇒ www.WritePaper.info ⇐ I have used it several times in college and was absolutely satisfied with the result.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Your great buying advice and helpful tips really helped with me buy a cheap car at auction. ☀☀☀ https://w.url.cn/s/Aaxmqpl
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Kedar kulkarni ev10 nov09-pmi_india_aninchora_mile

  1. 1. An Inch or a Mile?Proven Practices in Measuring Learning ImpactNovember, 2010 <br />KedarKulkarni<br />
  2. 2. ESI & KA Strategic Partnership<br />© ESI International, Inc. 2010.<br />
  3. 3. ESI<br />© ESI International, Inc. 2010.<br />US Offices<br />Arlington/Washington D.C.<br />Chicago<br />New York<br />San Francisco<br />ESI Partners<br />Japan Poland <br />South Korea <br />Global Offices<br />Australia Canada China Germany Hong Kong India New Zealand Singapore South Africa Spain Sweden Taiwan United Arab Emirates United Kingdom <br />
  4. 4. KnowledgeAdvisors & Metrics that Matter®<br />© ESI International, Inc. 2010.<br />KnowledgeAdvisors is ahuman capital analytics firm.<br />We are the world’s largest provider of learning and talent measurement solutions. <br />Experts in metrics that matter<br />
  5. 5. Agenda<br />Current market trends<br />Top five challenges for measuring impact<br />Six Proven Practices in Measuring Learning Impact<br />A Note on Informal Learning and Measurement Sustainability<br />Project Management Training—Imagine the Impact to Your Projects (Case study)<br />© ESI International, Inc. 2010.<br />
  6. 6. Current Market Trends<br />More focus on demonstrating business outcomes<br />Open to collaborating with partners and suppliers to build and execute measurement plans<br />Focus on transparency and project reporting<br />More focus on how to establish “Current” State <br />Examples: Knowledge / Competencies / Organizational / Portfolio / Project Assessment Tool<br />© ESI International, Inc. 2010.<br />
  7. 7. Current Market Trends—The Undisputed Truth<br />© ESI International, Inc. 2010.<br />Kirkpatrick Four Levels Continuous Learning Process<br /><ul><li>Level 1: Reaction
  8. 8. Level 2: Learning
  9. 9. Level 3: Transfer
  10. 10. Level 4: Results (ROI)</li></ul>Effective Assistance (Coaching/Counseling) by Manager<br />Manager<br />(Sender)<br />Manager<br />(Developer)<br />Assess<br />Learn<br />Reinforce/Validate<br />Pre-course<br />preparation<br />and assessment<br />Intensive learning<br />through the course<br />(Classroom/e-Training)<br />Post-course<br />implementation<br />and assessment<br />Learning and Development by Him/Herself<br />Person<br />(Trainee)<br />Person<br />(Developed)<br />
  11. 11. Scrap Learning/Manager Support<br />© ESI International, Inc. 2010.<br />Typical Training without Strong Manager Support<br />COSTS<br />? %<br />DID NOT TRY IT AT ALL<br />? %<br />TRIED IT TO SOME EXTENT BUT<br />GAVE UP <br />? %<br />TRIED IT AND GOT POSITIVE RESULTS<br />UNREALIZED VALUE<br />RETURN<br />The preferred method of optimizing training<br />COSTS<br />? %<br />DID NOT TRY IT AT ALL<br />? %<br />TRIED IT TO SOME EXTENT BUT<br />GAVE UP <br />85%+<br />TRIED IT AND GOT POSITIVE RESULTS<br />RETURN<br />Source: Robert Brinkerhoff<br />
  12. 12. Scrap Learning is a Real Problem<br />© ESI International, Inc. 2010.<br />60%of learning is not applied to the job….that’s scrap, but as managers YOU can make a difference!<br /><ul><li> How much learning is applied to the job?
  13. 13. When is learning applied to the job?
  14. 14. What are the barriers to impact?
  15. 15. What can be done to reduce scrap?</li></ul>Source: KnowledgeAdvisors <br />
  16. 16. Current Market Trends— Measuring Learning to Impact Project Performance<br />© ESI International, Inc. 2010.<br />
  17. 17. A New Paradigm<br />© ESI International, Inc. 2010.<br />Build on Existing Data Collection Points<br />ROI<br />VOI<br />
  18. 18. Determine where you need to go<br />But no matter where you want to go, your must….<br />© ESI International<br />
  19. 19. Add Value—<br />Understand the value <br />Identify value metrics <br />Establish a baseline for each metric—pragmatically<br />Identify leading indicators for each metric<br />Measure continuously and manage your stakeholders<br />© ESI International, Inc. 2010.<br />Influence vs. Interest <br />
  20. 20. Top Five Challenges<br />Lack of resources<br />No executive support<br />Lack of qualified people to determine and track metrics<br />Afraid of the results<br />The need has not been there <br />© ESI International, Inc. 2010.<br />www.esi-intl.com/valuesurvey<br />
  21. 21. Agenda<br />Current market trends<br />Top five challenges for measuring impact<br />Six Proven Practices in Measuring Learning Impact<br />A Note on Informal Learning and Measurement Sustainability<br />Project Management Training—Imagine the Impact to Your Projects (Case study)<br />© ESI International, Inc. 2010.<br />
  22. 22. Showing Business Impact<br />© ESI International, Inc. 2010.<br />Requires actual results data and advanced analysis techniques<br />High Investment<br />Once you determine Business Impact what analysis is appropriate, select a method:<br />Causal Modeling<br />Actual Results Correlations<br />Typically employs surveys<br />Business Impact Templates<br />Advanced Business Results Evaluation<br />Human Capital Approach<br />Smart Sheets<br />Low Investment<br />Complex<br />Simple<br />
  23. 23. Smile Sheets  Smart Sheets<br />© ESI International, Inc. 2010.<br />Level 1<br />REACTION<br />Level 2<br />Learning<br />Level 3<br />Job Impact<br />Level 4<br />Results<br />Level 4<br />RESULTS<br />MANAGEMENT SUPPORT<br />Level 5<br />ROI/ROE<br />Evidence of Success<br />
  24. 24. Making Decisions With Impact Data<br />© ESI International, Inc. 2010.<br />Cut or improve programs that aren’t providing impact to the business<br />“We need our measurement tools and process now more than ever; if we don’t have access to this data, how will we be able to make intelligent decisions on what to cut?”<br />- KnowledgeAdvisors Client<br />
  25. 25. Human Capital Approach<br />Uses improved job performance as the overall measure of business impact<br />Easy to use, practical, scalable, replicable<br />Provides a financial ROI relative to the improvement in human capital performance<br />© ESI International, Inc. 2010.<br />
  26. 26. Human Capital Approach (continued)<br />Uses principles of estimation, isolation, and adjustment<br />Assumption is that human capital’s value (i.e. salary) is like any other asset that has expected value that can appreciate or depreciate over time<br />© ESI International, Inc. 2010.<br />
  27. 27. Business Results Evaluation Approach<br />Use specific business results as a measure of business impact<br />Uses an advanced survey to estimate improvement in results<br />Leverage technology to make practical and scalable<br />Provides a financial ROI relative to the change in the business result that prompted the training<br />Uses principles of estimation, isolation, and adjustment specific to the business result<br />© ESI International, Inc. 2010.<br />
  28. 28. Business Results Evaluation Process<br />© ESI International, Inc. 2010.<br />
  29. 29. Case Example: Advanced Business Results<br />© ESI International, Inc. 2010.<br />Background: ERP company performing customer education<br />Business Issue: Show value of training to customers in terms of increased productivity and quality<br />Measurement Solution: Collected 12,770 end of program evaluations within 6 months asking participants to forecast impact<br />Analytics Results: 18.2% estimated increase in productivity from the training (adjusted down for self-reported bias)<br />
  30. 30. Business Impact Template<br />© ESI International, Inc. 2010.<br />Monetize Results <br />Estimate the Impact of Outside Factors<br />Compare to Cost of Training<br />
  31. 31. Use Business Impact Templates<br />© ESI International, Inc. 2010.<br />Background: Large CPG based in Canada,sales training<br />Business Issue: Visible sales program, requested to link between it and sales results<br />Analytics Solution: Used process to evaluate sales change and impact training had on sales<br />Analytics Results: 7% increase in sales, 20% of the increase was due to training, this was a $74,000 increase per sales person when the training was $5000 per person<br />
  32. 32. Use Actual Result Correlations<br />This is a method where actual results can be input or imported at an aggregate, periodic frequency. It is then correlated in time periods against training evaluation data for impact indicators.<br />The example below is from sales data and sale training.<br />© ESI International, Inc. 2010.<br />
  33. 33. Causal Modeling<br />High level statistical analysis using Structural Equation Modeling<br />Determines the key factors that influence business impact<br />Causation not correlation<br />The Causal Model can be used to predict future results and determine investments<br />© ESI International, Inc. 2010.<br />WorthwhileInvestment<br />0.483<br />0.625<br />Courseware Quality<br />IndividualLearning<br />0.337<br />0.556<br />0.083<br />InstructorEffectiveness<br />
  34. 34. Performance Optimization<br />© ESI International, Inc. 2010.<br />
  35. 35. Actionable Metrics<br />© ESI International, Inc. 2010.<br />Goals: Actual to Goal <br />Benchmarks: <br />Internal or External<br />Dashboards: Concise Data<br />Trends: Time based analysis<br />
  36. 36. IMAGINE…<br />© ESI International, Inc. 2010.<br />Online course end evaluations:<br />Level 1 & 2<br />Predictive Level 3<br />Predictive Business Impact<br />STANDARD REPORT<br />STANDARD REPORT<br />Online follow-up <br />60-90 days—with participant and with their manager about b & c above<br />
  37. 37. Agenda<br />Current market trends<br />Top five challenges for measuring impact<br />Six Proven Practices in Measuring Learning Impact<br />A Note on Informal Learning and Measurement Sustainability<br />Project Management Training—Imagine the Impact to Your Projects (Case study)<br />© ESI International, Inc. 2010.<br />
  38. 38. Informal Learning Measurement<br />© ESI International, Inc. 2010.<br /># of respondents<br />% of respondents supporting each type of informal learning<br />Source: KnowledgeAdvisors Informal Learning Study, 2009<br /><ul><li>Measure activity trends (ex. utilization of performance support tool)
  39. 39. Quick Poll evaluation at point of use (ex. Amazon-like star rating)
  40. 40. Comprehensive periodic evaluation to review quality and impact
  41. 41. In research, 100% who measured had higher future budget allocations</li></li></ul><li>Measurement Sustainability<br />© ESI International, Inc. 2010.<br />1<br />2<br /><ul><li>Roles and accountabilities for metrics are defined with consistent expectations on what should be measured.
  42. 42. Strong leadership for driving measurement with governance to support it. </li></ul>8<br /><ul><li>Culture supports and drives need for standards and common talent development processes and measurement</li></ul>3<br />Data Utilization<br />3<br /><ul><li>Broad base of skills in analytics and measurement and strong capabilities in end-to-end measurement methods</li></ul>7<br /><ul><li>Strong desire to use the data for decision making
  43. 43. Maturity in how data is reported and used to improve end to end talent processes</li></ul>Processes<br />Technology<br />6<br /><ul><li>Standardization for a consistent and single point of analytics with integration to and from feeder systems</li></ul>4<br /><ul><li>Established and consistent processes, technologies and communications for measurement</li></ul>Standards, Tools<br /><ul><li>Standards have been created for data collection
  44. 44. Data integrity exists </li></ul>5<br />
  45. 45. Agenda<br />Current market trends<br />Top five challenges for measuring impact<br />Six Proven Practices in Measuring Learning Impact<br />A Note on Informal Learning and Measurement Sustainability<br />Project Management Training—Imagine the Impact to Your Projects (Case study)<br />© ESI International, Inc. 2010.<br />
  46. 46. Project Management – Bringing Value to the Business<br />Effective and profitable management of complex projects<br />Global consistency and capability<br />Effective use of resources <br />Higher customer satisfaction<br />Increased employee satisfaction <br />Significant competitive advantage<br />© ESI International, Inc. 2010.<br />
  47. 47. What is Your Definition of Value<br />© ESI International, Inc. 2010.<br /><ul><li>Increased customer & staff satisfaction
  48. 48. Increased quality
  49. 49. Increased productivity
  50. 50. Increased revenue
  51. 51. Decreased costs / cycle times
  52. 52. Decrease risk</li></ul>Maximize return:<br /><ul><li>Improve business results;grow revenue and earnings, cash flow, and reduced cost of operations</li></ul>Increase agility:<br /><ul><li>Enable the business organization and operations to adapt to changing business needs</li></ul>Improve performance:<br /><ul><li>Improve business operations performance end-to-end across the enterprise
  53. 53. Increase customer and employee satisfaction</li></ul>Minimize risk:<br /><ul><li>Ensure continuity of internal business operations, while minimizing exposure to risk factors</li></li></ul><li>How does this impact Project Performance?<br />© ESI International, Inc. 2010.<br />
  54. 54. Think CSI<br />© ESI International, Inc. 2010.<br />“In the absence of proof, let evidence be enough.” <br />Donald Kirkpatrick<br />
  55. 55. Questions?Thank you!November, 2010<br />Raed S. Haddad, <br />Senior Vice President<br />ESI International<br />

×