Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Creating a Data Driven L&D Team - an xAPI Case Study - DevLearn 2018

From an organization-wide executive directive to become more data-driven, a retail corporate L&D team took an internal look at their own data practices. Realizing that they had an overwhelming lack of transparency into their learning initiatives and a great amount of data that had gone unused, the team developed a transformation vision to create a single system of record for learning to enable observability, granularity, and accountability for all team members. The team was committed to the vision of xAPI; however, the data and information they needed in order to make actionable change for their learners was locked away in non-interoperable formats, and they recognized the need to develop a data strategy and implementation plan.

In this case study session, you will see how the organization’s L&D team created an xAPI data roadmap to not only achieve early wins when it came to the executive business objectives but also begin work on a scalable plan to build out a modern, flexible ecosystem that has the needs of learners at its center.

* Originally presented on 10/25/18 at DevLearn by Allie Tscheulin and Margaret Roth

  • Login to see the comments

  • Be the first to like this

Creating a Data Driven L&D Team - an xAPI Case Study - DevLearn 2018

  1. 1. Creating a Data-Driven L&D Team +A RETAIL CO.
  2. 2. What does it mean to be data-driven? intro xapi next steps takeawaysdata assessment pilot
  3. 3. ● # of inbound leads ● # of website views ● Website bounce rates ● Cost per ad vs. # of qualified leads ● % conversion on a landing page ● MQL conversions ● Leads by platform ● Returning visitors ● ROI of marketing initiatives Marketing is Data-Driven intro xapi next steps takeawaysdata assessment pilot
  4. 4. ● # of deals in the pipeline ● % conversion rates ● # of leads worked ● New business vs existing business ● Meeting #s ● Lifetime value of a customer ● Cost of customer acquisition ● SQL conversions ● Customer retention rates Sales is Data-Driven intro xapi next steps takeawaysdata assessment pilot
  5. 5. ● Completion rates 3 months later ● # of people at in-person training. ● … ● … ● … ● … ● … ● ● We ran the training. That happened. Learning Has a Way to Go intro xapi next steps takeawaysdata assessment pilot
  6. 6. What Google Analytics did for Marketing, xAPI is doing for learning.” — Amy Parent, Learning Experience Designer intro xapi next steps takeawaysdata assessment pilot
  7. 7. How do we get there?
  8. 8. What is xAPI? Stacey Completed Course #1 Actor Verb ObjectExperience Data is captured using the Experience API (xAPI), a standardized Actor — Verb — Object data format. This interoperable data structure allows you to activate information across tools and technologies where Experience Data is created in real-time. intro xapi next steps takeawaysdata assessment pilot
  9. 9. Because xAPI is designed around activity-based data collection, xAPI data can provide high-resolution insights into specific learner activity. For example, watching a video is actually a summation of many, much more granular, actions.  This information can show us how engaging or valuable an experience is for a learner. Stacey Completed Course Launched Viewed Played Clicked Paused Clicked Opened Clicked Clicked Submitted Viewed Closed Course Title Page Video Rewind Video Resource Link Article Radio Button #1 Radio Button #2 Answer Final Slide Course But Really — What is xAPI? intro xapi next steps takeawaysdata assessment pilot
  10. 10. Where Does It Go? intro xapi next steps takeawaysdata assessment pilot
  11. 11. Creating a Data-Driven L&D Team +A RETAIL CO.
  12. 12. We need to move from a content focus to a data focus.” intro xapi next steps takeawaysdata assessment pilot
  13. 13. We don’t know what’s being used.” intro xapi next steps takeawaysdata assessment pilot
  14. 14. We recognize an increasing percentage of learning is happening outside of the LMS but we have no visibility into that.” intro xapi next steps takeawaysdata assessment pilot
  15. 15. To start to solve those problems A Retail Company turned to xAPI. Members of their team began to learn xAPI on their own, with the cohort and information online. Their first steps... intro xapi next steps takeawaysdata assessment pilot
  16. 16. ...led to big questions. We don’t know what questions to ask right now.” How do we figure out the right point to start?” intro xapi next steps takeawaysdata assessment pilot
  17. 17. Next Steps Data Assessment Pilot Time to make a plan! A great xAPI transformation starts with a great plan. A great plan leads to a smooth pilot. And a great pilot leads to the right iterations. intro xapi next steps takeawaysdata assessment pilot
  18. 18. Goals: 1. Gain a high level understanding of current systems, business objects, initiatives and technology in use 2. Design an efficient and effective approach to the implementation of xAPI within the learning ecosystem Implementation Onsite Workshop Data Assessment intro xapi next steps takeawaysdata assessment pilot
  19. 19. ● What are the current metrics I use? ● What else would I like to know about my learners? ● What reports do I currently use? ● What types of information does my boss ask for that I do not know? ● What are some specific initiatives that I want more insight into? ● What are some high impact programs that I need more data from? Business SideData Assessment intro xapi next steps takeawaysdata assessment pilot
  20. 20. Business Side Internal HR Resources Management Training Seller Readiness Learner Engagement Onboarding Training Specific Compliance Training ● What are the current metrics I use? ● What else would I like to know about my learners? ● What reports do I currently use? ● What types of information does my boss ask for that I do not know? ● What are some specific initiatives that I want more insight into? ● What are some high impact programs that I need more data from? Data Assessment intro xapi next steps takeawaysdata assessment pilot
  21. 21. ● What platforms are available? ● What platforms are already connected? ● How can I access the platform’s data? ● How do I manage identity? ● What are all my data assets I currently have available? ● Where would myself and/or my learners access the data dashboards? Technical SideData Assessment intro xapi next steps takeawaysdata assessment pilot
  22. 22. ● What platforms are available? ● What platforms are already connected? ● How can I access the platform’s data? ● How do I manage identity? ● What are all my data assets I currently have available? ● Where would myself and/or my learners access the data dashboards? Technical Side LMS Knowledge Base Micro-Learning Platform(s) Learning Experience Platform Communication Tool(s) Intranet Data Assessment intro xapi next steps takeawaysdata assessment pilot
  23. 23. Internal HR Resources Management Training Seller Readiness Learner Engagement Onboarding Training Specific Compliance Training Map the Ecosystem LMS Communication LMS Intranet Micro learning Knowledge Base LMS LMS LMS LXP Communication Data Assessment intro xapi next steps takeawaysdata assessment pilot
  24. 24. Evaluate & Prioritize Business Impact Technical Difficulty low low high high Data Assessment intro xapi next steps takeawaysdata assessment pilot
  25. 25. Evaluate & Prioritize Business Impact Technical Difficulty low low high high Data Assessment intro xapi next steps takeawaysdata assessment pilot
  26. 26. Pro-Tip #1: Connect your L&D team and your IT resources early on to make sure all parties are onboard Data Assessment intro xapi next steps takeawaysdata assessment pilot
  27. 27. Pro-Tip #2: Think about your data strategy in both a current state and an ideal future state Data Assessment intro xapi next steps takeawaysdata assessment pilot
  28. 28. Pro-Tip #3: Figure out a program/objective/goal that will show a ton of impact through a lower technical lift Data Assessment intro xapi next steps takeawaysdata assessment pilot
  29. 29. Pro-Tip #4: Touch base with other departments to hear what kinds of information they would like to know from L&D Data Assessment intro xapi next steps takeawaysdata assessment pilot
  30. 30. Implementation Technical Development Pilot Goals: 1. Prove value to other stakeholders on the potential of xAPI data usage 2. Build in a way that is reusable and future-proof 3. Gain transparency and visibility into the learner experience and content utilization intro xapi next steps takeawaysdata assessment pilot
  31. 31. Learner Engagement ● Gain transparency into learner’s content utilization and platform adoption. ● Identify critical experience data activity types that are valuable to support business objective ● Create organization specific xAPI Profile for future iteration and ecosystem build out LMS: Oracle Learn Intranet: SharePoint Micro-Learning: BrainStorm Pilot intro xapi next steps takeawaysdata assessment pilot
  32. 32. Identity Resolution through SSO Centralize & Standardize LMS: Oracle Learn Intranet: SharePoint Micro-Learning: BrainStorm Pilot intro xapi next steps takeawaysdata assessment pilot
  33. 33. Identity Resolution through SSO LMS: Oracle Learn Intranet: SharePoint Micro-Learning: BrainStorm Centralize & StandardizePilot intro xapi next steps takeawaysdata assessment pilot
  34. 34. Outcomes ● Transparency into the learner experience like never before ● Consolidated information from across platforms ● Real-time reporting for answers when you need them ● Data is accessible and actionable Pilot intro xapi next steps takeawaysdata assessment pilot
  35. 35. Mission control for learning & performance demo.yetanalytics.io intro xapi next steps takeawaysdata assessment pilot
  36. 36. Pro-Tip #5: Don’t try and boil the ocean, analytics can be phased. Start small for early wins Pilot intro xapi next steps takeawaysdata assessment pilot
  37. 37. Pro-Tip #6: Keep your narrative in mind, not everyone cares about ‘data’ but everyone cares about the outcomes Pilot intro xapi next steps takeawaysdata assessment pilot
  38. 38. Pro-Tip #7: Have visuals! A static dashboard to show outcomes is even stronger than the narrative Pilot intro xapi next steps takeawaysdata assessment pilot
  39. 39. Implementation Planning & Iteration Next Steps Goals: 1. Get admins using dashboards 2. Build internal evangelists 3. Identify next priority for implementation intro xapi next steps takeawaysdata assessment pilot
  40. 40. Cascading Adoption ● Start with the admins to build in-house evangelists ● Next roll out dashboards to the Manager level to expand ● Finally to the individual, who can ask questions directly to their manager for questions ● Every deployment with Yet comes with webinar trainings and direct support L&D Admins ManagerManager Learner Learner Learner Learner Next Steps intro xapi next steps takeawaysdata assessment pilot
  41. 41. Work in Steps ● Gather feedback ● Reflect on the process ● Choose which business objective to tackle next! ● Revisit and reassess with further data assessment Internal HR Resources Management Training Seller Readiness Learner Engagement Onboarding Training Specific Compliance Training Next Steps intro xapi next steps takeawaysdata assessment pilot
  42. 42. Pro-Tip #8: Think about how you will present findings to your stakeholders from the start Next Steps intro xapi next steps takeawaysdata assessment pilot
  43. 43. Pro-Tip #9: Hold in-person user testing meetings to get feedback as people use the data and have questions Next Steps intro xapi next steps takeawaysdata assessment pilot
  44. 44. Pro-Tip #10: Determine who else should have access to the data. Managers? Learners? Next Steps intro xapi next steps takeawaysdata assessment pilot
  45. 45. Making it real for you! Platform Insights Ideal Impact Worksheet intro xapi next steps takeawaysdata assessment pilot
  46. 46. Next Steps Data Assessment Pilot Remember it’s a Process! A great xAPI transformation starts with a great plan. A great plan leads to a smooth pilot. And a great pilot leads to the right iterations. intro xapi next steps takeawaysdata assessment pilot
  47. 47. Thank you! ● Visit us at booth #118 ● Check out the live demo - https://demo.yetanalytics.io ● Free LRS Sandbox - https://explore.yetanalytics.io ● Let us scan your badge and we’ll send these slides ● Want to discuss your project? Send us an email at hello@yetanalytics.com problem data assessment outcomes pro-tipsphases data strategy

×