Supporting the Acquisition of 21st Century Skills through Multimodal Learning Analytics
•Download as PPTX, PDF•
0 likes•265 views
Collaboration, communication, creativity, critical thinking and problem-solving are among the skills that are needed to study and work in this 21st century. As important as they are, evaluating, assessing and teaching them in a practical, scalable and efficient way is still a challenge not fully met by current pedagogical-technological practices. Multimodal Learning Analytics (MmLA), the processing and analysis of multiple sources of data to better understand and improve learning processes, has been posed as a possible solution to augment the natural capabilities of both instructors and students to provide and receive feedback to support the development of those skills. During this session, we will explore the affordances that low-cost sensors and current advances in artificial intelligence provide to automatically record and analyze face-to-face, complex learning processes as those involved for the development of 21st-Century Skills. Finally, we will discuss and ideate practical MmLA tools that could be built to augment your current teaching and learning practices.
Recommended
Recommended
More Related Content
Similar to Supporting the Acquisition of 21st Century Skills through Multimodal Learning Analytics
Similar to Supporting the Acquisition of 21st Century Skills through Multimodal Learning Analytics (20)
More from Xavier Ochoa
More from Xavier Ochoa (20)
Recently uploaded
Recently uploaded (20)
Supporting the Acquisition of 21st Century Skills through Multimodal Learning Analytics
- 1. Supporting the Acquisition of 21st Century Skills through Multimodal Learning Analytics Xavier Ochoa
- 3. Augmenting human capabilities in educational contexts
- 4. Increasing the capability of humans to approach a complex problem situation, to gain comprehension to suit their particular needs, and to derive solutions to problems. Douglas Engelbart Augmenting Human Intellect 1962
- 5. Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.
- 7. Fundamental
- 8. Images in public domain. Obtained from Wikimedia Foundation.
- 11. Difficult to provide personalized feedback
- 12. Multimodal Learning Analytics MmLA Capturing learning traces wherever they happen
- 13. Humans communicate (and leave traces) in several modalities What you say is as important as how you say it
- 15. We need to analyze the traces with variable degrees of sophistication And we have to do it automatically as humans are not scalable
- 17. We need to provide feedback in the real world Often in a multimodal way too
- 19. 4-Easy-Steps for Provide Feedback for Fundamental Skills with MmLA
- 20. Step 1: Capture relevant data
- 21. Step 1: Capture relevant data • Recommended Sensors: • Cameras • Normal • Fisheye (horizontal 360 degrees) • Array (360 degrees) • Micropohones • Normal • Array (Directional information) • Digital Pens
- 22. Step 1: Capture relevant data • Sensors for labs: • Eye-trackers • EDA – pulse wristbands • Brain activity
- 23. Step 1: Example Collaboration around a table
- 24. Setting: Collaboration around a table
- 25. Setting requirements • Non-intrusive • Capture the same information that a human observer • Mainly faces and speech of participants • No cables • Synchronized • Easy-to-operate • Record at least 2 hours
- 26. MmLA Sensors – Collaboration around a table ≈
- 27. MmLA Sensors Fisheye Camera Microphone Array (Respeaker) Computer (Raspberry Pi) Battery Pack
- 28. MmLA Sensors $ 30 $ 40 $ 40 $ 25 ≈ $ 150
- 29. Step 2: Extract low-level modalities
- 31. Step 2: Extract low-level modalities • Video: • Body posture • Head posture • Gaze • Facial Expression • Movement • Object Recognition / Tracking • Object Interaction • Gestures • Actions
- 32. Step 2: Extract low-level modalities • Audio: • Direction of arrival • Speech to Text • Prosodic features • Speaker diarization • Text: • Word frequency • Sentiment analysis • Questions asked • …
- 33. Step 3: Extract low-level modalities • Digital Pen: • Writing mechanics • Text • Sketch recognition
- 34. Step 2: Example Collaboration modalities
- 35. MmLA Low-Level Feature Extraction - Video • Face/Body counting (How many students around the table) Library: OpenPose / OpenFace
- 36. MmLA Low-Level Feature Extraction - Video • Body posture (2D position of main body joints/features) Library: OpenPose / OpenFace
- 37. MmLA Low-Level Feature Extraction - Video • Face posture / Rough Gaze Direction Library: OpenFace / Open Pose
- 38. MmLA Low-Level Feature Extraction - Video
- 39. MmLA Low-Level Feature Extraction - Video • Face Action Units – Facial Expression
- 40. MmLA Low-Level Feature Extraction - Audio • Direction of Arrival
- 41. MmLA Low-Level Feature Extraction - Audio • Speech to Text
- 42. What happen now that we are online?
- 43. Step 3: Fuse low-level features into learning constructs
- 44. Fundamental
- 45. Step 3: Example Collaboration features to collaboration constructs
- 46. Feature Fusion • Participation • Direction of arrival • Text • Joint attention • Gaze fusion • Turn taking • Direction of arrival • Speaker diarization • Collaboration moments • Participation • Text analysis • Idea uptake • Turn taking • Idea words • Leadership • Turn taking • Collaboration moments • Text analysis
- 47. Step 4: Provide Feedback
- 48. Feedback requirements • Enable self-reflection • Give a path to improvement • Non-intrusive, part of the experience
- 49. Step 4: Example Feedback for oral presentations
- 51. ZoomSense
- 52. Conclusions
- 53. It is easier to do Learning Analytics where there is a lot of learning traces But are we looking where it is useful to look?
- 54. Multimodal Learning Analytics is not a just a sub-field of of Learning Analytics It is its natural evolution
- 55. We need to explore as many contexts as possible Have an idea? Let’s talk!