2017-03-09 OE Global in Cape Town - MOOQ Interactive Workshop on Quality of MOOCs - Results from collaboration

1. MOOQ for the quality of MOOCs:
“We will make MOOCs better”
Quality Reference Framework with
indicators for design & comparison
www.MOOC-quality.eu
Frameworks: MOOQ

2. The quality of MOOCs:
Results from three
research surveys and
discussion of the QRF
Workshop @ EC-TEL 2016, Lyon
by Christian M. Stracke et al.
Open University of the Netherlands

3. Open CC License for
sharing & re-using slides
This work is free to share under the creative commons licence:
"Attribution – Noncommercial – Share Alike 3.0"
You can copy, distribute and transmit the work under the following conditions:
1. Attribution –
2. Noncommercial –
3. Share Alike
Licence: Attribution-Noncommercial-Share Alike
Some rights reserved, see: http://creativecommons.org/licenses/by-nc-sa/3.0/

4. Open University of the Netherlands
Global cooperation: ECNU & KNOU
Global initiative ICORE for OR & OE
Annual international LINQ Conference
eLC European Institute
Standardization Chair in ISO & CEN
Dr. Christian M. Stracke:
Open Learning & Education, Innovations,
Policies, Quality & Competences, Impact

5. Source: https://www.flickr.com/photos/opensourceway/

6. MOOQ for the quality of MOOCs:
“We will make MOOCs better”
Quality Reference Framework with
indicators for design & comparison
www.MOOC-quality.eu
Frameworks: MOOQ

7. Quality Reference Framework with
indicators for design & comparison
Our main goal is the collaboration with all
to improve future MOOC design for
better MOOC learning experiences

8. Three main surveys for learners,
designers and facilitators of MOOCs
Please support us to improve next MOOCs:
www.survey.MOOC-quality.eu
Your feedback is most welcome!

10. Global MOOC Survey

11. Hands-on-Session
Explore the quality indicators of
MOOCs

12. Our proposal
for discussion:
The Quality Reference
Framework (QRF)

13. What is Open Education (OE)?
Quality dimensions of OE in practice?
Quality indicators for OE in practice?
Quality Open Education

14. 1. Pedagogical
2. Technological
3. Business Model
MOOQ Matrix

15. 1. MOOC Learners
2. MOOC Designers
3. MOOC Facilitators
4. MOOC Providers
MOOQ Target Groups

16. Quality constructs
Constructs Learners Designers Facilitators
Pedagogical Decisions X
Learning Objectives X X X
Duration & Structure X X
Duration & Interaction X
Learning Resources X X X
Learning Support X X X
Flexibility & Inclusion X X
Learning Progress X
Learning Environment X X
Learning Assessment X X X
Learning Certification X X
Design Process X
Online Facilitation X

17. Stracke, C.M., 2016: “Openness for learning quality and change by Open Education in theory and practice - Overview, history, innovations and policies:
How can Open Learning, OER and MOOCs achieve impact for learners, organizations and in society?” [in print]
ID ISO/IEC 19796-1 New ID Proposed Modification
NA Needs Analysis
AANN AAnnaallyyssiiss
FA Framework Analysis
CD Conception / Design DDEE DDeessiiggnn
DP Development / Production
PPRR PPrroodduuccttiioonn
IM Implementation
LP Learning Process LLEE LLeeaarrnniinngg
EO Evaluation/ Optimization
EEVV EEvvaalluuaattiioonn
OOPP OOppttiimmiizzaattiioonn

18. Stracke, C.M., 2016: “Openness for learning quality and change by Open Education in theory and practice - Overview, history, innovations and policies:
How can Open Learning, OER and MOOCs achieve impact for learners, organizations and in society?” [in print]
Meso
Micro
Macro
Objectives Achievements
Realizations
AN
DE
PR
LE
OP
EV

19. OE Quality Dimensions
Evaluation
Analysis
Implemen-
tation
Learning
process
Design
Learner

20. Quality dimensions of OE in practice
Dimension 1: Analysis
Dimension 2: Design
Dimension 3: Implementation
Dimension 4: Learning Process
Dimension 5: Evaluation & Optimization
OE Quality Dimensions

21. Quality indicators for OE in practice
For Dimension 1:
For Dimension 2:
For Dimension n:
OE Quality Indicators

22. Quality indicators for analysis in OE (1):
• Institutional purpose
• Existing provisions
• Target audiences
• Technical limitations (front & back-end)
• Learning needs
• Licensing
• Budget
OE D1: Analysis

23. Quality indicators for analysis in OE (2):
• Team and staff
• Content expert
• Reputation
• Market analysis
• Societal need
• Inclusion
• Long-term usage of the MOOC
OE D1: Analysis

24. Quality indicators for design in OE:
• LD/ID process to be followed (list of examples,
such as …)
• Inclusion and accessability
• Sustainable solution (wrt resources)
• Pedagogical choices & learning theory
• Re-usage of former courses & OER
• Definition of success indicators
OE D2: Design

25. Q. indicators for implementation in OE:
• Project & time management
• Infrastructure & tools
• Experts & skills
• Usability testing
• Training & development for facilitators
OE D3: Implementation

26. Quality indicators for learning in OE:
• Usability questions & feedback from learners
• Quality of communication from learners
• Engagement from learners
• Upfront workload when MOOC is starting
• Learners‘ satisfaction
• Achievement of learners‘ learning objectives
• Achievement of designers‘ learning objectives
• Achievement of providers‘ learning objectives
OE D4: Learning Process

27. Quality indicators for evaluation in OE:
• Completion rate
• Satisfaction rate (learners‘, facilitators‘
and providers‘)
• Achievement of learners‘ learning objectives
• Achievement of designers‘ learning objectives
• Achievement of providers‘ learning objectives
• Learning analytics for evaluation of MOOC
• Optimization of MOOC & Innovation
OE D5: Evaluation