Quality Frameworks for Online Education by Darco Jansen, EADTU

1. Quality Frameworks
for Online Education
UDOL, Derby
10 January 2017

2. EDUCATION
RESEARCH INNOVATION
A variety in university profiles and visions
Universities collaborate with peers on the
basis of trust

3. Universities going online
• Nexus of (online) education with research and
innovation: the knowledge triangle
• Three area's of provisions emerging in the
Western world: degree education, continuing
education / continuous professional
development and Open education (OER,
MOOCs)
• Universities integrate online in all provisions and
become borderless by globalisation and the
internet

4. Three areas of provision
Degree education
Continuous
education /
CPD
Open
education
InternationalNational

5. The complex pedagogical landscape
Blended degree
education:
• Bachelor, Master, PhD
• Business model:
regulated, not-for-profit;
education as a public
good, not as a commodity
Blended and
online
continuous
education
• CPD, SLP’s and non-
accredited education
• Business model: non-
regulated, not-for-profit,
for profit
Open
Education
• MOOCs, OERs, Open
media, Open knowledge
• Business model: non-
regulated, not-for profit
InternationalNational

6. Three areas of provision: again different profiles
Degree education zone, backbone in the education system to develop complex academic
and professional competences: bachelor, master, PhD – increasingly blended solutions to
raise quality and to face growing student numbers. Online distance higher education provides
flexibility for lifelong learners.
Continuing education and training on demand, valorisation of knowledge to support
innovation in the public and private sector, based on research and development. Flexibility
requires online or blended solutions, such as (virtual) seminars, CPD, knowledge alliance and
corporate university initiatives, short learning programmes, master classes, expert schools,
etc. It includes knowledge networks for professionals or business sectors.
Open education and knowledge sharing area, pushing knowledge online into the public
domain: OERs, MOOCs, open media, open access/open innovation materials – preferably to
be designed and arranged according to the needs of user groups/networks

7. Internationalisation formats degree programs
Exchange mobility
(Erasmus)
Blended/online mobility
(Virtual Erasmus)
Intensive programmes,
summer schools;
blended/online
discussion groups, think
tanks, seminars and
webinars
Networked curricula and
double degrees
Joint curricula and joint
degrees
Joint PhD degrees
International
apprenticeships
Collaborative projects

8. Internationalisation formats continuing education
and CPD
Short learning
programmes
Executive
seminars
University-
business networks
(e.g. KICs of EIT)
Corporate
university
programms
International
apprenticeships
Collaborative
projects with
companies
involved
Continuous
professional
development
Webinars
Intergovernmental
CPD initiatiatives
(teacher training,
health care...)

9. Assets of ICT based learning
• International and large-scale outreach, by borderless education involving
university partners abroad
• Personalized teaching and learning , putting the students with their ambitions
and talents at the centre, with individual learning arrangements. Learning tools
and learning analytics with personalized feedback make this possible.
• Small scale and intensive education, looking for a balance between
education for qualifications, socialization and personal development. All this
together is Bildung. Smaller scale is made possible by learning communities.
• Rich learning environments relating to research, innovation and professional
employment, involving the student
• Openness to learners through flexible, inclusive structures and methods that
take higher education to students when and where they need it. Education is not
exclusive. More than 50 pct. of an age cohort follows a form of higher education
• Networked education and mobility, where students can learn across national,
sectoral and institutional boundaries.
• Cost-effectiveness by low variable cost

10. Mainstreaming “Online”
Mainstreaming online education in different blends
will support university strategies for degree
education, continuing education/CPD and open
education. Institutional responses will be
diverse.
Blended and online education will enhance
international partnerships and collaborative
courses /curricula and mobility

11. • Copying lectures doesn’t add enough value as the innovative potential
of online learning is not used
• the variable cost of high quality digital learning does not achieve
economies of scale if you maintain the same pedagogy (Laurillard,
2014)
• to enhance quality, effectiveness and scalability in digital education,
learning design is needed
Education is a design science
Going (partly) online …
It is not the same pedagogy

12. Five main challenges in designing a course
Learning activities
• which learning activities should be designed for students in the
course to reach the learning objectives?
Sequence
• How to sequence the learning activities
Student support
• How to support students?
Learner control
• How to increase learner control
Assessment
• How to assess students during the process and at the end of it?

13. Innovative pedagogies
Learning design
informed by
analytics
Flipped
classroom
Dynamic
assessment
Personal inquiry
learning
Learning
through
storytelling
Treshold
concepts
Digital
scholarship
Learning from
gaming
MOOCs
Massive open
social learning

14. Blended learning
• The most appropriate modes of teaching and learning for a
course should be used in an optimal way and in an optimal
blend.
• This is a qualitative judgement, based on multiple factors
(course content, student characteristics, course objectives,
learning activities to be designed, environment or software
available, etc.). The blend is not a quantitative issue.
• The design concerns the choice of media, the sequence of
activities and the optimal blend of online and f-2-f
education.

15. Quality assurance
As is the case for research publications, the possibility
exists to organise quality assurance ex ante by peer
reviews or by test implementations for a small group of
students (Elen, 2011; Laurillard, 2014).
The review should focus on the content as well as on
the educational design.
Only after a positive evaluation, the course will
subsequently be developed and anchored in the blended
learning environment. Eventually, the course can be
approved for a limited period in which a re-design is
prepared.

16. Are quality assurance frameworks sufficiently flexible to adapt to
differentiated approaches and innovations in pedagogy?
ESG 2.3
External quality assurance does not end with the report by the experts. The
report provides clear guidance for institutional action. Agencies have a
consistent follow-up process for considering the action taken by the
institution. The nature of the follow-up will depend on the design of the
external quality assurance.
ESG 2.6
In order for the report to be used as the basis for action to be taken, it needs
to be clear and concise in its structure and language and to cover - context
description (to help locate the higher education institution in its specific
context); - description of the individual procedure, including experts involved;
- evidence, analysis and findings; - conclusions; - features of good practice,
demonstrated by the institution; - recommendations for follow-up action.
Seeking and documenting examples of innovation in the curriculum are
necessary.

18. 4.31
It is clear that higher education institutions, their staff and
students, and the quality assurance agencies all have
concerns about relevant and effective quality assurance
for online and blended education.
They are still in the beginning of a transition period, which
has to be accelerated to fully exploit the opportunities of
new modes of teaching and learning and to keep track
with the international developments in higher education.

19. • National governments must review their legislative and regulatory frameworks
and practices for quality assurance and accreditation in higher education
(including recognition of prior learning) to ensure that they encourage, and do
not impede, the provision of more flexible educational formats, including degrees
and other ECTS-bearing courses that are fully online.
• National QA agencies should develop their own in-house expertise and
establish processes that are sufficiently flexible to include recognising and
supporting modes of teaching and learning. They should evaluate institutions
on their active support of innovation (or importantly, the lack of it), and its
impact on the quality of teaching and learning.
• ENQA and other relevant European networks should support the sharing of
good practice by national QA agencies in the development of criteria on the
recognition of new modes of teaching and learning.
CPL-Recommendations related to QA

21. E-xcellence: QA in e-learning instrument
• Curriculum design, Course design, Course
delivery, Services (student and staff support),
Management (institutional strategies)
• E-xcellence focuses on elements in course
provision that contribute to Lifelong Learning
schemes, like:
 ease of access to courses and services
 new forms of interaction (students and staff)
 flexibility and personalisation
• E-xcellence is a benchmarking instrument.

22. Quality Assurance spectrum
Check compliance
with quality standards
Support institutions in
quality enhancement
E-xcellence
Flexibility to:
 set relevant goals
 demonstrate quality of
performance
 plan for improvement

23. will help the university:
• to develop e-learning programmes
• to guide the internal discussion
• to improve the quality of e-learning performance
• to learn from other similar institutions
• to use existing good-practices
• to be up-to date on developments in e-learning
E-xcellence tool
E-xcellence Project

24. E-xcellence manual and framework
1. Strategic Management
2. Curriculum Design
3. Course Design
4. Course Delivery
5. Student Support
6. Staff Support
See: e-xcellencelabel.eadtu.eu/

25. E-xcellence resources
• E-xcellence manual (in PDF format)
– 35 benchmark statements
– Performance indicators
– Supporting text
• Quickscan tool
• Assessors’ notes
• Framework for action roadmap
See: e-xcellencelabel.eadtu.eu/

26. Quickscan
• Quick self-assessment of the HE
institution’s e-learning
• Uses the 35 E-xcellence benchmarks
• Helps institution to identifies strengths and
weaknesses
• Online version provides guidance and
feedback
• Best used collaboratively by an
interdisciplinary team

27. The Quickscan tool

28. E-xcellence review
• Onsite visit (1-2 days) or online meetings
• Engage with the HE institution in
constructive discussions about the quality
of their e-learning, and how it might be
improved
• Provide an independent external review,
with advice for improvement
• Collect feedback on the E-xcellence
resources, in order to improve them

29. Review format
• Preparation
– Decide programme(s) to be reviewed
– Form interdisciplinary team of the HE institution’s managers, course
designers, educators, students, technical staff
– Team meets to complete QuickScan self-evaluation
• Visit or online meetings
– HE team meets with E-xcellence reviewers (2 - 4 e-learning experts)
– Discuss institution’s e-learning offerings and the QuickScan self-
evaluation
– Reviewers give initial feedback and suggestions
• Reports
– Summary report from E-xcellence reviewers
– Reviewers consider and agree Roadmap for improvement from the
institution

30. E-xcellent manual
update
• Reflects recent trends in e-learning
– rapid rise of MOOCs
– surge of interest in learning analytics
– increasing use of learning design in course development
• Includes new topics
– increased focus on personalization
– flipped approaches to teaching
– virtual and remote laboratories
– digital badges and e-portfolios.

31. 2016

32. • OpenupEd is an open, non-profit
partnership for MOOCs
• OpenupEd aims to open up education to
the benefit of learners and the wider
society while reflecting values such as
equity, quality and diversity.
• The vision is to reach learners interested
in online higher education in a way that
meets their needs and accommodates
their situation.

33. 8 OpenupEd features
• Openness to learners
• Digital openness
• Learner-centred approach
• Independent learning
• Media-supported interaction
• Recognition options
• Spectrum of diversity
• Quality focus

34. Definition MOOCs
• MOOCs are online courses designed for large numbers of
participants, that can be accessed by anyone anywhere as long as
they have an internet connection, are open to everyone without
entry qualifications and offer a full/complete course experience
online for free.
• http://www.openuped.eu/images/docs/Definition_Massive_O
pen_Online_Courses.pdf (adopted by many EU MOOC projects)

35. Why quality matters?
• quality of the pedagogies employed
• low completion rates
• a failure to deliver on the promise of inclusive and equitable quality
education for all
• pathway to higher education (recognition options)

36. Quality check by…
• Checking the design of the course
• Criteria of the course (MOOC)
• If it indeed contribute to the objectives
• For learners
• For participants
• For institution…

39. Perspectives on MOOC quality
• Assess quality primarily from the learner’s point of view.
• Quality is connected to the pedagogical framework of the MOOC
• Quality is related to inputs (e.g., instructional design, the content
and resources, activities and assessment, and the technology
employed, or the quality of the teacher.
Flaw in quality multiple choice questions (MCQs) in the quizzes
• Quality with outcome measures, such as the number of learners
completing a MOOC or achieving certification.

40. Complexity with MOOC quality
• QA on MOOCs cannot be easily standardised as they have several
different aims.
• Even within one MOOC there is no uniformly aims between actors
involved (institution, the teaching staff involved and the
participants).
• MOOCs are designed for various target groups, and even within
'one target group' the motivation and intention of MOOC
participants vary a lot.
• Unbundling of educational services: quality emerges from the joint
enterprise and is not solely the responsibility of one partner

41. MOOC quality models
• quality principles developed for HE could be used to improve the
quality of MOOCs.
• from systems which check compliance to norms and often focus on
product, to systems that aim at quality enhancement by focusing
on process.
• low maturity systems are characterised by externally set norms,
whereas in high maturity systems institutions have embedded
processes aimed at quality enhancement towards their own
objectives.

42. More holistic MOOC quality models
• Ossiannilsson et al (2015) present a global survey of quality models
for e-learning. They find that most models take a holistic view of
quality, recognising the need to address many aspects of the
enterprise. Although the models vary considerably in the detail and
number of indicators, most covered a consistent set of important
dimensions.
• Example E-xcellence label -> OpenupEd Quality label for MOOCs
• benchmarking, self-assessment, roadmap

43. Quality models for MOOCs
http://eadtu.eu/documents/Publications/Quality_Frameworks_for_MOOCs_Springer.pdf

44. Thank you!!!
Darco.Jansen@eadtu.eu