Moocs, learning design and assessment 12/12/12

MOOCS and Open Learning
Variables, Cases, Learning Design: Dec 2012

Research and analysis: Sarah Lambert
Presented to colleagues 11/12/12

Open online courses
• Structured subjects or courses with
– A schedule and topics, and usually a start/end date
– Open resources eg iTunesU, YouTube, open academic journals,
open textbooks
– Open forums and blogs
• Some also have open aggregated webspaces that bring together material
created by participants eg course-wide blog with rss feeds from
participants’ blogs/tweets
• Some also have private aggregated webspaces that bring together
material created by participants and shared across cohort
– Open assignments and final exams/assessments
• Often with a peer-review component
– Facilitation by teachers and/or educational researchers with
support from volunteers including support to use the web
components, feedback on posts and assessments

Moocs
• A Typical Massive Online Open Courses (MOOCs)
– Often offered by consortiums of wealthy universities
– Not for credit (especially to their enrolled students)
• Used as an optional, free, extra resource to formal learning
• Can be used by separate tutoring, learning assistance orgs
– Have a start and end date
– Tens of thousands enroll
• Curious about the topic, curious about Moocs
– Minority engage, fewer complete/master topic
– Media frenzy, tranformative/disruptive
• “biggest change to education since Gutenberg” (Massachusetts
Bay Community College president John O'Donnell in reference to
edX)

Smoocs
• A typical Small Online Open Courses (SMOOCs)
– Hundreds enroll, 20-30% engage, 5-10%
complete assessments
– Have a start and end date
– Smaller and singular providers
– Perhaps a greater emphasis on curriculum,
incremental assessment, certification and
completion
• Equivalence for some professional development/in-
service type courses

The Roman World (La Trobe - iTunes U)

iTunesU courses
• More a lecture series than a course
• No assessment or facilitation
• However now being used to sequence other learning
materials eg handouts
• Getting close to a resource-based MOOC
• A good iTunes U collection + quiz series/final exam can be
relatively easily turned into a Mooc eg La Trobe’s recent
iTunesU offering on Ancient Greece attracted 45K
subscribers (Australian Education supplement, 8/8/12)

Open University Australia
• Not free, but “open” in that:
– No pre-requisites
– Fully Online
• Access to course materials and assessment is after paying
the course fee c$2500 (postgrad public health unit
Nov2012)
– Includes access to generic academic skills
courses
• Nov 2012: considering hosting Moocs for other
universities

OERU
• Aims to deliver the first for-credit course in the world
• Not-for-profit consortium of universities
– Contributing small number of subjects towards
a Bachelor of General Studies
– Resources free, assessment for credit at cost
• UOW a founding partner
• First subject has just gone live
– Regional relations in Asia and the Pacific
(AST1000) from USQ
– http://wikieducator.org/AST1000

Variables in Open online courses
• Length of course: 5-7/11-12/24 weeks
• Level: school, uni-prep, uni-transition, 1st year – 4th year
• Timing: co-incides with F2F/campus course vs arbitrary start-date
• Blended cohort: separate online cohort vs blend of online and
formal/campus cohorts
• Syncronous virtual events: 0/>4/fortnightly/weekly/biweekly
• Media resources: all new media vs re-use existing inc from other
providers vs mix of both
• Assessment for certification: n/a vs single vs mix/multiple
• Formative assessment: form/posts/quiz/mix
• Who owns it: Proprietory platforms and IP in resources vs Creative
Commons, re-usable by anybody content
• Distinctive features: these vary in each case eg cohesive website,
open-textbooks, ePortfolio for all students, virtual
graduation/certification events

Open learning cases

TESTING THE VARIABLES

POLSC101 (Saylor.org)
• Length of course: 12 weeks
• Level: 1st year
• Timing: roughly same time as spring/autumn sessions
• Blended cohort: separate online cohort
• Syncronous virtual events: 0/>4/fortnightly/weekly/biweekly
• Media resources: mostly re-use existing media from other providers
• Assessment for certification: final exam
• Formative assessment: form/posts plus assignments that are self-
assessed by comparing to answer-sheet/sample answer
• Distinctive features:
– ePorfolio
– cohesive website/elearning platform shows complete
subject offerings which can be taken in any sequence
to create individual learning ePortfolio

Intro to Mathematical Thinking
(Coursera/Standford)

• Level: uni-transition, 1st year
• Timing: intentionally co-incides with F2F/campus course
• Blended cohort: proactively encouraging teachers and their cohorts of campus learners to
join to created a blended cohort, aiming to create blended study groups
• Synchronous virtual events: nil
• Media resources: mix of new and re-purposed
• Assessment for certification: progress quizzes (auto marked “Problem sets”), final exam (peer
reviewed)
• Formative assessment: forms/study groups/, weekly assignments (worked problems,
feedback by comparing answers with students in forums), practice peer review activity (to get
used to the technical/website interface),
– 2 Education PhD students as core facilitators, pre and post course
evaluation part of their research
– Pre-course survey identified c900 volunteer “teaching assistants”
– open-textbook specially created for this course
– Quizzes embedded in lecture videos
– attempts to create mixed study groups of online/remote students plus
campus-based students doing formal maths study ie peer learning
• they are hoping the campus tutors will help the campus students who will in turn
help the online students who have no formal virtual coach/tutor

• Sarah’s experience
“I started this one but dropped out halfway. Video lectures were
great. The activities were comprehensive and provided good
opportunity for deep learning. But I was overwhelmed by the mass
of activity and impersonal nature of the forums. The forums were
desperately ‘on-topic’ and dry. Even the “Maths for musicians” one I
joined. No personal connection with other learners – maybe it would
have helped to motivate me to stick with it. . The Professor was
never in the forums that I saw (just weekly emails), no sense of his
presence in the website.

In the end, I drowned in Greek. I got busy at home and travelled on
the weekends (instead of doing some maths.) Subject matter went
from “gee, I can do this” to “erg I have not got a clue”. I fell behind,
and needed hours to catch up. Which I didn’t have. There were no
cost or work implications for dropping it. So when the going got
tough, Maths was the first thing to go.


An alarming post-script. Despite not doing the Final Exam (which I thought
was compulsory), and getting a final mark of 21.3%...I was awarded a
certificate “Statement of Accomplishment”!

(John Hopkins Bloomberg School of Public Health via Coursera)
Community Change in Public Health
• Timing: did not seem to co-incide with a semester
• Blended cohort: none that I could see
• Synchronous virtual events: nil
• Media resources: some new, mostly re-purposed
• Assessment for certification: 2 progress quizzes, final assignment(peer
reviewed)
• Formative assessment: fewer, well structured forums
– Really well scaffolded major assignment, with 2 rich case-
studies, 6 questions per case, and 2 small essays
• Peer-review guided by very good rubric and marking notes
• Took 2 hours to do, and 30-45minutes to mark
• Each student who submitted was asked to review 4 assignments

Community Change in Public Health
“Totally different experience to the Maths course. Tightly aligned
with experience in my private life. This was to see if I would like more
formal study in the area in which I currently volunteer.

The video lectures were great and I could apply my experience to
them, creating my own examples of applications of the theories I
was learning. I never even bothered to use the forums.

The workload was less, but engaged me more. 2 quizzes were easy,
and boosted confidence. The final assignment was brilliant learning
design, allowed me to review and apply all the theoretical models to
2 new cases. Reading others’ work further extended the learning,
questioning my answers in the light of other different ones.

I got 91%, reduced to 73% for the lateness of my quiz attempts. Can’t
wait to get my certificate. I’ll be putting this on my CV if I decide to
apply for formal study in the area.”

Anybody done one like this?
• Notes: At the session there were a number of colleagues who
had enrolled and/or completed a Coursera Mooc
– One did a course in Astrology
• Very active teacher, regular emails even after the course
is over, inc offering to sign completion signatures if
attending same conference
• 6 hours video lectures per week, hard to digest, did not
work on mobile phone
• Moderators keen for answers not to be shared too early
in the forums, would often delete posts
• Overall positive experience however

– Another colleague did a course in Environmental issues
• Very large cohort from all over the globe
• Forums overwhelming
• At one point the forums were “hi-jacked” by environmental
lobbiests promoting a particular couse
– In another example of a programming Mooc, a special
interest group of Python programmers were enlisted as
volunteer teaching assistants in the online forums
– New Stanford-based Mooc provider was noted by a
colleague http://class.stanford.edu
• Separate to Coursera and Venture Lab

PH100 Intro to Physics (Udacity)
Anytime
(asynchronous)

Realtime / live
(synchronous)


Tags let you
Open
filter/find
forums
particular
topics

Badges identify
people/roles

• Length of course: 7 weeks of content, but you can take whatever time you
like to do it
• Timing: anytime
• Blended cohort: not promoted as far as I can see
• Synchronous virtual events: weekly
• Media resources: mostly new, high-def video
• Assessment for certification: single/final exam
• Formative assessment: forums/lecture quiz/problems
– problem-based learning philosophy
– great web platform puts real time and anytime learning
tools together in one place,
– links students and CVs to employees to help students get
jobs (IT/computing science fields at this stage)

• Notes: Nobody at the 11/12/12 staff forum had done a
Udacity Mooc

Smooc for new uni teachers (JISC, HEA, Oxford Brookes)

Course hub is a
blog

Optional
Moodle
forums

Welcome audio
(Soundcloud)

Smooc for new uni teachers (JISC, HEA, Oxford Brookes)
• Level: probably equivalent to 1st year Education course
• Timing: not related to F2F course
• Blended cohort: not promoted as far as I can see
• Media resources: Mix of new and re-used
• Assessment for certification:
– 150 register for the Moodle site
– 60 logged in and participated
– 28 created their own blog to document their progress and
had this linked/aggregated to the course blog
– 12 completed the assessment (project)
• Formative assessment: forums/lecture /problems
– Students are new university teachers
– Short time frame

POT2012 Program for Online Teaching

Final
assessments
shared

Virtual
certificate
presentation
night

POT2012 Program for Online Teaching
• Length of course: 2 x 12 week session ie 1 year
• Level: probably equivalent to 2nd-3rd year Education course
• Timing: not related to F2F course
• Blended cohort: no
• Media resources: Mix of new and re-used
– Not sure how many enrolled
– 60 logged in and participated
– 16 completed the assessment (project presentation)
• Formative assessment: forums/lectures /problems
– Students are university teachers
– Long time frame, depth of learning

• Notes: Nobody at the 11/12/12 staff forum had done a
smaller-scale Mooc or SMooc

Designing a New Learning Environment
(Stanford, Venture Lab)
• Cedir Learning Designer Lesley Knapp is enrolled
• Interview at http://blogs.uow.edu.au/openuow/
• Much smaller than Coursera (c1000 enrolled)
• Groupwork is central to the subject design (and the
Venture Lab learning platform)
– Develop a new learning environment
• Following early task that describes a learning
problem

OERU pilot Regional Relations in Asia and the Pacific (USQ)
• Length of course: 1 session
• Level: 1st year
• Timing: same as campus cohort
• Blended cohort: not sure
• Synchronous virtual events: 0
• Media resources: New video signposts to be created
– 3 essays and reflective journal
• Formative assessment: students find/gather/share and analyse
their own materials
– Students find resources (‘discovery pedagogy”
rather than digesting materials found for them

OERU pilot Mathematical Journey (Empire State College, UNY)
• Length of course: 1 session
• Level: 1st year/Bridging
• Timing: same as campus cohort tbc
• Blended cohort: tbc
• Synchronous virtual events: Skype tutorials re final project
• Media resources: Some new plus written resources, podcasts
and other media (all OERs) - indexed for easier access
– Learning journal and ePortfolio assessed
• Formative assessment:
• Pre-test leads to learning plan (inc project)
• Interaction with peers via ePortfolio throughout course
– Individual learning plans and projects

International development in open learning

CRACKING FOR CREDIT
ASSESSMENT

Blended: Moocs + local colleges
• “EdX's popular introductory computer science course from MIT will
provide the backbone of a class at the community college -- a key
gateway to degree programs -- with supplemental teaching and help
from community college faculty on the ground.
• This is where the rubber meets the road for transforming higher
education. Community colleges are beset by waitlists (400,000 in
California alone) and bottlenecks in important introductory courses,
as well as low success rates.
• If scaled-up MIT-quality teaching can help with solve those problems,
MOOCS could be truly revolutionary. Massachusetts Bay Community
College president John O'Donnell calls edX an invention comparable
to Gutenberg's printing press.”
• “New frontier for scaling up online classes: credit”, The Australian,
Higher Education, Nov 20, 2012 (sourced online 10/12/12)
Re-use of materials between
consortium partners for local
assessment part of the OERU
model since its conception

Assessment, cheating, identity
• “Multiple-choice online quizzes are simple enough, but on more
open-ended assignments, MOOC students now are mostly grading
each other's papers. When they have questions, they're mostly asking
fellow students.
• "Crowd-sourced assessment" raises obvious questions. MOOC leaders
are exploring artificial intelligence solutions but admit many aren't
fully baked. …
• Cheating's another problem that suddenly matters with credit at
stake. EdX is working with a testing company to arrange for proctored
exams in centres around the world. Coursera says it will be easier for
far-away students to let them wave an ID card and take a test in front
of a webcam, proctored from afar. MOOCs won't offer those things
for free. But they could cost much less than, well, the full string
quartet.”
• “New frontier for scaling up online classes: credit” , The Australian,
Higher Education, Nov 20, 2012 (sourced online 10/12/12)

External review, accreditation
• “Last week, Coursera, which offers classes from 34 universities, announced the
American Council on Education would begin evaluating a handful of Coursera
courses and could recommend other universities accept them for credit (individual
colleges ultimately decide what credits to accept). Antioch University, Excelsior
College and the University of Texas system are already planning to award credit for
some MOOCs. …
• Molly Broad, president of the American Council on Education, said MOOCs are
promising, but her group will send faculty out to "kick the tires" and research
whether online courses enrolling 150,000 can really be credit-worthy. They'll talk
to both students who complete and those who drop out (at edX, 80 to 95 percent
who sign up don't finish the work).
• A likely outcome is more blended models like the Massachusetts experiment,
where MOOCs provide the backbone and resources local institutions can't offer,
but local institutions still handle the one-on-one and award the credit.
• Such models could be "the best of both worlds," said Coursera co-founder Daphne
Koller. Versions are already in places as varied as San Jose State and the National
University of Mongolia.”
• “New frontier for scaling up online classes: credit “, The Australian, Higher
Education, Nov 20, 2012 (sourced online 10/12/12)

Degree: a passport stamped by many
• Date: Sunday 12 August 2012
Publication: The Age
Title: Free courses from world's top unis a swipe away in online
revolution
• Imagine a university degree that is like a passport: a subject from
Swinburne stamped alongside another from Sydney University, with
courses from overseas colleges such as Stanford or Harvard thrown in.
You could earn your degree without travelling further than your laptop,
and far more cheaply than on campus.
• Coursera co-founder Andrew Ng, a Stanford professor, told The Sunday
Age that although the venture has committed to not becoming a
university, already some colleges are offering course credits for Coursera
certificates. …''What we have seen also is there are a few universities that
are incorporating some of our material into their own classrooms,'' he
says.

What are we doing to move towards open-learning?

IDENTIFY AND FIND RESOURCES
FOR PILOTS
We have a shortlist of subjects that would
work well in the open, and are talking to
Faculties about workload, timing,
development.

Looking at pilot and ongoing resourcing, what
roles needed, eg student media developers
from CACS302 Creative Arts interns.

Want to know more? Follow our blog

Public blog http://blogs.uow.edu.au/openuow

Moocs, learning design and assessment 12/12/12

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Moocs, learning design and assessment 12/12/12

Similar to Moocs, learning design and assessment 12/12/12 (20)

Recently uploaded

Recently uploaded (20)

Moocs, learning design and assessment 12/12/12