A presentation given to members of the School of Modern Languages at Durham University, exploring different ways of providing student feedback using an online learning environment such as Blackboard.

1. New Options for Online Student Feedback Learning Technologies TeamInformation Technology Service

2. Learning Technologies Team Judith Jurowska Faculty of Arts & Humanities Martin Edney Faculty of Science Julie Mulvey Faculty of Social Sciences & Health Dr Malcolm Murray Learning TechnologiesTeam Leader

3. Session outline Voting systems – Julie Mulvey Excel Spreadsheet – Judith Jurowska Blogs – Christine Bohlander & David Tual GradeMark – Judith Jurowska Audio/video – Martin Edney Preview: Feedback Tool – Malcolm Murray

4. New Options for Feedback Using Voting Systems Julie Mulvey

5. Voting systems Audience Response System (ARS) www.keepad.co.uk TurningPoint plug-in to Microsoft PowerPoint

6. Voting Systems Research by Marina Sawdon from Phase 1 Medicine in using an Audience Response System Assess prior knowledgeBransford 1999 Question asked before lecture starts ‘Learning occurred’ Same question asked at the end of lecture Knowledge retention/ decay at different time points Question asked again at 1 week and 4 weeks after lecture

7. Voting Systems

8. Voting Systems Evaluation The audience response system gives me feedback on my progress 99% student satisfaction The audience response system aids my knowledge recall 98% student satisfaction The audience response system consolidates my knowledge 98% student satisfaction

9. Voting Systems “Use of KEEpads was excellent, getting instant feedback is a good learning tool” “I liked the KEEpad system because it allows you to answer questions without anyone knowing if you got it wrong or not.” “I liked the use of the KEEpads, it helped break up the lectures and gave feedback” “KEEpad use…gave instant feedback and kept me focused and immediately applying the lecture content to new situations”

10. Voting Systems Marina’s Conclusions: Improves and facilitates learning Increases student participation “…it wakes me up!!!” Improves knowledge retention Instant feedback Low cost Students love it! Sawdon, M. 2009. Improving Knowledge Retention using KEEpad. Medical Education43(5): 487-487

11. Voting Systems KEEpad only one type of voting system PollEverywhere.com Online system Informal SIG at Durham ESTICT (Engaging Students through In-Class Technology) (http://estict.ning.com)

12. New Options for Feedback Automated Excel Spreadsheet Judith Jurowska

13. Frederico Federici Spreadsheet

14. Dr F M Federici Spreadsheet

15. Creating criteria

16. Entering the scale

17. Pulling the comments together

18. Enter Candidate details

19. Printing the report

20. Finished student sheet

21. Results Summary Marking time reduced by 60% Setup time was considerable. (once only) Double marking was possible Second marker was impressed by the volume of feedback provided Needs further work before it can be used widely.

22. Liverpool John Moores UniversityDr Phil Denton

23. QED article by Kevin Hinde QED Issue 8 Michaelmas Term 2008

24. New Options for Feedback Using Blogs for feedback on written work Christine Bohlander & David Tual Centre for Foreign Language Study

26. Students’ follow-up on feedback

27. Repetitive feedback to different students

28. Collecting work, keeping records, photocopying/filing

29. Ask student to re-write and resubmit

35. We feedback on their work using the following colour code: - ‘slips/lapses’, ‘first-order mistakes’ and ‘second-order mistakes’ - correction of ‘errors’ - word(s) or clusters to be deleted (because wrong or corrected in green)

39. Students’ follow-up on feedback

40. Repetitive feedback to different students

41. Collecting work, keeping records, photocopying/filing

42. Ask student to re-write and resubmit

44. Students’ feedback

45. Students’ feedback

47. Emphasis on learning process and monitoring of progress

50. They can also post comments and

54. The opportunity to be able to go back for amendments and revision is important for me.

55. I can learn from reading my peers’ work and the tutor’s feedback on their work.

57. A continuous process of learning with easy monitoring

61. detecting text matches against a student database, journals and the internetWhat is GradeMark? GradeMark provides online marking and feedback

62. Create a GradeMark Assignment Create a Turnitin assignment(you can turn off the plagiarism check if necessary) Mark using GradeMark’s functions: Rubrics Comments ‘Quick marks’ Composition Format Punctuation Usage

63. How can GradeMark help? save time marking provide in context, legible feedback to your students provide consistent feedback for large numbers of assignments manage the marking process where there is more than one marker build a reusable comment resource

64. Case Study – Steve Lyon Steve Lyon Senior Lecturer : Department of Anthropology

65. Context 1st Year Social Anthropology course Purpose (aims and objectives) Assess understanding of the topic Improve essay writing skills Teaching method lecture, seminars formative essay with feedback Resources: Postgraduate teachers Marking rubric GradeMark software (available through Turnitin in duo) Final assessment - summative essay

66. Rationale …when we started doing this [on paper] we had students coming and saying, “I got lots of comments and so and so only got two little lines.” I wanted somehow to, not impose, but encourage a more consistent amount of feedback for everyone and ensure similar things were being flagged up. Quote from Steve Lyon

67. Rubric Criteria (6) Answer: Does the essay answer the assigned essay question? Sources: Does the essay properly cite all sources? Are all cited references included in the bibliography? Logic: Does the answer, build a demonstrable argument based on credible and appropriate evidence? Style: Is the essay articulate and coherent? Does it use appropriate language and vocabulary for the purpose? Is it too chatty and informal? Is it unnecessarily complicated? Spelling: Are there spelling and grammatical errors? Do these impede communication of the ideas? Presentation: Is the bibliography correctly formatted? Is the text in some unusual font? Are the margins correct

68. Rubric weightings 5 weightings for the 6 criteria Poor (10) Fair (49) Good (59) Very good (69) Excellent (80) Grade by clicking the boxes in the rubric. the maximum for each criteria is capped at 80% can manually give more for something beyond the criteria.

69. Results: Staff perspective Saved time for markers Fatigue is less of an issue… The twentieth bad essay no longer invokes lots of exclamation marks, ‘What is this!’ Paraphrased from Steve Lyon QED article

71. The ‘marking rubric’… helps automate the process of allocating marks against set criteria. This was particularly helpful to the teaching assistants.Paraphrased from Steve Lyon QED article

73. A ‘rubric’… helps automate the marking process

74. GradeMark allowed Steve to manage this process ensuring a higher degree of quality control and consistency despite the distributed approach to teaching and marking.Paraphrased from Steve Lyon QED article

75. Results: Student Perspective Improved feedback The student gets at least six comments even if the postgraduate marker [is] less confident… Students really like it especially as I give them the rubric before the assignment so they know what they are working to. Students like being able to get the feedback online. They get more feedback … the essays will be riddled with these little comments that are specific to a paragraph Paraphrased from Steve Lyon QED article

76. QED article Lyon, Steve. “Making the grade: Helping postgraduate teaching assistants with their marking and feedback..” QED (Durham University), Michaelmas term 2008.

77. Example Rubric

78. Comments options

79. Quickmarks

80. Accessing GradeMark

81. Accessing GradeMark

82. How it works

83. How it works - Markup

84. How it works – Student View

85. Click on the Rubric to Mark

86. New Options for Feedback Audio/Video Martin Edney

87. Audio / Video Feedback What? Lecturer records feedback as sound file video file (talking head) video file (screen capture of submitted work, with audio commentary)

88. Audio / Video Feedback Why? Students like it – they report They get more feedback They get quicker feedback “it’s not face to face but it’s certainly one to one” Speed (as quick as or quicker than writing)

89. Audio / Video Feedback Why (continued)? More effective Students pay attention Tone of voice / inflection Allows you to talk direct to student work

90. Audio / Video Feedback How? Record audio / video file e.g. use headset, digital dictaphone, Camtasia, Jing one file per student Deliver file to student duo Grade Centre – feedback file option shared area (e.g. duo Files tab) email

91. Audio / Video Feedback Tips Individual feedback to each student + generic feedback to group Look through submitted work before you start recording Say student name or number at the beginning Summary / mark at end of recording

92. Audio / Video Feedback Some feedback from students I think this is a much better system than paper feedback. I know I got a lot more from it than if you had just had to tick boxes and given me a comment in that little box on the piece of paper. It was also much easier to use than anticipated, and I definitely think you should recommend this to other lecturers I found the feedback for the formative much better than 'normal' written feedback. (reported by Ian Greener, Applied Social Sciences, Durham University) Students from Sheffield Hallam & Chester giving their opinion on audio feedback at a MEL-SIG event in Glasgow [sound file link]

93. Examples Audio example Comment on draft dissertation work Sound file available from “A Word In Your Ear 2009” website. Part of Davies, D. Rogerson-Revell, P. & Witthaus, G. An exploratory study of speech styles in audio feedback to M- level students Presented at “A Word In Your Ear 2009”, see http://research.shu.ac.uk/lti/awordinyourear2009/papers.html

94. Examples Video example English for Academic purposes course Russell Stannard, University of Westminster Video file available from http://www.russellstannard.com/king/king.html For full details, see http://www.english.heacademy.ac.uk/explore/publications/casestudies/technology/camtasia.php

95. Jing for video feedback David Tual

97. It takes no space on my J: drive, and there is no need to email or post big files on DUO.

98. It can be accessed from any pc with an internet connection.

99. Fast and easy to send to the students.

101. New Options for Feedback Preview: Future Feedback Tool in duo Malcolm Murray

102. Future Feedback tool A work in progress...

103. Future Feedback tool

104. Select Assignment

105. File Naming Policy

106. Feedback Template

107. Feedback Template - Excel

108. Feedback Template - Text

109. Downloading Files

110. Student Work & Summary

111. Marking & Feedback

112. Marking & Feedback

113. Marking & Feedback

114. Individual Feedback Files

115. One per Student

116. Bulk Upload

117. Grade Centre

118. Contact If you would like to know more or have any questions please contact us via itservicedesk@durham.ac.uk