Student feedback is a hot topic in higher education, with students demanding more of it, quicker. This session discusses a project that attempted to define the concept of feedback from both a student and faculty perspective and then develop workflows and possible extensions to Blackboard to improve the creation, delivery and learning from feedback.

1. RethinkingStudent Feedback 13th July 2011 3:00 pm Venetian Congress Center - Titian 2206 Dr Malcolm MurrayDurham University, UK

2. Acknowledgement Much of this presentation draws on conversations with and research by my colleagues Janet Lavery and Judith Jurowska janet.lavery@durham.ac.ukj.e.jurowska@durham.ac.uk

3. Slides freely available http://www.slideshare.net/malcolmmurray

4. Recorded Delivery http://www.audioh.com/projects/recorded_delivery.html

5. Session Plan Perceptions of Students Why give feedback? What we’ve tried What staff want What students want What next?

6. http://www.flickr.com/photos/rightreading/1243239706/sizes/m/in/photostream/ Perceptions of Students

7. Student as Consumer Ham KhanTimes Higher Educational Supplement14th December 2007

8. Education by Numbers http://www.bbc.co.uk/news/education-13874483

9. Student as Participant in anAcademicCommunityof Practice http://knowledgemanagement-review.blogspot.com/2011/02/knowledge-management-and-community-of.html

10. Implications Consumers Focus on the end result Never look back Participants Learning the rules Want to gain acceptance

11. http://www.flickr.com/photos/fin5bjh/4442706561/sizes/l/in/photostream/ Why Feedback?

12. Five Functions of Feedback Learners can use feedback to: confirm add to overwrite tune restructure information in their memory Butler & Winne (1995)

13. Feedback is a Dialogue Feedback on assessment whether formative or summative is a personal dialogue between a lecturer and a student about the student’s assessment and aspirations. It is not necessarily a face-to-face discussion, but it is a dialogue. Janet Lavery

14. Feedback is about improvement “Feedback that does not tell you how to improve is pointless” said one of the students in a focus group to the roar of approval from the other students. For improvement to occur feedback has to be personal, i.e. in context of the specific assessment, identify strengths and weakness in the assessment, and provide insights into how the student can improve in time for the next assessment. Janet Lavery

15. Laurillard’s Conversational Framework http://www2.smumn.edu/deptpages/~instructTech/lol/laurillard/index.htm

16. Effect of Modularization 59% of students responded that feedback was given too late to be helpful, as they got it after the end of the module Hartley & Chesworth (2000)

17. Speaking Different Languages “students who do not yet share a similar understanding of academic discourse as the tutor would… …have difficulty in understanding and using the feedback” Melanie Weaver (2006)

18. Feedback by Larson

19. What we have tried

20. Tablet Annotation

22. Don't have to fiddle with envelopes

23. Admin processing time is significantlyreduced

24. You can type comments on the work as well so you don’t have to write the same thing out again and again

25. Rubbing out your writing is easy

26. The output looks quite professionalDr Grant Ingram School of Engineering & Computer Sciences

28. Zooming needed to get good quality handwriting - makes getting an overview of the work harder

29. The quality of your handwriting is much poorer than with an ordinary pen

30. A great deal of electronic fiddling: concatenating the marking matrix, naming the PDF files correctly, converting from Word format for students who don't follow instructions and so on.

32. Rubric Deployed

33. Dr Steve LyonAnthropology Rationale “…when we started doing this [on paper] we had students coming and saying, “I got lots of comments and so and so only got two little lines.” I wanted somehow to, not impose, but encourage a more consistent amount of feedback for everyone and ensure similar things were being flagged up.”

34. Staff Feedback Fatigue is less of an issue… The twentieth bad essay no longer invokes lots of exclamation marks, ‘What is this!’ The ‘marking rubric’… helps automate the process of allocating marks against set criteria. This was particularly helpful to the teaching assistants. Lyon, Steve. “Making the grade: Helping postgraduate teaching assistants with their marking and feedback..” QED (Durham University)2008.

36. Students really like it especially as I give them the rubric before the assignment so they know what they are working to.

37. Students like being able to get the feedback online.

39. Student Voice “it’s not face to face but it’s certainly one to one” I think this is a much better system than paper feedback. I know I got a lot more from it than if you had just had to tick boxes and given me a comment in that little box on the piece of paper. http://www.flickr.com/photos/jiscimages/436585751/sizes/m/in/photostream/

41. Staff edit the posts, marking mistakes in red and any corrections in green.

42. Students are invited to correct their mistakes in blue(crossing out the original mistakes, but not deleting them) and to look at the rest of the corrections

44. Staff Rationale “the previous year I was disappointed because students were doing their translations uncritically, without thinking or looking at the feedback.” Alia Moser “With the blog you have the work and feedback online and we can look back and say, “Look, we have mentioned adjective endings before, so why is it still going wrong?” Students can look back at areas where they did well or were weak. This helps build confidence. It was important to make them feel we had a record they could be proud of.” Katherine Griffiths

45. Peer Feedback: Blogs It ensures corrections are completed and are easily re-marked – cuts out time, and if this had been hand written + marked, I would probably put it straight in my file without bothering with corrections!

46. Student Feedback: Blogs I prefer posting my written work on the blog to handing in handwritten work because of its positive effect on my learning. The opportunity to be able to go back for amendments and revision is important for me. I can learn from reading my peers’ work and the tutor’s feedback on their work. I am happy for my peers to comment on my work. 80%of students agreed

47. Understanding how Students work

48. Summary of findings

49. Feedback – staff Find simpler means to development and deliver feedback online The current mechanisms for developing and delivering digital feedback (text, audio, etc…) are according to many staff: “inefficient”, “require too many steps”, and generally too difficult. Staff want access to sophisticated feedback generation tools that are easy to use and provide simple delivery mechanisms.

50. Feedback - students Praise for feedback limited to that ‘like school’ or ‘what we are used to’. More unhappiness then happiness with the actual content of feedback. Enthusiastic about receiving feedback developed using new technologies such as audio recordings or digital mark-up systems. Students very happy with their current feedback were noticeably less enthusiastic about the possibility of new technologies – prefer handwritten comments on a copy of the essays and face-to-face discussions.

51. http://www.flickr.com/photos/techsavi/3946238357/sizes/o/in/photostream/ Next Steps

52. Potential Feedback Tool

53. Complications – Naming Policy

54. Choice: Feedback Format

55. Simplifying Download

56. Simplifying Mark Entry

57. Simplifying Upload

58. Mobile Marking

59. My Grades 2/10 Blackboard -Must try harder…

60. Feedback Dashboard

61. Simple, but Consistent Look at the processes for submission & feedback Push for standardization in the lecturer’s experience in the student’s experience

62. Conclusion

63. Rubric cubism (sorry) feedback [is] an essentially problematic form of communication involving particular social relationships… …external conditions interplay, mediate (and are mediated by) patterns of power, authority, emotion and identity Higgins, Hartley & Skelton (2001) http://www.forevergeek.com/2009/09/rubiks_cube_fancy_dress_costume/

64. Findings so far… Creative ways exist for staff & peers to provide feedback Most have ugly workflows, discouraging wider adoption No one-size fits all – (5 purposes of feedback) Need to make things better without breaking those that already work Electronic delivery can make feedback more visible, thus more likely to result in learning Must provide/promote timely feedback We need to learn how our students could use feedback to feed forward…

65. Please provide feedback for this session by emailingBbWorldFeedback@blackboard.com. The title of this session is: Rethinking Student Feedback

66. Get in touch: malcolm.murray@durham.ac.uk @learntechdurham @malcolmmurray http://www.dur.ac.uk/lt.team/blog/ Questions?