Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Peer Feedback in an Online Environment

205 views

Published on

Online peer feedback

Published in: Education
  • Login to see the comments

  • Be the first to like this

Peer Feedback in an Online Environment

  1. 1. Running head: CRITICAL REVIEW OF PEER FEEDBACK Critical Review 2: Using Peer Feedback to Enhance the Quality of Student Online Postings: An Exploratory Study Jennifer Chang Wathall Student ID 663027 Central Michigan University EDU 800 Instructor: Dr. Michael Dennis Deschryver
  2. 2. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 2 Problem (about 1 page) 1. Identify the clarity with which this article states a specific problem to be explored. This article somewhat clearly stated the problem to be explored which was to investigate peer feedback to enhance the quality of the postings in an online learning environment. To achieve this, the authors investigated students’ perceptions about the value of giving and receiving peer feedback in an online course and the quality of student online postings. These postings were judged using Bloom’s taxonomy from pre-course to post- course. 2. Comment on the need for this study and its educational significance as it relates to this problem. Many studies have looked at the impact of peer feedback in a traditional classroom setting; however, there are relatively few studies conducted to look at the effects of peer feedback in an online environment. Research has shown that the quality of feedback can be improved through the use of constructive feedback providing it is prompt, consistent, and ongoing (Ertner & Stepich, 2004). Both Lynch (2002) and Pallof and Pratt (2001) reported that a lack of feedback in an online environment can lead to higher student dropout rate and a sense of disconnect from the course. Furthermore, Cunningham (1992) reported that interactions in an online environment is crucial to promote learning within the community. Lang (as cited in Black, 2005) emphasized the importance of discourse to improve understandings of a topic or issue. According to Black (2005), online discussions are often at a superficial level with little critical analysis or higher order thinking. This study looked at peer feedback as a way to promote higher order thinking by encouraging students to give feedback to each other about the quality of their postings. The need to investigate how peer
  3. 3. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 3 feedback can enhance the quality of online posting is educationally significant for the reasons stated above. 3. Comment on whether the problem is “researchable”? That is, can it be investigated through the collection and analysis of data? Both quantitative and qualitative data was collected to investigate students’ perceptions on the value of feedback and the quality of the postings. Students perceptions of peer feedback in online courses is researchable, and surveys were conducted to collect qualitative data. The quality of the postings was measured against a numeric scale using Bloom’s taxonomy and quantitatively measured using t-tests. Theoretical Perspective and Literature Review (about 3 pages) 4. Critique the author’s conceptual framework. The authors’ conceptual framework is based on several well- researched and established learning theories such as Vygotsky’s (1978) Zone of Proximal Development (ZPD). ZPD describes learners’ understanding as resulting from inter-personal interactions where students are given opportunities by the instructor to construct their own meaning of the content through these interactions—in other words, providing a constructivist environment. This is aligned with other perspectives that support the authors’ conceptual framework and includes Nicol and Marfarlane’s (2006) seven functions of good feedback, which are: · Give specific and clear goals and criteria · Provide opportunities for self-reflection and the developing self-assessment · Give quality comments about student learning · Encourage peer and teacher discourse · Be positive and supportive in your comments that motivate learners
  4. 4. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 4 ·Provide clear directions to students so they know how to close the gap between current achievement and goals · Utilize formative assessment to inform instruction Additionally, Cunningham (1992) discussed the importance of interaction in an online environment to promote learning while Notar, Wilson, and Ross (1992) reported that feedback in an online environment should be “diagnostic and prescriptive, formative and iterative, and involving both peers and group assessment” (p. 646). Ertmer et al., (2007) have reported that the quality of student discussions in an online environment can be increased by “the use of constructive feedback that is prompt, consistent and ongoing” (p. 414). This implies that online feedback may promote higher order thinking by improving quality of postings in peer feedback. This study also used the conceptual framework of Bloom’s taxonomy to assess the quality of online postings by the students. Liu, Lin, Chiu, and Yuan (2001) suggested when students are asked to provide peer feedback in an online environment they engage in higher order thinking by going beyond cognitive processes required for a task. The final part to the authors’ conceptual framework includes the advantages of using peer feedback in terms of community building and student perceptions about the value of giving and receiving peer feedback (Corgan, Hammer, Margolies, & Crossley, 2004). Pallof and Pratt (1999) reported that giving feedback is not a naturally acquired skill. Students often experience anxiety about giving and receiving feedback so clear guidance should be provided. 5. How effectively does the author tie the study to relevant theory and prior research? Are all cited references relevant to the problem under investigation? The authors cited references and research about the following:
  5. 5. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 5 • the importance of interaction in online courses by Cunningham (1992). This research is relevant to the problem under investigation since peer feedback can promote more online interaction. • the value of discourse and whether online or face to face, can lead to “increasingly sound, well grounded, and valid understandings of a topic or issue” according to Lang (as cited in Black, 2005, n.p.). This research is relevant to this exploratory study as this refers to the quality of the postings as a way to reflect improved understandings as a result of feedback. • many online discussions according to Black (2005) lack critical analysis or higher order thinking and mainly relate to sharing and comparing information on a superficial level. This study used peer feedback to promote higher order thinking where students provided feedback to each other. This related to the quality of the postings with the goal to encourage students to learn and co-construct knowledge and understanding (Roehler & Cantlon, 1997). • Nicol and Marfarlane’s (2006) seven functions of good feedback are mentioned and listed in the section above. • According to Swartz and White (as cited in Mory, 2004), students expect feedback to be: 1) prompt, timely and thorough 2) ongoing including summative and formative 3) constructive, supportive and substantive 4) specific objective and individual 5) consistent. This study ties in the need for peer feedback to support the five expectations as an instructor alone, would be struggling to meet all of these. • Ko and Rossem (2001) reported that a lack of feedback can result in learners feeling disconnected from a course. The authors did not discuss the role of feedback to build
  6. 6. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 6 a sense of community in their actual investigation but this research may be indirectly related to students’ perceptions about giving and receiving feedback. • Ertmer and Stepich (2004) found “that the quality of student discussion responses can be increased through the use of constructive feedback that is prompt, consistent, and ongoing.” (as cited in Ertmer et al., n.p.). The research study supports the authors’ investigation about whether peer feedback can increase the quality of student online postings. 6. Does the literature review conclude with a brief summary of the literature and its implications for the problem investigated? There is a very brief summary on page 416 stating the fact that there is limited research on using peer feedback as an instructional strategy in an online learning environment and that very few studies have looked at peer feedback as a way to improve the quality of online discourse. The problem investigated, therefore, was an attempt to fill this gap in research by investigations students’ perceptions on the value of feedback, and to see whether peer feedback can enhance the quality of students’ online postings. There is no brief concluding summary of the literature review. 7. Evaluate the clarity and appropriateness of the research questions or hypotheses. Three research questions were addressed in this case study, and they were clearly stated as: RQ1: What is the impact of peer feedback on the quality of students’ postings in an online environment? Can the quality of discourse/learning be maintained and/or increased through the use of peer feedback? This research questions directly relates to the problem being investigated which is whether the quality of student online postings can be improved with the use of peer feedback.
  7. 7. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 7 RQ2: What are students’ perceptions of the value of receiving peer feedback? How do these perceptions compare to the perceived value of receiving instructor feedback? I am not sure how relevant this is to investigating the role of peer feedback in enhancing the quality of online discussions. Students’ perceptions are important, however, I cannot see the connection to how this research question directly relates to main issue discussed which was to investigate whether peer feedback can improve the quality of online postings. RQ3: What are students’ perceptions of the value of giving peer feedback? Again, I am not sure how this question is directly related to investigating improving the quality of online postings and promoting higher order thinking in an online environment. There are only a couple of research articles cited in this article that relate to student perceptions on feedback, which are described by Topping (1998) and Arbaugh (2000). Research Design and Analysis (about 3 pages) 8. Critique the appropriateness and adequacy of the study’s design in relation to the research questions or hypotheses. This study represented a case study framework with the use of qualitative data, however, the study also collected quantitative data and used an experimental research design (Randolph, 2007). According to Yin (2003), “a case study is an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between the phenomenon and context are not clearly evident” (p. 13). Several sources of evidence are normally collected through direct observation, interviews for example, and then analyzed through pattern making and explanation building. In this study students were given feedback by the instructor for the first half of the course so students could see examples of quality feedback. In the last half of the course, students were required
  8. 8. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 8 to give peer feedback to two discussion posts and answer one discussion questions themselves. Three research questions were addressed: 1. What is the impact of peer feedback on the quality of students’ postings in an online environment? Can the quality of discourse/learning be maintained and/or increased through the use of peer feedback? The case study framework is well suited to address this question. The experimental design for the collection of quantitative data used a numerical score based on Bloom’s taxonomy, as well as descriptive comments that supported the assigned score. 2. What are students’ perceptions of the value of receiving peer feedback? How do these perceptions compare to the perceived value of receiving instructor feedback? Data collected here was based on participant interviews and responses to entry and exit survey questionnaires which included 13 Likert- style items and 5 open-ended questions. 3. What are students’ perceptions of the value of giving peer feedback? Again, data collected here was based on participant interviews, and responses to entry and exit survey questionnaires which had a combination of both quantitative and qualitative data. 9. Critique the adequacy of the study’s sampling methods (e.g., choice of participants) and their implications for generalizability. The sample of 15 graduate students that was selected did not represent a random sample and is considered too small to be representative of the population. Most recommendations for qualitative data collection that will undergo statistical analysis is to have at least 30 data points (Morse, 1994). This small sample cannot be taken to be representative of the entire population in terms of generalizability and also could be described as a “convenience sample” as participants were all graduate students enrolled in
  9. 9. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 9 the same online course (Given, 2008). This study used a t test, which has a specific calculation for the recommended sample size that involves looking at standardized effect sizes (Hulley et al., 2013). This computation was not used to calculate the recommend sample size for this particular study and may lead to low statistical power. 10. Critique the adequacy of the study’s procedures and materials (e.g., interventions, interview protocols, data collection procedures). To collect quantitative data about the quality of student online postings, two researchers rated the students’ postings separately and then verbally discussed their scores. The raters looked at 59 different postings and there was an 86% agreement between the two raters which reflects a high level of agreement. The authors used a paired sample t test to compare the average scores obtained prior to peer feedback (weeks 3-5) and the scores after peer feedback (weeks 7-13). T tests were also employed to compare students’ rating of the value of peer and instructor feedback from the pre and post surveys. Participants perceptions about the value of peer feedback used data from open ended survey questions and interviews. Participants interviews were conducted to obtain qualitative data about students’ perceptions about the peer feedback process. 11. Critique the appropriateness and quality (e.g., reliability, validity) of the measures used. Reliability of the measures used refers to the consistency of the measurements and if the experiment was to be repeated whether the experiment would yield the same results. There are some reliability issues in terms of the small number of participants. This sampling problem can cause a random error in the measurements and data collected and if this experiment were to be repeated, it could yield completely different results. The most
  10. 10. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 10 common measure of reliability is using the reliability coefficient and this study did not include this calculation to check for reliability (Rosnow & Rosenthal, 1991). Typical methods in social research include: test-retest, alternative forms, split halves, inter-rater reliability and internal consistency. There is no mention in the study of measuring the internal consistency using Cronbach’s alpha coefficient (Cronbach, 1951). The first two limitations of the small sample size of 15 graduate students used and the short timing of the course all call into question the reliability of the results. Validity is concerned with how well does a test measure what the researchers have set out to measure and can relate to the credibility of qualitative research. There are four types of validity: statistical conclusion validity, internal validity, construct validity and external validity (Drost, 2011). The limited scale of Bloom’s taxonomy used to measure the quality of students’ posting, may affect the validity of the results. Did the limited scale actually measure the quality of the feedback postings? The authors stated they had a relatively high degree of face validity (within construct validity) but then contradict themselves in the conclusion by stating this was one of the limitations of the study. This study is lacking construct validity, which refers to how well an idea had been transformed or translated into a “functioning and operating reality” (Drost, 2011). Did the limited scale used actually measure the quality of the feedback postings? How do we know Bloom’s taxonomy for scoring the quality accurately measured higher order thinking and the quality postings? C. Interpretation and Implications of Results (about 3 pages) 13. Critique the author’s discussion of the methodological and/or conceptual limitations of the results. The authors reported the results and limitations as following:
  11. 11. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 11 • At the beginning of the course, feedback in the online environment was found to be slightly more important than traditional settings and students reported the feedback should be timely and high quality • Students were not so concerned about the quantity of feedback. • By the end of the course, students’ perceptions about the importance of giving and receiving feedback significantly increased. • Students reported having concerns about giving feedback in terms of consistency, fairness and not being qualified to give feedback. • A paired t test revealed no significance difference in terms of the quality of students’ postings either with instructor or peer feedback. Even though there was no increase in quality, authors pointed out there was no decrease in the quality of student postings, suggesting that peer feedback can maintain the quality of postings once a level has been reached. This could be a benefit to the instructor in terms of saving time and reducing work load. • Another limitation mentioned by the authors is the actual discussion questions used in the course. Many of the questions were devised by student leaders and did not elicit a lot of higher order thinking, so ratings for responses could have scored low on Bloom’s taxonomy due to this reason. In summary, discussion questions did not encourage responses that connected to the higher levels of thinking which involve analysis, synthesis or evaluation. • Many students also started off with relatively high scores for the quality of their postings, which leaves very little room for improvement.
  12. 12. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 12 • Data collected from interviews suggest that students did use peer feedback information to improve the quality of their postings. • Instructor feedback was perceived to be more valuable than peer feedback and this has been reported by several researchers such as Ko and Rossen, (2001) and Topping (1998). The three limitations mentioned include: • the small sample size of 15 graduate students used • the short timing of the course • the limited scale used to assess the quality of the student postings The limited scale used Bloom’s taxonomy to assess the quality of postings resulted in several students reporting the taxonomy was difficult to apply, too vague and not appropriate to use at a graduate level. 14. How consistent and comprehensive are the author’s conclusions with the reported results? The authors’ conclusions are fairly consistent with the reported results. The conclusions drawn were that online discussions play an important role in maintaining the quality of the postings and can reduce instructors’ work load without any negative impacts on student learning. Two research questions looked at students’ perceptions about the value of giving and receiving feedback and the authors reported that students valued both giving and receiving feedback as equally important and student perceptions about the importance of feedback significantly increased over the duration of the course. Results showed that students do value the importance of giving and receiving feedback despite the fact they value instructor feedback over peer feedback. The authors suggested that instructors need to validate peer feedback responses so students can see the benefits more.
  13. 13. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 13 The initial aim of this study was to investigate whether peer feedback could improve the quality of student online postings. The findings showed there was no increase and no decrease in the quality of postings with the introduction of peer feedback. Since the quality of the postings were maintained during the course, the authors suggested that peer feedback could be used to manage instructors’ workload without negatively impacting student learning. This finding relates to the study by Swartz and White (as cited in Mory, 2004) about the use of peer feedback to help manage feedback expectations which are that feedback should be: 1) prompt, timely and thorough; 2) ongoing including summative and formative; 3) constructive, supportive and substantive; 4) specific objective and individual; and 5) consistent. Based on the results of this study the authors recommend seven areas to consider when designing an online course and they are: • Help learners understand why and how the feedback process works • As an instructor model and give examples of effective feedback • Provide structure and guidelines to leaners about giving and receiving feedback • As a facilitator monitor the peer feedback process by providing feedback on the peer feedback • When possible ensure the feedback is anonymous • Use multiple peer ratings to have an overview of the value of learners’ responses • Make the peer feedback process easy to follow and straight forward 15. How well did the author relate the results to the study’s theoretical base?
  14. 14. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 14 The authors’ results supported the theoretical base that peer feedback can be a beneficial instructional strategy in an online environment and allows students to critically analyze their postings and self-reflect on what a high-quality posting looks like (Henderson, Rada & Chen, 1997; Topping, 1998). Topping (1998) also found that even though peer feedback was not as high quality compared with from an instructor, the immediacy, greater frequency and volume compensates for this. In this study, Ertmer et al. (2007) found that students are not concerned with the quantity of feedback, but more the quality of the feedback, which is contradictory to Topping’s (1998) findings. The immediacy in this case study was not met, as feedback sometimes took as long as two weeks, which counters one of the positive effects of peer feedback. Despite its limitations, utilizing Bloom’s taxonomy encouraged students to reflect on higher order thinking such as providing justifications and providing valuable critique. The authors reported one of the greatest potential benefits of peer feedback is how students are encouraged to construct their own meaning and understanding of peer feedback which relates to the theoretical base of Vygotsky’s (1978) constructivism and Zone of Proximal Distance. 16. In your view, what is the significance of the study, and what are its primary implications for theory, future research, and practice? The significance of this study was the idea that the power of using peer feedback in an online environment to maintain the quality of students’ postings, reduce the work load of the instructor and to help learners self-reflect, and engage in higher order thinking. The authors also found that students’ perceptions of feedback are that it should be timely and be of high quality rather than high quantity. The seven recommendations about feedback from this study include practical strategies to use feedback effectively in an online environment.
  15. 15. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 15 Future research could investigate the benefits of peer feedback as a community building strategy and one that promotes collegiality and trust and ultimately increasing motivation for individuals in an online environment thus improving learning and retention rates (van Ginkel, Gulikers, Biemans & Mulder, 2017; Espitia & Cruz Corzo, 2013). Future research could also investigate what models of effective feedback could be used to teach learners how to give and receive feedback and what factors do actually improve the quality of student online posting with the introduction of peer feedback. Different metrics could be employed to judge the quality of postings to measure the critical thinking and analysis and also better online discussion prompts could be devised to provide learners with the opportunity to demonstrate their higher order thinking.
  16. 16. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 16 References Black, A. (2005). The use of asynchronous discussion: Creating a text of talk. Contemporary Issues in Technology and Teacher Education, 5(1). Retrieved October 3, 2005 from http:// www.citejournal.org/vol5/iss1/languagearts/article1.cfm Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. Cunningham, R. D. (1992). Beyond educational psychology: Steps toward an educational semiotic. Educational Psychology Review, 4, 165–194. Drost, Ellen. (2011). Validity and reliability in social science research. Education Research and Perspectives, 38, 105-124. Ertmer, P. A., & Stepich, D. A. (2004, July). Examining the relationship between higher-order learning and students’ perceived sense of community in an online learning environment. Paper presented at the proceedings of the 10th Australian World Wide Web conference, Gold Coast, Australia. Ertmer, P. A., Richardson, J. C., Belland, B., Camin, D., Connolly, P., Coulthard, G., . . . Mong, C. (2007). Using peer feedback to enhance the quality of student online postings: An exploratory study. Journal of Computer-Mediated Communication, 12(2), 412-433. doi:10.1111/j.1083-6101.2007.00331.x Espitia, M., & Cruz Corzo, C. (2013). Peer-feedback and online interaction: A case study. Ikala, Revista De Lenguaje Y Cultura, 18(2), 131-151. Given, L. (2008). "Convenience Sample". The SAGE Encyclopedia of Qualitative Research Methods. SAGE Publications. doi:10.4135/9781412963909.n68. Henderson, T., Rada, R., & Chen, C. (1997). Quality management of student-student evaluations. Journal of Educational Computing Research, 17, 199–215. Hulley, S. B., Cummings, S. R., Browner, W. S., Grady, D. G., & Newman, T. B. (2013).
  17. 17. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 17 Designing clinical research. Philadelphia: Wolters Kluwer Health / Lippincott Williams & Wilkins. Ko, S., & Rossen, S. (2001). Teaching Online: A Practical Guide. Boston, MA: Houghton-Mifflin. Lynch, M. M. (2002). The Online Educator: A Guide to Creating the Virtual Classroom. New York: RoutledgeFalmer. Morse, J. M. (1994). Designing funded qualitative research. In Denizin, N. K. & Lincoln, Y. S., Handbook of qualitative research (2nd Ed). Thousand Oaks, CA: Sage. Mory, E. H. (2004). Feedback research revisited. In D. H. Jonassen (Ed.), Handbook of Research on Educational Communications and Technology (pp. 745–783). Mahwah, NJ: Lawrence Erlbaum. Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. Palloff, R. M., & Pratt, K. (1999). Building Learning Communities in Cyberspace. San Francisco, CA: Jossey-Bass. Palloff R. M., & Pratt, K. (2001). Lessons from the Cyberspace Classroom: The Realities of Online Teaching. San Francisco, CA: Jossey-Bass. Randolph, Justus J. (2007). Multidisciplinary Methods in Educational Technology Research and Development, HAMK Press, HTML (chapters 1 & 2). Roehler, L. R., & Cantlon, D. J. (1997). Scaffolding: A powerful tool in social constructivist classrooms. In K. Hogan & M. Pressley (Eds.), Scaffolding Student Learning: Instructional Approaches and Issues (pp. 6–42). Cambridge, MA: Brookline.
  18. 18. CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS 18 Rosenthal, R. and Rosnow, R. L. (1991). Essentials of Behavioral Research: Methods and Data Analysis. Second Edition. Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249–276. van Ginkel, S., Gulikers, J., Biemans, H., & Mulder, M. (2017). Fostering oral presentation performance: Does the quality of feedback differ when provided by the teacher, peers or peers guided by tutor? Assessment & Evaluation in Higher Education, 42(6), 953-966. Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA: Harvard University Press. Yin, R. K. (2003). Case Study Research: Design and Methods (3rd ed.). Thousand Oaks, CA: Sage.

×