A Tool for Determining an Optimal Model of Student Engagement in Wikis for Learning English for Academic Purposes
•
1 like•1,115 views
Paper presented at the 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April The problem of learners’ engagement is receiving renewed attention against the background of the transition from traditional to technology enhanced learning environments and the ensuing shift from the transmission to the (co)construction of knowledge model in teaching/learning. Recent studies on the use of social media for academic purposes have also revealed transformations in student and teacher roles and behavior. This study is part of an ongoing research on various pedagogical aspects related to early adoption of Web 2.0 tools (wikis) in a blended language learning program of English for Academic Purposes attended by undergraduate students in social sciences in a Romanian university. The authors’ prior research on student behavior suggested low participation levels during the semester and high activity under the pressure of finals. Our current objective is to continue the research in an attempt to provide a more accurate learner profile that could help teachers improve the learning environment to further activate their students. We applied the ROC analysis, a cost/benefit analysis of diagnostic decision making, in order to diagnose the features of students who would use successfully a Web 2.0 tool for learning. The study aims to exemplify how ROC analysis can be used to offer a classification of students with positive/negative inclinations to adopt a Web 2.0 tool by eliminating the false positives,respectively the false negatives. Consequently, it may become a tool to select a possible optimal model to be considered in the process of needs analysis and of taking important decisions regarding teaching methods, course contents and design.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (18)
Similar to A Tool for Determining an Optimal Model of Student Engagement in Wikis for Learning English for Academic Purposes
Similar to A Tool for Determining an Optimal Model of Student Engagement in Wikis for Learning English for Academic Purposes (20)
Recently uploaded
Recently uploaded (20)
A Tool for Determining an Optimal Model of Student Engagement in Wikis for Learning English for Academic Purposes
- 1. A TOOL FOR DETERMINING AN OPTIMAL MODEL OF STUDENT ENGAGEMENT IN WIKIS FOR LEARNING ENGLISH FOR ACADEMIC PURPOSES Cristina Felea & Liana Stanca Babes-Bolyai University 2014 "Let’s build the future through learning innovation!". The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April
- 2. Overview Background • Wikis in HE • Engaging students • Prior research Research Study • Objectives • Data and Method • Statistical Study Conclusions The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April
- 3. Background The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April
- 4. 1.1 Wikis in Higher Education The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April ● allow access to more extended and varied information and knowledge resources; ● increase considerably documenting sources ● offer opportunities for new learning situations; ● move the focus from teaching to learning; ● facilitate the adoption of active, participative methods; ● promote independent, innovative and creative work; ● stimulate cooperation and dialogues by integrating specific collaborative environments; ● develop learners' autonomy and flexibility; ● blend individual and social learning; Source: Timsoft
- 5. 1.2 Engaging Students Engagement = “the interaction between the time, effort and other relevant resources invested by both students and their institutions intended to optimize the student experience and enhance the learning outcomes and development of students and the performance, and reputation of the institution.” (Trowler, 2010) -> situated on a continuum ranging from positive engagement, non-engagement and negative engagement The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April Fredericks, Blumenfeld and Paris (2004)
- 6. 1.3 Prior Research English for Academic Purposes (2009 -2013) - 667 first year undergraduate students distributed in CEFR level groups - course management pages, individual work pages, collaborative project work - research based on data collected by wiki content/statistics and in-class informationThe 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April
- 7. Infrastructure Resources Support Communication & Feedback Class-based face-to-face activities Wiki-based online activities Lectures Readings Pair/Group work Self-paced/ independent Collaborative work Printed handouts Wiki Tutorials Learning units Open educational resources Online tutorials T’s feedback Discussion forum Course management and individual work spaces Data collected from wiki English for Academic Purposes Wiki INPUT Cristina Felea wiki mail
- 8. Processes Outcomes Impact Minimal ICT skills Easy revision Automatic data collection Study skills (self/peer- evaluation, time management etc) Collaborative tasks & presentations Developing transversal competencies Self-directed learning Language learning in authentic context Teacher-centred -> Student-centred Issues English for Academic Purposes Wiki OUTPUT Cristina Felea
- 9. Relevant conclusions of prior research 1. Within wiki environment, two types of collaboration may occur: S-S & S-T; S-S is less frequent, showing a low level of behavioural/cognitive involvement; 2. Ss seem reluctant to contribute to wiki contents - this may reside in Ss’ “fear of the unknown” (i.e. no prior experience in learning with technology, low language level); this is accompanied by low class attendance, little or no independent activity, showing behavioural, emotional, cognitive non- engagement; 3. Most students only get involved under the pressure of deadlines/constraints, showing an overall non-engagement with the learning process.
- 10. Research Study Objective: to create the profile of a student who is most likely to get engaged on all three dimensions (behavioural, cognitive, emotional) in a blended learning environment. The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April
- 11. 2.1 Data Collection 1. class attendance 2. learning outcomes: final grades 3. student-wiki interaction: views/all wiki pages (including personal) 4. independent work: edits on personal pages Questionnaire 5. Demographic data (age, gender, education) 6. Self-reported data on • interest in learning English, • length of formal study, • global language level (cf. Common European Framework) • experience in using an e-learning platform/online environment for learning a new discipline. The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April
- 12. 1. Descriptive analysis of categorical/continuous variables - mixed data 2. Clustering - the TwoStep algorithm was used to optimally determine the number of clusters within the data set 3. ROC (receiver operating characteristic) analysis - used to evaluate and compare the performances of the classification model, i.e. learners’ engagement 2.2 Research Method The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April
- 13. 2.3.1 Data Description The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April Demographics 77 participants distributed in 3 age groups: 18-21 - 75.3%; 22-25 - 18%; 26-50 - 6.7%; gender distribution: 77.3% female and 22.5% male Education: length of study of English 4 groups: 1-4 years - 9.1%; 5-8 years - 28.6; 9-10 years - 33.8%; 11+years - 28.6%. Interest for the study of English 5 groups: very interested – 48.7%; interested – 23.7%; relatively interested – 18.2%; low interest – 5.2% and no interest 5.2%.
- 14. 2.3.2 Clustering Hypothesis: There are differences in the students’ wiki engagement behaviour. The representative set of attributes for forming clusters is: • interest for studying English • length of study • language level • class attendance • grade for wiki page activity • final grade (learning outcome) • wiki page views • wiki page edits The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April
- 15. 2.3.2 Clustering - behaviour types Attributes Cluster 1 - medium to high engagement 42.9% Cluster 2 - non-engagement 55.8% Gender 46.7% females; 53.3% males 53.3% females; 46.7% males Age 18-21, respectively 26-50 18-21, 22-25 respectively 26-50 Interest interested to very interested average to no interest Length of study 5-8 years > 9 years Language level A2-B1 (cf. CEFR) B1- B2 (cf. CEFR) Class attendance 1.15 (2 is max.) 0.87 (2 is max.) Wiki page activity (views and edits) 0.75 (1 is max) - average to high 0.39 (1 is max.) Average final grade (learning outcome) 6.56 with stdev=2.203 (average to low) 4.36 with stdev=3.71 (low)
- 16. 2.3.3 ROC analysis ROC curve - evaluation metrics = to evaluate & compare performance of the classification model -> structure of meta-attributes was built showing the characteristics necessary for a student to engage successfully in a learning environment while being assisted and guided by a knowledgeable other. The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April
- 17. Conclusions
- 18. Conclusions The findings have revealed two categories of engagement behavior. The analysis showed that the average engagement is correlated with a blended approach, namely with participation in face-to-face and virtual environment. A definite advantage is the possibility of superior interaction (both synchronous and asynchronous) and better acquaintance with the participants. The 10th eLearning and Software for Education Conference - eLSE 2014, Bucharest 24-25 April
- 19. Conclusions The results confirm prior research, showing that, rather than engagement, most students manifest compliance by learning only under constraints while those with advanced language level perform poorly in this environment due to lack of engagement in regular activities. In order for students to benefit from engagement, they “must invest time and effort into academic activities and practices … that correlate highly with positive educational outcomes” (Trowler, 2010)
- 20. Selected References Conole, G., & Alevizou, P. (2010). A literature review of the use of Web 2.0 tools in Higher Education. A report commissioned by the Higher Education Academy McLoughlin, C., & Lee, M. J. (2007, December). Social software and participatory learning: Pedagogical choices with technology affordances in the Web 2.0 era.In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007 (pp. 664-675). Twu, H. L. (2010). A Predictive Study of Wiki Interaction: Can Attitude toward Wiki Predict Wiki Interaction in High-Context Cultures Groups?. Journal of Educational Technology Development & Exchange, 3(1) Cole, M. (2009). Using Wiki technology to support student engagement: Lessons from the trenches. Computers & Education, 52(1), 141-146. Hazari, S., North, A., & Moreland, D. (2009). Investigating Pedagogical Value of Wiki Technology. Journal of Information Systems Education, 20(2). Kearsley, G., & Shneiderman, B. (1998). Engagement Theory: A Framework for Technology-Based Teaching and Learning. Educational technology, 38(5), 20-23 Trowler, V. (2010). Student engagement literature review. York: Higher Education Academy
Editor's Notes
- The 10th eLearning and Software for Education Conference - eLSE 2014
- The advent and development of Web 2.0 technologies, with their characteristic openness, connectivity, participation in the production of knowledge, has opened new doors to educators in search for new ways to improve student learning and performance in an increasingly dynamic and diverse world. More and more, researchers have emphasized their transformative potential in various ways, from the usage of their communicative and collaborative facilities to influencing social behavior and pedagogic practices [2], [3], [4], [5]. The collaborative and user-generating, peer-editing features of the wikis, in particular, have been successfully used, accompanied by a significant body of research, in higher education writing instruction and foreign language courses [6], [7], [8], [9]. Moreover, wikis seem to be favored for their pedagogical value in engaging online collaborative learning communities as well as for their added value for language learning due to authentic, task- or enquiry-based learning design . However, for these activities “to generate better and higher learning engagement”, students need to have some prior experience in e-learning or Web 2.0 technologies. Similarly, their participation to collaborative tasks and “active knowledge sharing and information arranging” needs guidance from the teacher, with special attention to students’ beliefs and attitudes towards collaboration [10]. Many of these issues are related to the way student and teachers engage in the learning process.
- The concept of “student engagement” is certainly not new in educational theory. Its frequent use in relation to technology-enhanced learning, especially Web 2.0 technology and wiki-based learning environments as a synonym for participation or involvement [11], [12], [13] has revealed its multidimensional nature. As early as 1999, Kearsley & Shneiderman proposed an “engagement theory” rooted in their experience with technology-enhanced teaching and learning. According to them, “engaged learning” includes “active cognitive processes such as creating, problem-solving, reasoning, decision-making, and evaluation. In addition, students are intrinsically motivated to learn due to the meaningful nature of the learning environment and activities. Their study contrasted individual human-instructional program interaction characteristic of original CALL with “human interaction in the context of group activities”, measured by considering a larger context rather than “single responses.” Most importantly, the authors contend that “the difference between engagement and interactivity reflects the shift in thinking about computers in education as communication tools rather than some form of media delivery devices.” In their view, “engagement theory places a great deal of emphasis on providing an authentic (i.e., meaningful) setting for learning, something not present in previous models.” [14]. In a literature review commissioned by the Higher Education Academy, Trowler defines engagement as “the interaction between the time, effort and other relevant resources invested by both students and their institutions intended to optimize the student experience and enhance the learning outcomes and development of students and the performance, and reputation of the institution.” Based on prior specialist literature, the author adopts a complex three-dimensional view of student engagement: behavioral, emotional and cognitive, and situates it on a continuum ranging from positive engagement, non-engagement and negative engagement [15]. This approach is extremely relevant for the purpose of the current study, as can be seen below, in a brief description of the authors’ prior research.
- This paper is based on the research on the blended learning environment created in 2010 for first year social science undergraduates studying English for Academic Purposes. The wiki-platform was initially built to provide students with direct access to course management and materials, and to provide them supplementary resources catering for a variety of needs (language level, learning styles, medium and modality). Personal work spaces (wiki pages) were created to offer students the opportunity to keep a learning diary with their independent work (activities and assignments). Communication and feedback were possible by means of wiki mail and discussion tabs. Collaborative writing projects were set to develop students’ language and cooperative skills. Several changes have resulted as an effect of this decision, which gradually required for a blended approach. Practice has shown that teaching in this environment means reconsidering most components: from performing self-reflective analyses to be able to change roles in the instructional process, to finding out more about students’ needs so as to be able to cater in a personalized way. Therefore, besides wiki log data and observation based on interaction (classroom and online), we collected information on a yearly basis so as to get familiar with the learners’ profile: demographic and educational background, experience/length of study and level in language and computer/Web 2.0 technology skills. Students’ perceptions and attitudes have been analyzed [16], [17]. Our research interests so far have focused mainly on students’ acceptance of the online tools and subsequent degree of engagement. Most of the conclusions match current specialist literature on the topic. The following are relevant for the present study: (1) within the wiki environment, two types of collaboration may occur: student-student(s) and student-teacher. According to Zorko and our own observations, the former is less frequent. For example, students’ reluctance to change or to propose changes on peers’ pages shows a low level of behavioral/cognitive engagement [18]; (2) in a similar vein, students seem to be reluctant to contribute to wiki contents [19], [20]. The results of our three-year studies have confirmed that this reluctance may reside in what we could call “fear of the unknown” of students with no prior experience in learning with technology and a low language level; this category is also characterized by low class attendance and little or no activity in their personal online work space, showing behavioral, emotional and cognitive non-engagement [21]. (3) our research has also demonstrated that most students only get involved in the course under the pressure of deadlines and other constraints, showing an overall non-engagement with the learning process [22], [23], [24].
- The wiki-platform was initially built to provide students with direct access to course management and materials, and to provide them supplementary resources catering for a variety of needs (language level, learning styles, medium and modality). Personal work spaces (wiki pages) were created to offer students the opportunity to keep a learning diary with their independent work (activities and assignments). Communication and feedback were possible by means of wiki mail and discussion tabs. Collaborative writing projects were set to develop students’ language and cooperative skills. Several changes have resulted as an effect of this decision, which gradually required for a blended approach. Practice has shown that teaching in this environment means reconsidering most components: from performing self-reflective analyses to be able to change roles in the instructional process, to finding out more about students’ needs so as to be able to cater in a personalized way. Therefore, besides wiki log data and observation based on interaction (classroom and online), we collected information on a yearly basis so as to get familiar with the learners’ profile: demographic and educational background, experience/length of study and level in language and computer/Web 2.0 technology skills. Students’ perceptions and attitudes have been analyzed [16], [17]. Our research interests so far have focused mainly on students’ acceptance of the online tools and subsequent degree of engagement. Most of the conclusions match current specialist literature on the topic. The following are relevant for the present study: (1) within the wiki environment, two types of collaboration may occur: student-student(s) and student-teacher. According to Zorko and our own observations, the former is less frequent. For example, students’ reluctance to change or to propose changes on peers’ pages shows a low level of behavioral/cognitive engagement [18]; (2) in a similar vein, students seem to be reluctant to contribute to wiki contents [19], [20]. The results of our three-year studies have confirmed that this reluctance may reside in what we could call “fear of the unknown” of students with no prior experience in learning with technology and a low language level; this category is also characterized by low class attendance and little or no activity in their personal online work space, showing behavioral, emotional and cognitive non-engagement [21]. (3) our research has also demonstrated that most students only get involved in the course under the pressure of deadlines and other constraints, showing an overall non-engagement with the learning process [22], [23], [24].
- 2.1 As outlined above, the low participation to and subsequent non-engagement in learning in a wiki-based environment was influenced mostly by the lack of student experience in learning in online environment. The purpose of our study is to create the profile of a student who is most likely to get engaged on all three dimensions in a blended learning environment. We considered the following set of attributes: behavioral - class attendance, student-wiki interaction (number of views for all wiki pages, including personal work spaces), independent work (number of edits on personal pages). By means of the questionnaire, we collected demographic data (age, gender, education), and self-reported data on interest in learning English, length of formal study, global language level (cf. Common European Framework), experience in using an e-learning platform/online environment for learning a new discipline. Learning outcomes were measured by considering final grades (composed of end-of-semester written exam and involvement in activity on personal work spaces). 2.2 The present study is based on the use of categorical and continuous variables, which amounts to mixed data. According to studies by [25], [26], [27], simulations aimed at gathering persons with similar mixed variables in clusters, should use the TwoStep clustering method. The TwoStep method can optimally determine the number of clusters existing within a mixed data set. The statistical analysis consists of data descriptive analysis, the application of the TwoStep algorithm to determine the clusters and the ROC analysis. Next, the resulting clusters were measured by the ROC curve to evaluate and compare the performances of the classification model, namely the learners’ engagement behavior in the process of learning. First, a hierarchical logistical regression was used, followed by discriminant analysis and determination of the specific classifier ROC curve.
- 2.1 As outlined above, the low participation to and subsequent non-engagement in learning in a wiki-based environment was influenced mostly by the lack of student experience in learning in online environment. The purpose of our study is to create the profile of a student who is most likely to get engaged on all three dimensions in a blended learning environment. We considered the following set of attributes: behavioral - class attendance, student-wiki interaction (number of views for all wiki pages, including personal work spaces), independent work (number of edits on personal pages). By means of the questionnaire, we collected demographic data (age, gender, education), and self-reported data on interest in learning English, length of formal study, global language level (cf. Common European Framework), experience in using an e-learning platform/online environment for learning a new discipline. Learning outcomes were measured by considering final grades (composed of end-of-semester written exam and involvement in activity on personal work spaces). 2.2 The present study is based on the use of categorical and continuous variables, which amounts to mixed data. According to studies by [25], [26], [27], simulations aimed at gathering persons with similar mixed variables in clusters, should use the TwoStep clustering method. The TwoStep method can optimally determine the number of clusters existing within a mixed data set. The statistical analysis consists of data descriptive analysis, the application of the TwoStep algorithm to determine the clusters and the ROC analysis. Next, the resulting clusters were measured by the ROC curve to evaluate and compare the performances of the classification model, namely the learners’ engagement behavior in the process of learning. First, a hierarchical logistical regression was used, followed by discriminant analysis and determination of the specific classifier ROC curve.
- Demographics There were 77 participants in the study, who were distributed in three age groups, namely 18-21 - 75.3%; 22-25 - 18%; 26-50 - 6.7%. The gender distribution is the following: 77.3 female and 22.5 male participants. Education – length of study of English As the study of a foreign language is compulsory in Romanian education, we considered it necessary to determine students’ distribution according to the number of years they had studied English formally. The resulting four groups are: 1-4 years - 9.1%; 5-8 years - 28.6; 9-10 years - 33.8%; 11+years - 28.6%. Interest for the study of English The data collected based on a scale of interest in the study of English resulted in the following five groups: very interested – 48.7%; interested – 23.7%; relatively interested – 18.2%; low interest – 5.2% and no interest 5.2%.
- The participation/involvement in wiki-based activities is expressed by the average values of edits and views along a five-month interval (first semester, including session of exams and preparation for re-examination):
- Fredericks, J. A., Blumenfeld, P. C. & Paris, A. H. (2004). School engagement: potential of the concept, state of the evidence. Review of Educational Research, 74, 59-109.