From LIBEs’ framework to users experience of LIBE courses: analysing the Portuguese context
•Download as PPT, PDF•
0 likes•538 views
Presentation of University of Porto LIBE online courses tested in Portugal, and main project results.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to From LIBEs’ framework to users experience of LIBE courses: analysing the Portuguese context
Similar to From LIBEs’ framework to users experience of LIBE courses: analysing the Portuguese context (20)
More from Marta Pinto
More from Marta Pinto (7)
Recently uploaded
Recently uploaded (20)
From LIBEs’ framework to users experience of LIBE courses: analysing the Portuguese context
- 1. From LIBEs’ framework to users experience of LIBE courses: analysing the Portuguese context João Caramelo, Susana Coimbra, Marta Pinto caramelo@fpce.up.pt, susana@fpce.up.pt, martapcarvalho@fpce.up.pt LIBE Final Event December 2015 PROJECT REF. NO. 543058-LLP-1-2013-1-IT-KA3-KA3MP This project has been funded with support from the European Commission. This communication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.
- 2. Index Introduction Key competences and skills UPORTO Courses Pilot study: global quantitative results Pilot study: global qualitative results Final reflection December 2015
- 3. Introduction European project LIBE: common framework December 2015 Problem tackled: European countries dealing with issues deriving from low educational achievers (16-24) in literacy, numeracy and problem solving (OECD, 2013).
- 4. Introduction December 2015 • Cognitive skills and reflective thought processes. • Using tools interactively. • Interacting in heterogeneous groups. (Rychen & Salganik, 2003; Oates, 2003; OECD, 2013) • Basic ICT skills. • Literacy. • Numeracy. • Managing information in digital environments. Linked to • Problem solving in technology rich environment. (PIAAC, 2013; OECD, 2013) Which are the competences and skills that LIBE courses looks to develop? Competences and skills needed for people aged 16-24, with a profile of low achievers and at risk of social exclusion as a result of not having completed basic education levels and not having entered the labor market:
- 5. Introduction December 2015 LIBE PROPOSES TO CREATE AND IMPLEMENT E-LEARNING COURSES AIMED TO HELP YOUTH AND YOUNG ADULTS (16-24) WHO STRUGGLE TO MASTER THESE BASIC SKILLS. LIBE Courses foresee… Cognitive strategies and contents for literacy, numeracy, and problem solving skills in technology rich environments (OECD; PIAAC). “Computer and information literacy” that integrates e-skills - ICT practitioner skills; ICT user skills; and e-Business skills (European Commission, 2013).
- 6. Key Competences and Skills: focus groups Focus groups on key Competences and Skills for young low achievers’: hearing the voices of Portuguese students, trainers and teachers. December 2015 12 teachers/trainers: 2 focus groups 7 students “low achievers”: 1 focus group
- 7. Key Competences and Skills Key Competences and Skills most relevant for young low achievers: December 2015 1. Literacy 2. ICT competences+ 3. Social competences+ 4. Pedagogical Support + 5. Self-efficay +
- 8. • Topic: Awareness about food. • Skills and Competences focus: Problem solving in technological rich environments + Literacy. • Topic: reduce waste through 3Rs initiatives – Recicle, Reuse, Reduce. • Skills and Competences focus: Literacy. UPORTO Courses December 2015 UPORTO 2 LIBE courses
- 9. UPORTO Courses December 2015 Pre-test 4 activities Course 11 activities 2 Videos Post-test 4 activities Pre-test 6 activities Course 11 activities 2 Videos Post-test 6 activities
- 10. Piloting UPORTO Courses December 2015 Piloting of UPORTO LIBE courses in Portugal 246 students total 175 finished both courses: 91 male | 84 female Age: 14 to 23 M=17.41 | SD=1,43 3 Schools
- 11. Pilot study: Dimensions, examples of questions and descriptive statistics INITIAL QUESTIONNAIRE Questions | Scales of answers Self-efficacy toward ICT How well can you do each of the following tasks on a computer? •Move files from one place to another in a computer? •Download files of software from the internet? (1) I can do this very well by myself – (4) I don’t know what this means Attitudes toward potential of internet Think about your experience with computers: to what extent do you agree with the following statements? •Using the computer for learning is troublesome. •The internet is a great resource where to obtain information from, for my school work. (1) Strongly agree – (4) Strongly disagree Attitudes toward limits of internet December 2015 RESULTS N Minimum Maximum Mean Std. Deviation Self-efficacy toward ICT 175 2,75 4,00 3,70 ,26 Attitudes toward potential of internet 175 2,33 4,00 3,67 ,40 Attitudes toward limits of internet 175 1,00 4,00 2,34 ,63
- 12. Pilot study: Dimensions, examples of questions and descriptive statistics FINAL QUESTIONNAIRE Questions | Scales of answers Difficulty of questions Course 1 What did you think about the difficulty of the questions in this course? (1) All Easy – (4) All Challenging Difficulty of questions Course 2 Motivation Course 1 Did you feel motivated while working on this course? (1) Strongly agree – (4) Strongly disagreeMotivation Course 2 December 2015 RESULTS N Minimum Maximum Mean Std. Deviation Difficulty of questions Course 1 175 1,00 4,00 1,74 ,84 Difficulty of questions Course 2 175 1,00 4,00 1,63 ,77 Motivation Course 1 175 1,00 4,00 3,21 ,69 Motivation Course 2 175 1,00 4,00 3,17 ,75
- 13. Pilot study: Dimensions, examples of questions and descriptive statistics FINAL QUESTIONNAIRE Questions | Scales of answers Usability Course 1 What do you think about the e-learning system of this course? (1) Strongly disagree – (5) Strongly agree Usability Course 2 Final assessment Course 1 Overall, you would rate the user-friendliness of this course as: (1) Worst Imaginable – (7) Best Imaginable Final assessment Course 2 December 2015 RESULTS N Minimum Maximum Mean Std. Deviation Usability Course 1 175 1,00 5,00 3,63 ,79 Usability Course 2 175 1,00 5,00 3,28 ,73 Final assessment Course 1 175 1,00 7,00 5,34 1,24 Final assessment Course 2 175 1,00 7,00 5,18 1,34
- 14. Pilot study: Predictors of assessment Course 1 December 2015 (F=13.762; p≤.001; Adjusted R Square=.382) Model Unstandardized Coefficients Standar Coeffic. t Sig. Collinearity Statistics B Std. Error Beta Tol VIF ,99 1,65 ,60 ,55 Difficulty Course 1 ,04 ,11 ,02 ,35 ,73 ,79 1,27 Motivation Course 1 1,01 ,13 ,57 8,08 ,000 ,76 1,31 Usability Course 1 ,30 ,11 ,18 2,65 ,009 ,79 1,27 ICT Self-efficacy ,15 ,32 ,03 ,47 ,64 ,84 1,19 Attitudes toward potential of internet -,10 ,20 -,03 -,49 ,62 ,91 1,10 Attitudes toward limits of internet ,08 ,13 ,04 ,59 ,56 ,82 1,22 Age -,03 ,06 -,03 -,45 ,65 ,95 1,06 Gender_Male ,00 ,16 ,00 ,02 ,98 ,91 1,10
- 15. Pilot study: Predictors of assessment Course 2 December 2015 (F=4.303; p≤.001; Adjusted R Square=.138) Model Unstandardized Coefficients Standar Coeffic. t Sig. Collinearity Statistics B Std. Error Beta Tol VIF 1 1,45 2,11 ,69 ,49 Difficulty Course 2 -,22 ,13 -,14 -1,68 ,10 ,76 1,31 Motivation Course 2 ,52 ,15 ,30 3,54 ,001 ,73 1,37 Usability Course 2 ,09 ,14 ,05 ,62 ,54 ,90 1,11 ICT Self-efficacy ,54 ,40 ,11 1,35 ,18 ,86 1,17 Attitudes toward potential of internet -,07 ,25 -,02 -,26 ,80 ,92 1,09 Attitudes toward limits of internet ,19 ,16 ,09 1,15 ,25 ,88 1,14 Age ,00 ,07 ,00 ,04 ,97 ,93 1,08 Gender: Male -,16 ,20 -,06 -,79 ,43 ,90 1,12
- 16. Pilot study global results: Positive feedback from students to course 1 and 2 December 2015 System related User experience Course related E-learning system. Learning experience. Motivation. Interest. Relevant topics. More visual content. Less number os questions. Less amount of plain text. “Allows you to learn important things in a more playful way.” “It was a fun and educational experience” “Interesting, appealing and important for day to day.” "This course is important because there are things that we should all know and pass on, from generation to generation, because it’s something that contributes positively to all of us and to protect the planet.“ . . .
- 17. Pilot study global results: Negative feedback from students to course 1 and 2 December 2015 System related User experience Course related Technical errors. Usability. Navigation buttons: check/next/go back. Demotivation. Takes too long to finish the course. Less visual content. Too many activities. Greater amounts of plain text. Repetition of pre-test and post-test. “Has errors. Needs more interactive questions and more information.” "It has to be taken into account that the platforms was not functioning properly…” “Too extense, boring and confusing” “Make the website graphics more captivating, to make the user feel even more at ease, and not feel it’s such a dull thing.” “Has too much text. Some questions are confusing. Should have more images.” “Don’t repeat the tests.” . . .
- 18. Final reflection December 2015 Receptivity to learning experience through e-learning. Mismatch between the sense of self-efficacy regarding ICT and its usefulness in the immediate academic achievement and lifelong learning. Importance of social and peer support as a form of pedagogical support in e-learning, and an intuitive choice for collaborative work, developing social competences. Topics are more relevant when creating a meaningful relation between familiarity and usefulness. Heterogeneity of “low achievers”.
- 19. Thank you! LIBE Final Event December 2015 PROJECT REF. NO. 543058-LLP-1-2013-1-IT-KA3-KA3MP This project has been funded with support from the European Commission. This communication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. João Caramelo, Susana Coimbra, Marta Pinto caramelo@fpce.up.pt, susana@fpce.up.pt, martapcarvalho@fpce.up.pt
Editor's Notes
- Involves 5 countries (Portugal, Italy, Netherlands, Norway, United Kingdom) intends to tackle the problem of European countries dealing with issues (higher unemployment rates) deriving from low educational achievers (16-24) in literacy, numeracy and problem solving (OECD, 2013)
- >>The cross analysis of recent research literature and relevant OECD, EU and IEA reports and assessment frameworks were carried away with the simultaneous intention to identify and select learning outcomes globally identified as relevant for professional and personal development of youth and young adults and to define possible pedagogical strategies to promote successful learning outcomes in a technology rich environment. >>Relating the international scope studies from OECD (2013b) and research regarding the type of competences necessary for a successful life and a well-functioning society (Rychen & Salganik, 2003; Oates, 2003), it can be stated that for young adults with a profile of low achievers, the competences and skills they need to develop in order to surpass this category, are: · Cognitive skills and reflective thought processes (thinking about thinking, creative abilities and taking a critical stance). · Using tools interactively. This refers to the ability to use language, symbols and text interactively, the ability to use knowledge and information interactively and the ability to use technology interactively. · Interacting in heterogeneous groups; including the ability to relate well to others, the ability to co-operate and the ability to manage and resolve conflicts. >>The report for the Programme for the International Assessment of Adult Competencies (PIAAC) survey (2013) reveals that proficiency in literacy, numeracy and managing information in digital environments is linked to the proficiency in problem solving in digital environments (OECD, 2013a). Adults or young adults who fail the ICT core competencies generally have low proficiency in literacy and numeracy, which suggests that low literacy may cause difficulty in the acquisition of basic ICT skills (OECD, 2013a). But in most countries, younger adults have higher proficiency than older adults in all three of the skills assessed. If those skills are developed and effectively used on the job, it can translate into better economic and social outcomes (OECD, 2013a). Literacy, numeracy and problem solving within technology-rich environment skills, are defined in the PIAAC 2013 report (OECD, 2013b) as follows: · “Literacy is defined as the ability to understand, evaluate, use and engage with written texts to participate in society, to achieve one’s goals, and to develop one’s knowledge and potential. Literacy encompasses a range of skills from the decoding of written words and sentences to the comprehension, interpretation, and evaluation of complex texts. It does not, however, involve the production of text.” · “Numeracy is defined as the ability to access, use, interpret and communicate mathematical information and ideas in order to engage in and manage the mathematical demands of a range of situations in adult life. To this end, numeracy involves managing a situation or solving a problem in a real context, by responding to mathematical content/information/ideas represented in multiple ways.” · “Problem solving in technology rich environments is defined as the ability to use digital technology, communication tools and networks to acquire and evaluate information, communicate with others and perform practical tasks. The assessment focuses on the abilities to solve problems for personal, work and civic purposes by setting up appropriate goals and plans, and accessing and making use of information through computers and computer networks.”
- >>Three focus-group were developed in order to establish and understand the skills, competencies and learning needs of low achiever students, concerning literacy, numeracy and ICT skills. >>In Portugal, LIBE researchers performed 3 focus groups: FG with 6 trainers of 3 vocational training centres, with experience working with students considered to be low achievers, notably migrants and students that drop out from regular schooling paths; FG with 6 teachers of 4 urban secondary schools, working with students that have below average grades in national standardized exams in different curriculum subjects such as Mathematics and Portuguese language; and a FG with 6 students of training vocational courses (TVET) from one training centre (part of the European Association for Cities, Institutions and Second Chance Schools). >>About 7-9 well-developed topics/issues will be explored through multiple questions in the 1-2-hour focus group interview. The topics were: Topic 1 – Low achievers knowledge and skills Topic 2 – Activities and topics Topic 3 - Successful experiences with low achievers
- The most important category of analysis, across different participants’ points of view, was literacy skills the ability to communicate effectively with others, read better and interpret what they read was considered as the most relevant. Thus would imply a stronger series of activities on this area in LIBE modules. ICT competences, which also integrates literacy skills, the needs have a specific focus on the ability to access, retrieve and evaluate the information on the Internet it was described a common lack in distinguishing trustworthy from unreliable information; also the ability to communicate and manage information online was addressed, and can somewhat be linked to the social competences needed to be learned for the school context and for an internet (mainly use of social networks) context. The need to offer low achiever students pedagogical support from teachers, trainers and from peers. This support seemed quite relevant and determinant of students self-efficacy. The more students feel confident, motivated and supported, the more enhanced is their participation in school and learning. This is unquestionably relevant for the proposal of LIBE courses, that will need to ponder the type of support given in a virtual learning environment. Learner-centred approach where commitment to learning is mainly due to the motivation towards the activity is based on different topics related to students day-to-day lives.
- Based on te results of the previous focus groups, topics were chosen and learning activities designed for 2 LIBEs elearning courses.
- After the courses were implemented in LIBEs Moodle plaform, they were tested in the pilotstudy stage of the project. The sample design: Participants are purposefully selected based on their match with the target audience. Recruitment will be on a voluntary basis and linked to activities carried out by high schools and vocational education centres. The sampling frame will consist of students between 16 and 24 years old with mixed ability levels, mixed gender, and from both vocational learning centres and high schools. Each country will collect data from at least 300 students.
- The following tables display the dimensions assessed, as well as examples of the questions and some descriptive statistics for each dimension. This table is related to self-efficacy toward TIC and attitudes toward the potential and limits of the use of the internet. Participants report high levels of self-efficacy toward TIC and positive attitudes toward using internet as a source for their study (high levels of perceived potential – useful, fun resourceful and low levels of perceived risk- troublesome, not suitable, unreliable). Here we consider that it is worthy of notice that no gender differences were observed in any dimension, except for self-efficacy regarding ICT: boys (M=3.75; SD=.25) report levels considerably higher of self-efficacy when compared with girls (M=3.64; SD=.26) [t (1, 173)= 2.88; p=.004]
- Regarding perceived difficulty and motivation, we may observe that participants present low levels of perceived difficulty (1.7 and 1.6) and high levels of motivation for both courses (around 3, in a likert scale ranging from 1 to 4).
- Finally, participants report high levels of usability (above 3 in a scale of 1 to 5), and the final assessment of user-friendliness for the courses is high (above 5, in a scale of 1 to 7). We additionally observed that participants score significantly higher the 1st course regarding usability and global user-friendliness, when compared to the 2nd one [t (1, 175)= 6.79; p≤.001; t (1, 175)= 2.27; p=.025], and no significant differences were found regarding difficulty, or motivation. This may be related to some technical errors that emerge more often during the 2nd course.
- We ran a multiple regression analysis using enter method in order to identify which variables may function as predictors of the final assessment of user-friendliness of each course. For the course 1, motivation and usability are significant and positive predictors of a moderate and low magnitude, respectively. This model is significant and explains around 38% of the outcome variable.
- For the 2nd course, motivation is the only one significant predictor, positive and of low magnitude. This model is also significant and explains around 14% of the outcome variable.