1. Assessment in Mathematics GROUP 2 Lee Wei, June Lenny Azlina Ong Fei Min, Flora Deborah Tan Yap Thiam Chuan
2. Overview of Presentation 1. Deciding on a developmental continuum 1.1 The Context, the task & Purpose of task 1.2 Framework of Mathematics in National Curriculum 1.3 Components in National Curriculum framework 1.4 Learning Framework: the Dreyfus model 2. Developing a standards referenced assessment framework 2.1 Building a pedagogical framework 2.2 Identifying performance indicators 2.3 Breaking down the domain 2.4 Quality criteria & initial rubrics 3. Drafting the assessment items 3.1 Questions on test 4. Panelling 4.1 Procedure 4.2 Issues, concerns and themes 4.3 Comments from panellists 5. Revision of Assessment framework and test items 5.1 Quality criteria & amended rubrics 5.2 Further amendments & revised rubrics 5.3 Final performance matrix 5.4 Revised test items 6. Implementation of assessment 6.1 Test conditions 6.2 Instructions for administrator of test 7. Analysis of results 7.1 Zone of proximal development 7.2 Guttman chart 7.3 Analysis: Commonalities 7.4 Analysis: Anomalies 8. Reporting 8.1 Scope of assessment and audience 8.2 Reporting for students & parents 8.3 Reporting for teachers & school 9. Discussion 9.1 Intervention 9.2 Reliability & Validity 9.3 Scaling up 10. Group Reflection 10.1 Reflection on the assessment task 10.2 Reflection on how the interpretation was done 10.3 Reflection: last words
3. 1 Deciding on a Developmental Continuum 1.1 The Context, the Task & Purpose of task 1.2 Framework of Mathematics in National Curriculum 1.3 Components in National Curriculum framework 1.4 Learning Framework: the Dreyfus model
10. 2 Developing a standards referenced assessment framework
11. 2.1 Building a Pedagogical Framework Pedagogy DOMAIN: These are sets of skills, knowledge, behaviours and dispositions that enable us to sample and define phases, the strands or constructs STRANDS & CAPABILITY: The big ideas that are learnable, teachable. They too are the set of skills, knowledge and expectations or learning outcomes 1 INDICATOR: Identified behavioural indicators. These indicative behaviours were things that a student could do, say, make or write , and from which we infer their capability in an area CRITERIA: Finally, observational statements that detailed ‘how well’ each behaviour could be performed were created. These criteria underpin the profiling of learning pathways. The criteria are represented by ordered levels of increasing difficulty, sophistication, elegance, etc… 2 3 n
14. 2.4 Quality Criteria & Initial Rubrics Manipulation of standard form Manipulation of algebraic expressions Application of Concepts Mathematical reasoning, communication and connections Expert Compare and contrast alternative methods in the use of different laws Explain errors / misconceptions in the: – algebraic manipulation, - use of the laws of indices Proficient Perform the four operations in standard forms expressions Manipulate algebraic fractions and polynomials (with positive, negative, zero and fractional indices), showing consistent and clear working Apply multiple laws of indices concurrently Competent Identify large and small numbers such as giga, micro, pico Manipulate algebraic expressions with positive, negative, zero or fractional indices Differentiate the laws of indices and apply laws appropriately Identify errors in the four operations on indices, surds and use of laws of indices Novice & Advanced Beginner Express numbers in standard form Perform four operations on simple algebraic fractions
15. 2.4 Quality Criteria & Initial Rubrics Students are introduced to the nomenclature, and learn to express familiar numbers in simple standard forms. At competent level, the manipulation of very small and very large numbers would be done. Moving to the proficient level, the students would be able to perform the operations, namely the addition, subtraction, multiplication and division of standard form expressions. At the competent level, students would be able to manipulate algebraic expressions and fractions with indices (or powers). The nature of the math curriculum is spiral, such that students would be taught algebraic manipulation with increasing complexity. At this secondary three level, they would be taught how to manipulate algebraic fractions with indices. The laws of indices is a major concept at upper secondary, so it was not considered as a novice or advanced beginner level. At the competent level, students are expected to be able to differentiate which laws to use, and to apply the laws appropriately. They would only be required to apply the isolated laws. As mentioned in the learning framework, there would be conscious deliberate planning. Students would show standardized and routinized procedures in applying each law. The ability to articulate reasons, and communicate strategies and connections was beyond a novice or advanced beginner level, as students at lower secondary level need only solve questions in routinized manner, and rarely requiring them to state the reasons or laws used. At competent level, the students are able to identify and explain errors in the use of indices. It would be inferred that they have understood and concept, and could communicate their understanding using mathematical reasoning. Manipulation of standard form Manipulation of algebraic expressions Application of Concepts Mathematical reasoning, communication and connections Expert Compare and contrast alternative methods in the use of different laws Explain errors / misconceptions in the: – algebraic manipulation, - use of the laws of indices Proficient Perform the four operations in standard forms expressions Manipulate algebraic fractions and polynomials (with positive, negative, zero and fractional indices), showing consistent and clear working Apply multiple laws of indices concurrently Competent Identify large and small numbers such as giga, micro, pico Manipulate algebraic expressions with positive, negative, zero or fractional indices Differentiate the laws of indices and apply laws appropriately Identify errors in the four operations on indices, surds and use of laws of indices Novice & Advanced Beginner Express numbers in standard form Perform four operations on simple algebraic fractions
24. 5.1 Quality criteria & amended rubrics Amended rubric with Katie’s comments on 4 Feb 2010 Indicators > Levels Manipulation of numbers in standard form notations Manipulation of algebraic expressions Application of Concepts Mathematical reasoning, communication and connections Expert Compares and contrasts alternative methods in the use of different laws, and multiple laws Explains errors / misconceptions in the: – algebraic manipulation, - use of the laws of indices Proficient Performs operations in standard forms expressions Manipulates algebraic fractions and polynomials (with positive, negative, zero and fractional indices), showing consistent and clear working Applies multiple laws of indices concurrently Identify errors in the four operations on indices, surds and use of laws of indices Competent Identifies large and small numbers such as giga, micro, pico Manipulates algebraic expressions with positive, negative, zero or fractional indices, showing clear working. Differentiates the laws of indices and apply laws Novice & Advanced Beginner Expresses numbers in standard form Performs operations on simple algebraic fractions
25.
26. 5.3 Final performance matrix See slide notes for explanation Manipulation of numbers in standard form notations Knowledge of mathematical nomenclature in standard form and numbers Comparison of numbers Manipulation of algebraic expressions Application of laws of indices and concepts Mathematical reasoning and communication in algebraic manipulation Mathematical reasoning and communication in indices Cut off points Expert Manipulates algebraic fractions and polynomials (with positive, negative, zero indices), showing consistent and clear working Explains strategies used in the algebraic manipulation 13 to 14 Proficient Applies multiple laws of indices concurrently States the types of algebraic manipulation States the operations on indices, surds or laws of indices used in the working 10 to 12 Competent Performs operations involving standard form expressions Manipulates algebraic expressions with positive and negative indices, showing clear working. Differentiates the laws of indices and applies the law(s) 7 to 9 Advanced beginner Manipulate numbers with negative, positive powers and standard forms by moving decimal places Compares small and large numbers by using strategies e.g. converting to common powers or common forms Performs operations on single algebraic expressions 4 to 6 Novice Manipulates numbers with either negative or positive powers by moving decimal place in a single direction Expresses small and large numbers using standard form, giga, micro, pico etc. Compares numbers without showing clear working. 1 to 3
34. 7.1 Zone of Proximal Development (ZPD) Vygotsky “ ...a state of readiness in which a student will be able to make certain kinds of conceptual connections, but not others; anything too simple for the student will quickly become boring; anything too difficult will quickly become demoralising”. So, when is a student ready to learn? A zone in which an individual can learn more with assistance than he or she can manage alone.
42. 7.3 Analysis: commonalities Pupils Learning difficulties Possible interventions This applies to all students except Norazah. For example, both Isabella & Suzanne could apply multiple laws of indices concurrently, but seemed to have difficulty in reasoning for algebra in this assessment. General observation of greater difficulty in algebra compared to indices. Revision of concepts and understanding 14 out of 22 students These students seemed to have difficulty with the language for Math i.e. Identifying large and small numbers such as giga, pico. Emphasis of the importance to remember the language
43. 7.4 Analysis: anomalies Pupils Learning difficulties Possible interventions Ke Tian She seemed to have problems stating the laws of indices in this assessment, though she was able to apply multiple laws. Cause – Problem with language Intervention – Enhancing the use of math language Stacy She seemed to have problems differentiating the laws of indices in this assessment, though she was able to apply multiple laws. Cause – Learning the steps by rote learning Intervention – Starting from the laws of indices, students learn how questions can be derived from individual laws. Amanda She seemed to exhibit reasoning for strategies she used. She had problems with manipulation and application of indices and algebra in this assessment. Cause – Understanding of the concepts were not in depth Intervention – Enhancing conceptual understanding Nur Zahwah She seemed to be able to differentiate the laws of indices. She had problems with manipulation and application of indices and algebra in this assessment. Cause – Understanding of the concepts were not in depth Intervention – Enhancing conceptual understanding
44. 8 Reporting 8.1 Reporting for students and parents 8.2 Reporting for Math teacher and school
45.
46. 8.1 Scope of Assessment & Audience Scope of Assessment Term Tests & Semester Examinations Class Tests Students & Parents Teachers School
51. Summary statements of learning at each level Level 4: At this level, the student is learning to apply multiple laws of indices concurrently. She/he is also learning to state the types of algebraic manipulation and the operations on indices, surds or laws of indices used in the working. Level 1: At this level the student is learning to manipulate numbers with either negative or positive powers by moving decimal place in a single direction. She/he is learning to use the knowledge of common terms like standard form, small and large numbers such as giga, micro, pico. She/he is also learning to compare numbers without showing clear working. Level 5: At this level, the student is learning to manipulate algebraic fractions and polynomials (with positive, negative, zero indices), showing consistent and clear working. She/he is also learning to explain strategies used in the algebraic manipulation. Level 3: At this level, the student is learning to manipulate algebraic expressions with positive, negative, zero or fractional indices, showing clear working. She/he is also learning to differentiate the laws of indices and applying laws. . Level 2: At this level the student is learning to manipulate numbers with negative and positive powers by moving decimal places. She/he is also learning to compare numbers by using strategies e.g. converting to common powers or common forms. She/he is also learning to perform operations on simple algebraic fractions. Category: aesthetics Performance level descriptors Expert 13-14 Proficient 10-12 Competent 7-9 Advanced Beginner 4-6 Novice 1-3 Indicators 1. Manipulation of numbers in standard form notations 2. Language of common terms in standard form and number 3. Comparison 4. Manipulation of algebraic expressions 5. Application of concepts 6. Mathematical reasoning, communication for algebra 7. Mathematical reasoning, communication for indices
52. Student rocket report A B C D E F 50% of the class can be located within this range The student is estimated to be at the location Inter-quartile range Student Achievement Level Level descriptions Levels B. Student can perform simple sequences in the manipulation of numbers (i.e. with either positive or negative powers). D. Student can manipulate numbers in standard form and algebraic expressions. Student is able to routinely apply the law(s) of indices. F. Student can perform and explain the manipulation of complex algebraic expressions combined with laws of indices . Student is proficient in demonstrating clear strategies and stating their analytical approaches. that deep understanding of application of the laws of indices. C. Student can perform manipulation of numbers and single algebraic expressions. Student is able to compare magnitude of numbers using clear strategy. E. Student can manipulate numbers in standard form and algebraic expressions. Student is proficient in the application of multiple laws of indices concurrently . A. There is no evidence of student’s ability to manipulate numbers and algebraic expressions.
61. Target students Intervention strategies Area owner / description Resources Whole class then one-to-one Review results Teacher- student individual conferencing Teacher to review results and compare to previous results as well as prior knowledge about students to sieve out ‘abnormal’ results. Call students individually to ask what they were going through when they did the paper. Establish if the error is based on mathematical misconception. Teacher training to analyse based on individual profiles. Whole class Include revision before teaching new proper HOD to review input of previous year revision prior to new topic in SOW (teacher to recommend). Review inclusion of metacognition and building mathematical language during lessons. Consider including certain types of questions in formative assessment. Teacher training in building mathematical language teaching and assessment. Sharing during buzz sessions. Teachers need to have a database of questions they can tap on to be used for other intervention strategies. From the maths sharing portal, pick out questions that test different skills, concepts, processes etc and categorise these questions. Whole class n>20 Review of topic Teacher to take one period to go through the important skills or go through lesson to build mathematical language. Affected students 10<n<20 Remediation based on indicators Teacher to go through similar questions and common misconceptions Affected students 1<n<10 Short term buddy system based on indicators Individual students would be paired up with students who have been identified to be able to do these sub-sections well to verbalise strategies and try similar questions together at own time. Student need to verbalise strategies to buddy. Checklist / rubrics for buddy and partner to work on so they can monitor their own progress and report to teacher during stipulated times. Level 1 students One-to-one attention with teacher: face to face & online Students to meet up with teacher to go through the questions they have problems in. Have online questions to do individually. Teacher can monitor students’ progress online. Online learning portal* All students have access to computer and internet.
63. Validity 9 Type of validity Suggestions for improvement Content validity - This assessment was showed that the content of the assessment tasks were closely related to the school syllabus and subject matter (national math syllabus). This was also contributed by the rich teaching experiences of three math teachers, including the assessor. Use of multiple tasks and multiple sources of evidence as the basis for judgment.
66. Reliability 9 Type of Reliability Suggestions for improvement The overlap between 0s and 1s is not fairly wide (Guttman chart ) which shows fair amount of consistency. A fairly reliable assessment allows a more clearly defined ZPD. Standard administration – The team established and documented clear assessment procedures/instructions for collecting, analysing and recording outcomes. The team used multiple tasks of evidence as the basis for judgment. Inter-rater reliability through paneling – There was a consistency of judgement and moderation of the judgements across different team members using the same assessment task and procedure. Involvement of expertise - The team members and assessor are experienced teachers, three of them are specialised in teaching math and one curriculum officer who have demonstrated competence in the field. Elimination of noise - ‘Noise’ due to individual bias is eliminated when the team reviewed and moderated the competencies in the assessment framework. The team members reflected on their judgement error in competency based assessment and biases. Reliability could be improved with the help of others. As we believe that professional development is social in nature, team effort is useful in helping to improve the reliability. We could use assessors with expertise in competency based assessment. Maintain representative sample of assessment tasks to compare from context to context/year to year and use a panel of independent assessors to evaluate this sample. Use multiple sources of evidence as the basis for judgment.
68. Scaling up 9 There are issues to consider when scaling up from class test to school formal assessments or from one departmen t to many or from written tests to projects and presentations. For example, will the leadership support teacher training or create more platforms for teachers to discuss in a professional learning team etc. These complexities will be discussed in details in the following slides.
69.
70.
71.
72.
73.
74.
75.
76.
77.
78. 4. Scaling up process at school level (an example) 9
83. 10.2 Reflection on how the interpretation was done 9 Reflection point Thoughts/Implications Importance of Teacher Judgment There were many situations where it was not clear cut that the students demonstrated or did not demonstrate the competency. Teacher’s expertise and experience are important to make accurate judgments. This implied that the teacher needs to be an expert and experienced one. If not, having more than one marker would help in both getting the more accurate judgments and developing the competencies of the teacher. Understanding the continuity of development We should not just look at this assessment as an isolated event. Instead, knowing the students and their development in Mathematics as a whole would help the teacher develop better intervention for the students.
84.
85.
Editor's Notes
The following cycle presents an overview of our presentation, as well as the process of the items developmental study by the team. The aim of this study is to develop and explore the use of a standards referenced assessment framework, and to propose interventions and reporting methods for the students using the Guttman chart and ZPD. While this is the process that we undertook for the project, this slide will serve as the structure for our presentation. It is somewhat our ‘content page’ as we arranged our slides based on the process that we went through. Just to highlight that this is the process that we went through as we complete the project. However, if given more time and opportunity to interact with the students, we would have extended the processes to a cycle which would involve archiving and further panelling for future teaching of the students to help them progress in their Mathematics. 1. Deciding on a Developmental Continuum – The team made reference to MOE national framework for Mathematics and the p rofessional learning experiences of the teachers in the team. Due to the competency requirements for units like Indices and Algebra, the team decided the use of Dreyfus model to observe or identify the skills and knowledge in the disciplinary learning of Mathematics. 2. Developing a standards referenced assessment framework - The rubrics were written with inputs from the whole team. The team deliberated on the strand indicators including domains, capabilities, performance indicators and quality criteria. 3. Drafting the assessment items - The assessment items were drafted with the understanding students’ background from the team. 4. Panelling - The team members reviewed and critiqued on the test items and rubrics, we also obtained the inputs from Dr. Katie Richardson. 5. Revising the assessment framework and test items – The team members made further amendments to the strand indicators including domains, capabilities, performance indicators and quality criteria. 6. Implementation of test - The assessment was administered by a teacher outside our team and the their answers were marked by the team. 7. Analysis of results – The Guttman chart for this group of students’ performance was plotted to find the Zone of Proximal Development (ZPD). The chart was analyzed for commonalities and anomalies, and possible interventions were proposed. We also deliberated on our calibration of five levels using cut-off points. 8. Reporting – The team reported the students’ individual developmental levels as well as the group developmental levels corresponding to the indicators. This was done by means of a rocket chart for the whole group of students and graphical charts of individual students. This also included anomalies. The students’ development and their possible interventions were discussed with the Math teacher teaching the class. 9. Reflection – The team reflected on the quality of the assessment framework, test items and rubrics, as well as reliability and validity of the study.
The school we had access to is a relatively good government school which has been granted autonomous status. It is an all-girls’ school and it offers all Science combinations when the girls are in Secondary 3 i.e. The girls take 2-3 science subjects, 2 Mathematics, 2 languages and 2 humanities subjects. The class is a Secondary 3 class (15 year olds). The teacher tells us that the class is generally weak in Mathematics as compared to their peers. In their cohort, they would be relatively ranked 6 th out of 9 classes in terms of Mathematics abilities. This class is the only one that offers Drama and Economics in their subject combination and the girls prefer language and the Arts as opposed to Mathematics/ Sciences.
When we looked at the school’s scheme of work for Mathematics and in our prior discussion with the Mathematics teacher, we narrowed the topics based on the students’ /school’s needs and took a step further to explicitly align our project based on the national curriculum standards.
When we looked at the school’s scheme of work for Mathematics and in our prior discussion with the Mathematics teacher, we narrowed the topics based on the students’ /school’s needs and took a step further to explicitly align our project based on the national curriculum standards.
Alignment to National Curriculum: The Ministry of Education (MOE), Singapore uses this framework to show the underlying principles of an effective mathematics programme that is applicable to all levels, from the primary to A-levels. It sets the direction for the teaching, learning, and assessment of mathematics. This framework guided us to tease out what we wanted to focus on in our rubrics. We acknowledge that in any particular Maths test, it would be difficult to put assess all the 5 tenets of Mathematics framework. The development of mathematical problem solving ability is dependent on five interrelated components, namely, Concepts, Skills, Processes, Attitudes and Metacognition. In the next slide, we will highlight the ones that we will focus on for this project.
The three components we will be exploring are ‘skills’, ‘concepts’ and ‘processes’. We find that it is difficult to measure ‘attitudes’ especially since the test will be administered by a proxy researcher whom we had no direct contact with to train to observe and identify measurable ‘attitudes’ traits / discourse or meta Mathematical concepts - cover numerical, algebraic, geometrical, statistical, probabilistic, and analytical concepts. Students should develop and explore the mathematics ideas in depth, and see that mathematics is an integrated whole, not merely isolated pieces of knowledge. Skills - include procedural skills for numerical calculation, algebraic manipulation, spatial visualization, data analysis, measurement, use of mathematical tools, and estimation. The development of skill proficiencies in students is essential in the learning and application of mathematics. Although students should become competent in the various mathematical skills, over-emphasising procedural skills without understanding the underlying mathematical principles should be avoided.Skill proficiencies include the ability to use technology confidently, where appropriate, for exploration and problem solving. It is important also to incorporate the use of thinking skills and heuristics in the process of the development of skills proficiencie Processes - refer to the knowledge skills (or process skills) involved in the process of acquiring and applying mathematical knowledge. This includes reasoning, communication and connections, thinking skills and heuristics, and application and modelling. Reasoning, communication and connections Mathematical reasoning refers to the ability to analyse mathematical situations and construct logical arguments. It is a habit of mind that can be developed through the applications of mathematics in different contexts. Communication refers to the ability to use mathematical language to express mathematical ideas and arguments precisely, concisely and logically. It helps students develop their own understanding of mathematics and sharpen their mathematical thinking. Connections refer to the ability to see and make linkages among mathematical ideas, between mathematics and other subjects, and between mathematics and everyday life. This helps students make sense of what they learn in mathematics. Thinking skills and heuristics Students should use various thinking skills and heuristics to help them solve mathematical problems. Thinking skills are skills that can be used in a thinking process, such as classifying, comparing, sequencing, analysing parts and wholes, identifying patterns and relationships, induction, deduction and spatial visualisation. Some examples of heuristics are listed below and grouped in four categories according to how they are used: • To give a representation, e.g. draw a diagram, make a list, use equations • To make a calculated guess, e.g. guess and check, look for patterns, make suppositions • To go through the process, e.g. act it out, work backwards, before-after • To change the problem, e.g. restate the problem, simplify the problem, solve part of the problem Applications Applications and modelling play a vital role in the development of mathematical understanding and competencies. It is important that students apply mathematical problem-solving skills and reasoning skills to tackle a variety of problems, including real-world problems. Mathematical modelling is the process of formulating and improving a mathematical model to represent and solve real-world problems. Through mathematical modelling, students learn to use a variety of representations of data, and to select and apply appropriate mathematical methods and tools in solving real-world problems. The opportunity to deal with empirical data and use mathematical tools for data analysis should be part of the learning at all levels. Attitudes refer to the affective aspects of mathematics learning such as: • Beliefs about mathematics and its usefulness • Interest and enjoyment in learning mathematics • Appreciation of the beauty and power of mathematics • Confidence in using mathematics • Perseverance in solving a problem Students’ attitudes towards mathematics are shaped by their learning experiences. Making the learning of mathematics fun, meaningful and relevant goes a long way to inculcating positive attitudes towards the subject. Care and attention should be given to the design of the learning activities, to build confidence in and develop appreciation for the subject. Metacognition - “thinking about thinking”, refers to the awareness of, and the ability to control one's thinking processes, in particular the selection and use of problem-solving strategies. It includes monitoring of one's own thinking, and self-regulation of learning. The provision of metacognitive experience is necessary to help students develop their problem solving abilities. The following activities may be used to develop the metacognitive awareness of students and to enrich their metacognitive experience: • Expose students to general problem solving skills, thinking skills and heuristics, and how these skills can be applied to solve problems. • Encourage students to think aloud the strategies and methods they use to solve particular problems. • Provide students with problems that require planning (before solving) and evaluation (after solving). • Encourage students to seek alternative ways of solving the same problem and to check the appropriateness and reasonableness of the answer. • Allow students to discuss how to solve a particular problem and to explain the different methods that they use for solving the problem.
We decided to tap onto the Dreyfus model for our learning framework. This model describes the progression from novice to expert. We deliberated and pegged the subject matter experts and connoisseurs in the domain. We asked ourselves questions on: What would a novice do? What would an expert do? We took note that the highest levels of criteria should stretch even the very best students yet we have to bear in mind that the assessment needs to be realistic and should be ‘do-able’. candidate and every student should find, in the criteria, a personal set of stretch targets. This is the way everyone can grow.
Based on the Dreyfus Model (learning framework) and the national math curriculum, the pedagogical framework is used to develop the assessment framework. The pedagogical framework consists of: Domain: The domain in our study is Math. Math has a set of skills, knowledge, behaviours and dispositions that enable us to sample and define strands and constructs specific to the subject. Strands: The strands and capability, or unit is “ Numbers & Algebra”. Numbers and Algebra is a major strand that consists of a few big ideas that are learnable, teachable. They too are the set of skills, knowledge and expectations or learning outcomes. They are often referred to as the units in the math curriculum. Indicators: The indicators are identified behavioural indicators. These indicative behaviours are things that a student could do, say, make or write , and from which we infer their capability in the strand. Criteria: The criteria are observational statements that detailed ‘how well’ each behaviour could be performed were created. These criteria underpin the profiling of learning pathways. The criteria are represented by ordered levels of increasing difficulty and sophistication The next slide will present these elements of the pedagogical framework in the context of this study.
There were 4 performance indicators identified as follows: 1.1 Manipulation of numbers in standard form notations. There are two sub-indicators, as shown in the slide. These were subsequently presented as individual indicators; i.e. after paneling and revision of the test items: Knowledge of mathematical nomenclature in standard form and numbers Comparison of numbers 1.2 Manipulation of algebraic expressions 1.3 Application of laws of indices and concepts 1.4 Mathematical reasoning and communication
The pedagogical framework shows the domain: Math. It also shows three of the major strands in the math curriculum: (1) Numbers and algebra, (2) Geometry and (3) Statistics. The focus of this study is on the (i) Numbers and algebra. Based on the national math curriculum, numbers and algebra are one of the major topics (or capabilities) that students need to possess. There are four indicators that are specific to this strand as shown in the slides:1.1 Manipulation of numbers in standard form notations; 1.2 Manipulation of algebraic expressions; 1.3 Application of concepts and 1.3 Mathematical reasoning, communication and connections. In the right most column, the list of criteria for each indicators is developed.
An initial rubric was developed as shown in this slide. The Dreyfus Model was adapted, with four levels: Expert, Proficient, Competent, and Novice-Advanced Beginner. (See next slide for further elaboration on how the group has derived at the levels – based on the Dreyfus Model as our learning framework; the National curriculum as our pedagogical framework; our teaching practice and experiences.)
An initial rubric was developed as shown in this slide. The Dreyfus Model was adapted, with four levels: Expert, Proficient, Competent, and Novice-Advanced Beginner. For Indicator 1: Manipulation of standard form: At the novice level, students are introduced to the nomenclature, and would be able to express familiar numbers in simple standard forms. At competent level, the manipulation of very small and very large numbers would be done. Moving to the proficient level, the students would be able to perform the operations, namely the addition, subtraction, multiplication and division of standard form expressions. For Indicator 2: Manipulation of algebraic expressions: At the novice level, students would be able to perform operations on simple algebraic expressions and fractions. At the competent level, students would be able to manipulate algebraic expressions and fractions with indices (or powers). The nature of the math curriculum is spiral, such that students would be taught algebraic manipulation for the past two years with increasing complexity, and by the time they are in third year (secondary three), they would be taught how to manipulate algebraic fractions with indices. At the proficient level, students would be expected to manipulate algebraic fractions and polynomials with positive, negative, zero and fractional indices. Note: This was adjusted to the expert level subsequently, when checked against students’ levels and sequence of learning – where more algebraic manipulation would be tested in the later part of the year. For Indicator 3: Application of concepts in laws of indices: The laws of indices is a major concept at upper secondary, so it was not considered as a novice or advanced beginner level. At the competent level, students are expected to be able to differentiate which laws to use, and to apply the laws appropriately. They would only be required to apply the isolated laws. As mentioned in the learning framework, there would be conscious deliberate planning. Students would show standardized and routinized procedures in applying each law. At the proficient level, students would be expected to apply multiple laws concurrently. At the expert level, students would be able to compare and contrast alternative methods in the use of different laws. This criteria was removed subsequently because it was beyond the level of the students’ ability, and it also required a higher level of reasoning. For Indicator 4: Mathematical reasoning, communication and connections: The ability to articulate reasons, and communicate strategies and connections was beyond a novice or advanced beginner level, as students at lower secondary level need only solve questions in routinized manner, and rarely requiring them to state the reasons or laws used. At competent level, the students are able to identify and explain errors in the use of indices. It would be inferred that they have understood and concept, and could communicate their understanding using mathematical reasoning. At the expert level, students are able to provide explanations with regards to errors in algebra and the laws of indices concurrently. These last two criteria were amended and adjusted, as it was beyond the students’ level.
Seven questions were then set based on the initial indicators and rubrics.
A first round of panelling was conducted. Panelling refers to: checking of the test items and initial rubric using a group (panel) of specialists chaired by one member of the group appointed as a leader The test items and rubrics were checked by 5 teachers. T he seven questions were checked against the indicators and the respective criteria. The reviewing or panelling of the draft test items consisted of: - making specific ideas for change - checking the learning area and year level - identifyng content range and gaps Each panel member review the test items and the rubric, and makes notes about faults and recommendations to fix the fault
The comments from the teachers’ reviews were collated. The main issues and comments that require review fall into the following three areas for improvement: 1. Refine questions to target students’ level and syllabus 2. Rephrase questions to avoid ambiguity and biases 3. Adjust criteria and levels
This slide and the next slide, show some of the teachers’ comments, organized under the three areas for improvement: 1. Refine questions to target students’ level and syllabus 2. Rephrase questions to avoid ambiguity and biases 3. Adjust criteria and levels
This slide and the next slide, show some of the teachers’ comments, organized under the three areas for improvement: Refine questions to target students’ level and syllabus 2. Rephrase questions to avoid ambiguity and biases 3. Adjust criteria and levels
An initial amendment was made to the rubric, and this tentative rubric was sent to Katie for comments. However, the rubric and questions needed further refinement, as shown in the next few slides.
Subsequently, the (i) rubrics (indicators and criteria) for each individual test item and (ii) the test items, were refined to better reflect the skills and observable behaviors more accurately. Considerations were made with regards to (i) content of the test items (ii) students’ developmental sequence had to be rechecked against the test items and rubric. The test items and criteria were also rechecked against he national curriculum and standards. We also noted that not all behaviours can be directly observed using the pen-paper test items. In addition, we also noted that while the indicators used to define the continuum are related, there is no causal or dependent relationship between them. It is neither necessary nor obligatory to observe lower order indicators in order to observe higher order behaviours. The existence of higher order indicators implies the ability to demonstrate lower order indicative behaviour. The relationship is probabilistic, not causal .
A final performance matrix was developed that consisted of the five levels in the Dreyfus Model. There are seven indicators in all, thereby breaking the indicators down to more specific criterion for the assessment purpose. The details and alignment with the learning framework, as well as teaching practices and learning levels are explained here: For Indicator 1: Manipulation of standard form: At the novice level, students are able to express numbers in simple standard forms by moving the decimal places in a single direction; i.e. in rigid adherence to taught procedures, having no discretionary judgment. At advanced beginner level , the students would be able to manipulate the numbers with basic guidelines; i.e. they can recognize and discern the direction required to move the decimal places, only after some prior experience, in order to manipulate the numbers in standard forms. At competent level , in order of increasing difficulty, the students would be able to perform the operations, namely the addition, subtraction, multiplication and division of standard form expressions. Students would show conscious deliberate planning in using standardised and routinised procedures to perform the operations, to solve problems that have single or best answers. For Indicator 2: Knowledge of mathematical nomenclature This indicator and corresponding criterion are based on the national curriculum that requires students to have knowledge. This is based on Dreyfus Model where at the novice level, students are able to remember (recall) mathematical terminology, and in this strand (or unite), it refers to students having knowledge of the mathematical nomenclatures in standard forms and numbers. This is also a lower level, in line with the national curriculum and standard. For this assessment, it had to be an isolated indicator, rather than merged under Indicator (1) “manipulation of numbers in standard form” (as planned initially) because it was noted that students who may know the nomenclatures may not be able to perform the manipulation, and vice-versa. In order to allow us to assess the students with more clarity and specificity in terms of the content knowledge, we found that there was a need to separate the indicator. For Indicator 3: Comparison of numbers This indicator (and its corresponding criteria) was also pulled out as an isolated indicator, as it was an aspect that we could assess from the questions, but we were unable to if it was subsumed under indicator 1. There are only 2 levels, and the questions were aligned, such that the advanced beginner level would show strategy in comparing numbers in standard forms. For Indicator 4: Manipulation of algebraic expressions: At the novice level, students would be able to perform operations on simple algebraic expressions and fractions. At the competent level, students would be able to manipulate algebraic expressions and fractions with indices (or powers). The nature of the math curriculum is spiral, such that students would be taught algebraic manipulation for the past two years with increasing complexity, and by the time they are in third year (secondary three), they would be taught how to manipulate algebraic fractions with indices. At the expert level, students would be operating beyond the competent level where students show standardized and routinized procedures (competent level). The students would be expected to manipulate algebraic fractions and polynomials with positive, negative, zero and fractional indices. This was adjusted to the expert level in order to align it to the students’ levels and the sequence of learning – where students would be exposed to more algebraic manipulation in the later part of the year. For Indicator 5: Application of concepts in laws of indices: The laws of indices is a major concept at upper secondary, so it was not considered as a novice or advanced beginner level. At the competent level, students are expected to be able to differentiate which laws to use, and to apply the laws appropriately. They would only be required to apply the isolated laws. As mentioned in the learning framework, there would be conscious deliberate planning. Students would show standardized and routinized procedures in applying each law. At the proficient level, students would be expected to apply multiple laws concurrently. The criteria at the expert level was removed because having students “to compare and contrast alternative methods in the use of different laws” was beyond the level of the students’ ability, and it also required a higher level of reasoning, to better the align student levels, test items and the curriculum standards. For Indicator 6 and 7: Indicator 6 Mathematical reasoning, communication and connections in algebraic manipulation : Indicator 7 Mathematical reasoning, communication and connections in indices : The ability to articulate reasons, and communicate strategies and connections was beyond a competent level, as students at lower secondary level need only solve questions in routinized manner, and rarely requiring them to state the reasons or laws used. These criteria were amended and adjusted, to better align it to the students’ level. Students would be operating beyond the competent level (students show standardized and routinized procedures). At proficient level, instead of explaining errors as intended initially, the students would be able to state their strategies in algebraic manipulation (Indicator 6) and the use of indices (Indicator 7). Based on the learning framework (Dreyfus Model), it would be inferred that they have understood and concept, and could communicate their understanding using mathematical reasoning where: At proficient level, being able to state their strategies, it would be inferred that they can see situations holistically and are able to identify strategies, differentiate and discern the laws of indices. At expert level, the students no longer relies on rules, guidelines or maxims. They are able to explain, not just state, the strategies used in algebraic manipulation (Indicator 6).
There are four test items and seven indicators. The first three indicators were assessed using questions 1, 2 and 3. Question 4 assessed the last four indicators.
These were the instructions to students: 1.Answer all the questions in the space provided. 2.Show workings clearly. Omission of essential workings will result in loss of marks. Duration: 30 minutes For Indicator 1: Manipulation of standard form: At the novice level, students are able to express numbers in simple standard forms by moving the decimal places in a single direction; i.e. in rigid adherence to taught procedures, having no discretionary judgment. At advanced beginner level, the students would be able to manipulate the numbers with basic guidelines; i.e. they can recognize and discern the direction required to move the decimal places, only after some prior experience, in order to manipulate the numbers in standard forms. At competent level, in order of increasing difficulty, the students would be able to perform the operations, namely the addition, subtraction, multiplication and division of standard form expressions. Students would show conscious deliberate planning in using standardised and routinised procedures to perform the operations, to solve problems that have single or best answers. For Indicator 2: Knowledge of mathematical nomenclature This indicator and corresponding criterion are based on the national curriculum that requires students to have knowledge. This is based on Dreyfus Model where at the novice level, students are able to remember (recall) mathematical terminology, and in this strand (or unite), it refers to students having knowledge of the mathematical nomenclatures in standard forms and numbers. This is also a lower level, in line with the national curriculum and standard. For this assessment, it had to be an isolated indicator, rather than merged under Indicator (1) “manipulation of numbers in standard form” (as planned initially) because it was noted that students who may know the nomenclatures may not be able to perform the manipulation, and vice-versa. Separating this indicator, to assess it, would allow us to assess the students with more clarity and specificity in terms of the content knowledge. For Indicator 3: Comparison of numbers This indicator (and its corresponding criteria) was also pulled out as an isolated indicator, as it was an aspect that we could assess from the questions, but we were unable to if it was subsumed under indicator 1. There are only 2 levels, and the questions were aligned, such that the advanced beginner level would show strategy in comparing numbers in standard forms.
For Indicator 4: Manipulation of algebraic expressions: At the novice level, students would be able to perform operations on simple algebraic expressions and fractions. At the competent level, students would be able to manipulate algebraic expressions and fractions with indices (or powers). The nature of the math curriculum is spiral, such that students would be taught algebraic manipulation for the past two years with increasing complexity, and by the time they are in third year (secondary three), they would be taught how to manipulate algebraic fractions with indices. At the expert level, students would be operating beyond the competent level where students show standardized and routinized procedures (competent level). The students would be expected to manipulate algebraic fractions and polynomials with positive, negative, zero and fractional indices. This was adjusted to the expert level in order to align it to the students’ levels and the sequence of learning – where students would be exposed to more algebraic manipulation in the later part of the year. For Indicator 5: Application of concepts in laws of indices: The laws of indices is a major concept at upper secondary, so it was not considered as a novice or advanced beginner level. At the competent level, students are expected to be able to differentiate which laws to use, and to apply the laws appropriately. They would only be required to apply the isolated laws. As mentioned in the learning framework, there would be conscious deliberate planning. Students would show standardized and routinized procedures in applying each law. At the proficient level, students would be expected to apply multiple laws concurrently. The criteria at the expert level was removed because having students “to compare and contrast alternative methods in the use of different laws” was beyond the level of the students’ ability, and it also required a higher level of reasoning, to better the align student levels, test items and the curriculum standards. For Indicator 6 and 7: Indicator 6 Mathematical reasoning, communication and connections in algebraic manipulation : Indicator 7 Mathematical reasoning, communication and connections in indices : The ability to articulate reasons, and communicate strategies and connections was beyond a competent level, as students at lower secondary level need only solve questions in routinized manner, and rarely requiring them to state the reasons or laws used. These criteria were amended and adjusted, to better align it to the students’ level. Students would be operating beyond the competent level (students show standardized and routinized procedures). At proficient level, instead of explaining errors as intended initially, the students would be able to state their strategies in algebraic manipulation (Indicator 6) and the use of indices (Indicator 7). Based on the learning framework (Dreyfus Model), it would be inferred that they have understood and concept, and could communicate their understanding using mathematical reasoning where: At proficient level, being able to state their strategies, it would be inferred that they can see situations holistically and are able to identify strategies, differentiate and discern the laws of indices. At expert level, the students no longer relies on rules, guidelines or maxims. They are able to explain, not just state, the strategies used in algebraic manipulation (Indicator 6).
The test was conducted during a Maths lesson, administered as a pop quiz. We had communicated with the trainee’s co-operating teacher who requested to the trainee for this test to be slotted in. The last topic prior to the test was indices although there was a break because the girls had their cohort camp in between the time the topic was taught and the test duration. Algebraic manipulation on the other hand was taught the year before when they were in Secondary 2.
In the context of the school, any written tests/ exams are conducted in a similar manner, so the girls already know the drill especially these girls are in Secondary 3. However, the above instruction allows teachers or anybody even from another school, to repeat the administration of the task.
Griffin also highlighted that ZPD is the point at which the student is most ready to learn. It is not t he students’ level of achievement but the level at which they are working or developing. Here, Vygotsky emphasized that children can be on the ‘verge’ of being able to solve a problem at any given time. This points to the importance of providing appropriate and timely interventions to help in the learning of the students. To assist their problem solving, different students will require different strategies, for example, structure, encouragement, reminders, demonstrations, co-operative learning with discussion. Relooking at our own curriculum, the team could consider the implications for structure and timing of teaching and learning e.g. Are there strategies associated with the levels?
General learning difficulties: The students seemed to be scoring better in more indicators for indices i.e. indicator 1a, 1b, 1c, 3a, 3b. The students also seemed to have difficulty in a few indicators for indices, i.e. 5a, 5b, 7a. On the other hand, students seemed to have more difficulty with more indicators for algebra, i.e. indicator 4a, 4b, 4c, 6a, 6b. This seemed to imply a weak foundation in algebra. An example using two students is illustrated in the next slide. Point of interventions for students at each level of development: The ZPD shows the region of readiness for each cluster of students to be developed in order to progress towards the next level. Hence, it also identifies and recommends the next point of intervention for each cluster of students. For example, the eleven students at level 2 (advance beginner) are able to perform manipulation of numbers in positive and negative powers, but unable to perform manipulation of algebra and numbers in standard form. This shows the possible next point of intervention for this group of students.
Both Isabelle & Suzanne could apply multiple laws of indices concurrently, but seemed to have difficulty in reasoning for algebra in this assessment. This was echoed by the general observation of greater difficulty in algebra compared to indices. What could be the cause? This could be attributed to “fragile knowledge” where there was no deep conceptual understanding. It could also be attributed to the gap in remembering last year’s unit on algebra. What resources might be needed to help the class move up the developmental continuum?
14 out of 22 students seemed to have difficulty with the language for Math i.e. Identifying large and small numbers such as giga, pico. This is related to nomenclature could be attributed to the lack of recall or emphasis towards the language. Perhaps the use of memory aids like mnemonics or stories can help to enhance the recall. Though this indicator to test recall is reflected in the math syllabus by MOE, how far does it indicate the students’ ability in math?
Ke Tian was assessed as being “proficient”, but she seemed to have problems stating the laws of indices in this assessment, though she was able to apply multiple laws. Is this an anomaly on this occasion? What could be the cause? What resources might be needed to help Ke Tian move up the developmental continuum?
Stacy was assessed as being “proficient”, but she seemed to have problems differentiating the laws of indices in this assessment, though she was able to apply multiple laws. Is it an anomaly on this occasion? What could be the cause? What resources might be needed to help Stacy move up the developmental continuum?
Amanda was assessed as being “advanced beginner”, but she seemed to exhibit reasoning for strategies she used. She had problems with manipulation and application of indices and algebra in this assessment. Is it an anomaly on this occasion? What could be the cause? What resources might be needed to help Amanda move up the developmental continuum?
Nur Zahwah was assessed as being “advanced beginner”, but she appeared to have been able to differentiate the laws of indices. She had problems with manipulation and application of indices and algebra in this assessment. Is it an anomaly on this occasion? What could be the cause? What resources might be needed to help Nur Zahwah move up the developmental continuum?
These were the common learning difficulties that the team observed of the whole group. The general observation that the students seemed to have greater difficulty in algebra compared to indices could be attributed to “fragile knowledge” where there was no deep conceptual understanding. It could also be attributed to the gap in remembering last year’s unit on algebra. On the other hand, the general observation that students failed to do well in question related to nomenclature could be attributed to the lack of recall or emphasis towards the language. Perhaps the use of memory aids like mnemonics or stories can help to enhance the recall.
For deeper conceptual understanding of mathematical concepts, these students should be given a variety of learning experiences to help them develop a deep understanding of, and to make sense of various mathematical ideas, as well as their connections and applications. The use of manipulatives (concrete materials), practical work, and use of technological aids should be part of the learning experiences of the students. Although not yet explored in this study, but depicted in the national mathematical framework by MOE (Singapore), the components of attitudes and metacognition also play important roles in encouraging students to question, think aloud, seek alternative ways of solving the same problem and to check the appropriateness and reasonableness of the answer. This deepens their thinking.
The Rocket report would allow the students to know what are their overall level of performance and where they are relative to the rest of the class. The individual report on the performance on each indicator would allow the students to know their strength and weakness. This would allow them to make sense of the intervention that follows. For the teacher, by grouping the students by competencies would allow the teacher to see the general performance across the class. This would help the teacher to determine the kind of intervention required for each competency.
This figure shows a framework that guides the reporting of assessment. This framework shows how the scope of the assessment and the audience is related. For class test that only be administrated to one class on a topic, the audience would be the students with their parents and the teachers (subject and form teachers). However, for term tests and examinations that are administrated to the whole level, the results would be reported to beyond the teachers to the school administrators such as the Head of department, Vice-principal, and Principal. Students and parents are always at the center of the reporting process.
This slide focuses on the reporting audience and extending from previous slide, it shows the method and purpose of reporting to the audience. In the next few slides, we will further show detailed samples of what student and parents will get and what teachers would use before proposing intervention.
This is the individual report card. The components of this report card will be further emphasized in the next few slides. Since Norazah did well in her assessment, the teacher would get her to be the buddy so that she can work on the other 2 areas that were not included in the test which is to develop the attitude and metacognition (national framework). By being a buddy, she would be able to think about her thought processes as she worked out the questions.
Celestine is weak in reasoning, communication and algebraic expressions. It is best for her to have one-to-have interaction with her buddy so that she can verbalise her processes.
In this slide, the level descriptions of the students at each level are presented. The descriptions are based on the performance of the students based on 7 indicators of the rubrics. The performance level descriptors are also pegged to the rocket report.
The rocket report is aimed at allowing the students and their parents to understand the competencies of the students based on their performance in the assessment. We have 6 levels with level A corresponding to a level with no evidence of learning. Level B to F correspond to the five levels of competency respectively: novice, advanced beginner, competent, proficient, and expert. With the level descriptors stated, the student would be able to understand his or her achievement level. Furthermore, he or she would be able to know the next achievement level and that would allow the intervention to be worked out between the students and the teacher. In addition, the rocket report also allows the student know where he or she stands with respect to the class. As the grey area represents the middle 50% of the class, if the achievement level is above the grey area, he or she would be in the top 25%.
Reporting for student and parents’ use. This would complement the rocket report. While rocket report gives an overall view of the achievement of the individual student, this chart would allow the student and parents to pinpoint the various aspects. For Norazah, it is clear that she is proficient in all indicators.
For Celestine, she is only proficient in indicator 3 and not as weak in indicator 1 followed by indicator 5 and 4 and finally 6 & 7. This would give the student and the teacher an idea on how to strategize the intervention.
With a simple assessment, it might not be possible to list different strands for comparison. In that case, we suggest to present the assessment data in this manner for the teacher. In this manner, the teacher would be able to plan for intervention and who the intervention would target. For example, if the teacher plans to work on the manipulation of numbers in standard form notations, Diane would definitely be involved judging from her performance in that indicator.
Similarly, by looking at indicator 4, is was clear that the manipulation of algebraic expressions weak for many students and few students would achieve the high level of proficiency.
In this slide, we present the various aspects to be considered when planning for individual and group. This is also aligned to the reporting presented earlier. For example, the rocket report and the chart showing individual’s achievements in various indicators allow the student to develop individual intervention with the teachers. When a student is weak in more than one area, the intervention would need to look into the ZPD of the student. Generally, in a class, personal customization might not be always possible. As a result, the intervention would need to address groups of students. The charts that we suggested for teachers would provide for such purpose as it would be easier for the teachers to identify the group’s developmental level.
When the teacher looks at the report given to her, she would be able to triangulate the student’s performance with her previous knowledge of the student’s Math performance, work attitude or general knowledge of the student. For example, she might be able to see why a student did not perform too well for the test and attribute it to mere competence or the student going through other types of problems at home etc. In this case, teacher could call up the students to have a one-to-one conferencing to establish what is the root of the problem. Looking at the individual indicators, teacher may wish to target specific areas for intervention. For example, for indicator 7, everyone did not get any score on them so teacher may wish to try the whole class intervention. If only 18 students have problems in that area, she may wish to have modular remediation meaning a one or two sessions specific to that area. The maximum number of students that can come down for remedial is capped at 20 (school policy). The girls are highly motivated and are able to check their own progress. Buddies are also responsible and are exposed to becoming good motivators and buddies as this strategy cuts across the school. For the level 1 students, teacher would give her one to one attention so that her Mathematical misconceptions can be addressed. Since the students are self-motivated, they would be able to complete the online questions individually and check their progress while keeping the teacher updated. The online portal also allows teacher to monitor the student’s work and gives the student direct access to asking the teachers questions online.
Content validity – As illustrated in the previous slides, t he assessment task as a whole, represents a range of the knowledge and skill specified within the competency standard, as shown by the range of variables and the evidence guides in the competency standards, the prepared and reviewed detailed task specifications covering the knowledge and skills to be assessed, as well as involvement of experienced teachers and curriculum officers in both the assessment task design and the review of the task’s match to the competency(ies).
When we looked at the school’s scheme of work for Mathematics and in our prior discussion with the Mathematics teacher, we narrowed the topics based on the students’ /school’s needs and took a step further to explicitly align our project based on the national curriculum standards.
When we looked at the school’s scheme of work for Mathematics and in our prior discussion with the Mathematics teacher, we narrowed the topics based on the students’ /school’s needs and took a step further to explicitly align our project based on the national curriculum standards.
The eighth and ninth point are related. From the Guttman chart, we would see that assessment has a fair amount of reliability and when we reflected, the team effort in marking and deliberation has contributed to the reliability other than the design of the questions and the rubrics. The reliability is defended in the administration procedure for assessment, marking and interpretation of rubrics, judgement call and professional experiences of the team, as well as control of noise by the team due to personal bias. Standard administration – The team established and documented clear assessment procedures/instructions for collecting, analysing and recording outcomes. The team used multiple tasks of evidence as the basis for judgment. Inter-rater reliability through paneling – There was a consistency of judgement and moderation of the judgements across different team members using the same assessment task and procedure. For example, there was a comparison across different assessment items, using assessors who demonstrate professional teaching experiences in discipline, and consistent judgment across team members to reduce bias Involvement of expertise - The team members and assessor are experienced teachers, three of them are specialised in teaching math and one curriculum officer who have demonstrated competence in the field. Elimination of noise - ‘Noise’ due to individual bias is eliminated when the team reviewed and moderated the competencies in the assessment framework. The team members reflected on their judgement error in competency based assessment and biases.
School Leadership School leaders have a strong influence on the likelihood of educational change (Fullan, 2001). They play the significant role in supporting and sustaining the change (Hargreaves and Earl, 2001) through creating the conditions in which school reform can succeed (Fullan, 1991).
Teaching practices Teaching practices need to be aligned with the change in assessment and reporting. Teaching practices needs to be evidence based. Teachers need to model the skills they want students to exhibit, and teachers need to focus on intervention strateges for differentiated teaching. Teachers also needs to move out of their ‘egg crates’ (isolation) and learn to work collaboratively, sharing research, and best practices to improve instruction. Teachers’ Lives & Their Work Teachers grapple with the intellectual and emotional elements (Hargreaves and Earl, 2001) of any educational change. It involves thinking about the initiative, their desirability and consequences ((Hargreaves and Earl, 2001). School leaders could provide differentiated approaches to accommodate teachers’ various stages of concern related to innovations, to create school-wide and individual buy-in to the innovations. This also includes policies to protect teachers from unnecessary work amid the school’s improvement plan and numerous distractions, keeping the focus on student learning and teachers’ well-being both professionally and personally. Other teacher factors influencing the success of the change include: Teacher beliefs and attitudes on Teachers’ pedagogical skills in Teachers’ knowledge based, which leads us to the next factor for considertion – Teacher PD.
Teacher Learning & Professional Development There are 5 areas where teachers need to be equipped in. Teachers’ professional development should be situated in collaborative work via learning communities (or PLT) in genuine work environment so that desired cognitive tools such as ideas, theories, and concepts can shape teacher practice in the right direction. This would improve teaching and learning through the use of data, teachers would be equipped to do the following: Link assessment to teaching and learning through the use of data and evidence-based decision making; Critical understanding and use of the language of assessment; Analysis of current and emerging issues in assessment and examinations; Use and evaluate approaches to assessment; Link and use relationship between assessment, teaching and learning; Report using a developmental learning framework. Teachers should be skilled in : choosing assessment methods appropriate for instructional decisions developing assessment methods appropriate for instructional decisions. administering , scoring and interpreting the results of both externally-produced and teacher-produced assessment methods. using assessment results when making decisions about individual students, planning teaching, developing curriculum, and school improvement. valid student [grading] procedures which use student assessments. communicating assessment results to students, parents, other lay audiences, and other educators. recognizing and [ avoiding ] unethical , illegal, and otherwise inappropriate assessment methods and uses of assessment information.
The emphasis here is towards formative assessments. As highlighted by Black & William (1998b), questioning and feedback are pivotal in improving communication links between teachers and students about the assessment aspects of their work. The face-to-face discussions also provide qualitative evidence for students’ attitudes and metacognition. In feedback by marking, the students get to respond to quality feedback on students’ thinking. In peer – and self-assessment, students would have greater ownership towards deciding how to make judgements and how to structure their next piece of work. They also recognise that teachers are not the sole source and evaluators of answers, they also play an important role. In this aspect, we expect the student to be able to more self-regulated in assessment and to: have a critical understanding and had able to use the language of assessment in a constructive and all manner. be able to analyse current and emerging issues in assessment examinations; use and evaluate specific approaches to assessment and reporting; link and use the relationship between assessment teaching and learning; be a reporter on learning using a developmental learning framework to make decisions about current and future learning and to target intervention
This few slides capture our reflection which took place throughout the whole process of completing this assignment. For the first point, Question 1 required students to do ranking and that could be potentially confusing for students. This question could have been asked in a clearer manner to avoid the confusion. Several students misinterpreted the requirement of the question. Also, the use of language in crafting questions e.g. ‘state the reason or strategy’ versus ‘explain the strategy. The first is thinking about the reason or strategy, while the latter is thinking about the student’s thinking. The latter is expecting the student to perform metacognition, which the students are not familiar with in their typical class tests. More guidance could be provided for the students. The next point is about allowing for different methods of solving the problem would make it difficult to assess the specific skills intended to be assessed. For example, when we asked the students to compare the magnitude of different numbers, students might not use the standard form notation. However, if the question asked to test specific competency in standard form using larger powers (positive and negative) , students will need to manipulate in standard form, rather than using a novice method like decimals.
The next point is about the importance of teacher expertise and experience in ensuring reliability and accuracy of the assessment. Understanding that development is continuous, the intervention would not be based on an isolated assessment. Instead, this would allow the teacher to link the information from each assessment to devise the best intervention for each individual students.
The holistic perspective allows us to look at the competencies instead of the accuracy of the answers. In this way, our approach would be developmental in nature taking into consideration how different aspects of teaching could come together to help the students learn. This assignment has helped us to have first hand experience of designing and implementing such assessment. This experience is important for us to appreciate the challenges and benefits. This would allow us to look for ways to overcome the challenges faced so as to help our student learn.
