Steiner—How to Improve Your Library Instruction: Assessment in Five Minutes
•Download as PPTX, PDF•
6 likes•4,764 views
Information literacy expert Sarah Steiner will show you how five minutes of skilled, targeted assessment is all you need to give you a precise understanding of students’ needs, learning, and retention.
Recommended
Recommended
More Related Content
What's hot
What's hot (13)
Viewers also liked
Viewers also liked (20)
Similar to Steiner—How to Improve Your Library Instruction: Assessment in Five Minutes
Similar to Steiner—How to Improve Your Library Instruction: Assessment in Five Minutes (20)
More from ALATechSource
More from ALATechSource (20)
Recently uploaded
Recently uploaded (20)
Steiner—How to Improve Your Library Instruction: Assessment in Five Minutes
- 1. How to Improve Your Library Instruction: Assessment in Five Minutes Sarah Steiner January 8, 2014
- 2. During Today’s Class, We Will… • Analyze the purpose and benefits of quick assessment in one-shot instruction sessions. • Draft learning outcomes for a session using Bloom’s Taxonomy. • Compose assessment questions based on our learning outcomes. • Select appropriate assessment tools based on your instruction goals and population. http://www.flickr.com/photos/topgold/4858921901/
- 3. Our Focus Classroom assessment (versus programmatic or institutional) http://www.flickr.com/photos/rogersg/3814863064/
- 4. What Can Quick Classroom Assessment Do? • It can identify instructional gaps or disconnects. • It can help you determine how you spend class time. • It can build your confidence. • It can provide evidence of efficacy and impact. • It should be a basis for change. http://www.flickr.com/photos/sercasey/324341982/
- 5. What Should I Assess? • To assess: – Comprehension – Retention – Achievement of learning outcomes • Not to assess: – – – – Your personal shortcomings Your speaking mistakes Your hairstyle The feng shui and temperature of the classroom http://www.flickr.com/photos/toniblay/52445415/
- 6. Have you ever written learning objectives or outcomes? • • • • Nope, not yet. Yes, a few times. Yes, regularly. I’m not sure. http://www.flickr.com/photos/dadaistic_fantastic/5089525186/
- 7. Learning Outcomes “A Learning Outcome is a statement of what the student should understand and be able to do as a result of what she has learned ... ‘the essential and enduring knowledge, abilities, and attitudes or dispositions’ that enable a learner to practice and apply her learning in the real world.” -Valencia Community College http://www.flickr.com/photos/gblakeley/5583354276/
- 8. A Good Learning Outcome Will… • • • • • • • Set a time frame and a context. Identify the audience. Be jargon-free. Be measurable. Be action-oriented. Be brief. Be linked to learner needs. “By the end of this class, you will be able to identify a scholarly source.” http://www.flickr.com/photos/auntiep/4310267/
- 9. Determine the Class Priorities • Choose three to five – What must the students accomplish? – What must they know in order to accomplish it? – What do they already know/find obvious? http://www.flickr.com/photos/paloetic/4795592340/
- 10. Write the Learning Outcomes Stem + Verb + Product/Outcome http://www.flickr.com/photos/jjpacres/3293117576/
- 11. Stem By the end of this class, you will be able to… http://www.flickr.com/photos/47108884@N07/4594962925/
- 13. Choose Verbs: Knowledge Students will… define, describe, identify, know, label, list, match, name, outline, recall, recognize, reproduce, select, state http://www.nwlink.com/~donclark/hrd/bloom.html http://www.flickr.com/photos/reway2007/4693202935/
- 14. Choose Verbs: Comprehension Students will… comprehend, convert, defend, distinguish, estimate, explain, extend, generalize, give an example, infer, interpret, paraphrase, predict, rewrite, summarize, translate http://www.flickr.com/photos/bsmith4815/112307904/
- 15. Choose Verbs: Application Students will… apply, change, compute, construct, demonstrate, discover, manipulate, modify, operate, predict, prepare, produce, relate, show, solve, use http://www.flickr.com/photos/28478778@N05/5729009434/
- 16. Choose Verbs: Analysis Students will… analyze, break down, compare, contrast, diagram, deconstruct, differentiate, discriminate, distinguish, identify, illustrate, infer, outline, relate, select, separate http://www.flickr.com/photos/billselak/2709756134/
- 17. Choose Verbs: Synthesis Students will… categorize, combine, compile, compose, create, devise, design, explain, generate, modify, organize, plan, rearrange, reconstruct, relate, reorganize, revise, rewrite, summarize, tell, write http://www.flickr.com/photos/littleredelf/3081832532/
- 18. Choose Verbs: Evaluation Students will… appraise, compare, conclude, contrast, criticize, critique, defend, describe, discriminate, evaluate, explain, interpret, justify, relate, summarize, support http://www.flickr.com/photos/christinamatheson/108818379/
- 19. Add Product or Outcome What do they need? • A book on their topic • A thesis statement • Keyword search strategies (basic or advanced) What is the context? • A class assignment • A real-world scenario http://www.flickr.com/photos/fdecomite/2907794565/
- 20. All Together Now! Stem + Verb + Product/Outcome By the end of this class, you will be able to identify a scholarly source on your paper topic. By the end of this session, you will be able to locate books on your paper topic in the library using the online catalog, GILFind. http://www.flickr.com/photos/revlimit/2186765034/
- 21. What’s Wrong Here? Today I’ll talk about… 1. The library website. 2. Database searching for peer-reviewed journals using boolean logic and the thesaurus. 3. How you can search the catalog, search the databases, and use ILL. http://www.flickr.com/photos/mythoto/2604818537/
- 23. Have you ever assessed students’ achievement of preset learning outcomes? • • • • Nope, not yet. Yes, a few times. Yes, regularly. I’m not sure. http://www.flickr.com/photos/dadaistic_fantastic/5089525186/
- 24. Today’s Assessment Tools • Polls • Quizzes (pre and post or post only) • Minute papers http://www.flickr.com/photos/azarius/225340667/
- 25. Polls
- 26. Poll Example Questions • How many of you have had a library instruction class here before? • Is this an article or a journal? • Is this website reputable? • Is this source peer-reviewed? / Is this source scholarly or popular? • Who is the author of this book? • Are you confused about topic x? http://www.flickr.com/photos/rakka/3451170932/
- 27. Quizzes
- 29. The Good, the Bad, the Ugly
- 31. Minute Paper Example Questions
- 32. Best Practices for Question Drafting • Convert your learning outcomes to questions. • Include three to five “meaty” questions, two or three general questions. • Avoid overly easy or “all of the above” answers. • Avoid nebulous or complicated questions. • Work in terminology to ensure it’s understood. http://www.flickr.com/photos/djmccrady/6118626233/
- 33. Best Practices for Question Drafting • Get a reviewer. • If at first you don’t succeed…. More help with constructing questions: http://ets.tlt.psu.edu/learningdesign/effective_ questions/why http://www.flickr.com/photos/djmccrady/6118626233/
- 34. Review Responses • On the spot? Yes! • Look for… • Percentage of correct answers. • Trends in missed answers. • What to change next time. http://www.flickr.com/photos/xingty/3421423768/
- 36. Survey and Poll Tools • SurveyMonkey: http://www.surveymonkey.com • Poll Daddy: http://polldaddy.com/ • Poll Everywhere: http://www.polleverywhere.com/ • Google Forms: http://www.google.com/google-d-s/forms/ http://www.flickr.com/photos/litlnemo/5045121567/
- 37. SurveyMonkey
- 38. PollDaddy
- 39. Poll Everywhere
- 40. Google Forms in Google Drive
- 41. Review • Write learning outcomes. • Base your questions on the outcomes and on your needs. • Review and reflect on the answers you get. http://www.flickr.com/photos/demonbaby/4228327167/
- 42. Thank you! Contact me at ssteiner@gsu.edu
- 43. Other Resources • “Bloom’s Taxonomy of Cognitive Domain.” http://classweb.gmu.edu/ndabbagh/Resources/Resources2/bloomstax.htm • Educational Oasis, “Framework of Learning Outcomes.” http://www.educationoasis.com/instruction/bt/learning_objectives.htm • London Deanery, “Setting Learning Objectives.” http://www.faculty.londondeanery.ac.uk/e-learning/setting-learningobjectives • Penn State, “Writing Effective Questions to Promote Learning.” http://ets.tlt.psu.edu/learningdesign/effective_questions/ • Utah State University Library, “Assessment.” http://library.usu.edu/instruct/assessment/index.php • “Virginia Tech Instruction Clearinghouse.” http://www.lib.vt.edu/RIS/clearinghouse/assessments.html
Editor's Notes
- When we think of assessment, we often think it needs to be really involved in order to be useful, or that we need a ton of time, but you can do a lot of meaningful work in just five minutes. You don’t always need to have pre and post tests, span an entire semester, have IRB approval and pre-testers and a million steps. All images are creative commons, cited in the bottom right corner.Do not feel like you have to write all of this down, because you’re going to get a link to the slides and to the recording, if I’m correct. I am going to pack in a lot of content so that you can have what you need to actually accomplish this
- Choose your questions and model based on the outcomes you need to meet.
- Today our focus is on classroom assessment rather than programmatic or institutional. What we’re talking about today is how to fit assessment in when you only have between 50 and 75 minutes. However, you’ll find that everything we talk about is easily expanded into those two larger spheres. Formative assessment is utilized to immediately determine whether students have learned what the instructor intended. This type of assessment is intended to help instructors identify material which needs to be clarified or re-taught and should not be used to evaluate or grade students. Results of formative assessment can assist instructors to ascertain whether curriculum or learning activities need to be modified during a class session or before the next class meets.
- Example of instructional gaps: terminology—article, journal, databaseAssessment is not an end in itself—it supports change and best practice in teaching.
- Did they take away what you thought they were going to take away? If not, why?My favorite lesson: don’t know what a database is. Don’t know the difference between an article and a journal. They tend to be highlyconstructive in their feedback.
- A lot of people have done this at one point or another, but even if they have, they aren’t based on Bloom’s, so we may have a different approach today than you took in the past. You may also have written the outcomes for yourself, as a plan for your instruction, but not used them as a jumping off point for assessment.
- Our first step in good, quick assessment is to write learning outcomes. The assessment questions which you choose will, in general relate specifically to those learning outcomes. They will also provide a roadmap for you, as the instructor, in terms of what you must cover. Whatever assessment method you choose later will tie back to these. A good learning outcome will state what the student will be able to do, and WHY. It will use Bloom’s taxonomy verbs of various cognitive levels.http://valenciacc.edu/facultydevelopment/tla/Candidate/documents/ExamplesandDefinitionsofLOs.pdfDifferent from an objective. Objective states what we aim to do. Outcome describes what the student will be able to do at the end. Learning outcomes will:Help you to focusHelp to reduce students being overwhelmed
- Review the outcomes for this sessionWhich one might be hard to measure?
- First step: consider the top three or four class priorities. Write them down. Find a bookIdentify a scholarly articleLocate a scholarly article“Learning outcomes should flow from a needs assessment. The needs assessment should determine the gap between an existing condition and a desired condition. Learning outcomes are statements which described a desired condition – that is, the knowledge, skills, or attitudes needed to fulfill the need. They represent the solution to the identified need or issue. Learning outcomes provide direction in the planning of a learning activity.”http://www.aallnet.org/Archived/Education-and-Events/cpe/outcomes.html
- Future tense—timeframeIdentifies learner (some people say “students”)
- Benjamin Bloom, an American educational psychologist, helped to write these in 1956. Domains of learning. Cognitive domain. Also Affective and Psychomotor domains.I want to go through these in depth a little, because they will directly inform your assessment questions. So, as you choose verbs, consider them not just as objectives, but as assessment questions for your post test.
- “Knowledge is the act of remembering previously learned material. It’s the lowest level of learning outcomes in the cognitive domain. “
- “Comprehension is the ability to grasp the meaning of material. Students can demonstration comprehension through interpretation or summarization of material.These learning outcomes are a stepbeyond just remembering. They represent the lowest level of understanding.”
- “Application refers to the ability to use learned material in new and concrete situations.”We do this a lot in one shots.
- “Analysis is the ability to break down material so that its organizational structure can be understood. Differentiation, comparing and contrasting, identification.
- “Synthesis is the ability to put parts together to form a new whole.”It can be hard to get to this in an hour. Our students often do this at the course level—they have to write a paper, a speech, a bibliography, a literature review. Synthesis is the ultimate goal, in many cases.
- “Evaluation is the ability to judge the value of material in a particular scenario. The judgments are to be based on definite criteria. Learning outcomes in this area are highest in the cognitive hierarchy because they contain elements of all the other categories, plus value judgments.Those of you who read quickly may be saying to yourself—I saw the same verb on more than one list. You are absolutely correct. Sometimes there are shades of gray with the verbs. Don’t worry too much about that—just strive to have things that seem to fall into the higher levels, whenever you think that’s feasible for your group of students. You may pick something that ultimately is too easy a task or too difficult a task. If you do that, that’s fine. Just change it next time and don’t beat yourself up. You have to start somewhere. Doing this whole process is just as much a learning exercise for us, as the instructors, as it is for the students.
- This should include context as often as possible without being too wordy or jargon-filled. So, for example, in this last I’ve said “advanced techniques” rather than saying boolean logic, subject header searches.We’ll get to those things, but not in our outcomes. You don’t have to have the context, but I like it. If you don’t include it in your written outcomes, still think about it and be able to articulate the context to the students.
- Stem and outcomeDemonstrable/assessableBloom’s cognitive domain—high levelNo jargonOften INCREMENTAL—step-based. They fit together to make a whole. You will have to make a determination whether you want to show these to your class or not. I didn’t use to show them, but now I always do, especially when I’m going to assess them, because the assessment questions will tie directly in with the outcomes. I’m not a huge fan of generational generalizations, but they say that Millennials are goal-oriented, and a lot of us are working with Millennials now.
- Learner not identified—teacher focusNot assessablePoor verbNo verbJargon, too specificHUGE in scopeTAKE A BREAK
- (End at 3:15ish)
- Two part question—have you done both of these things together? There are other kinds of assessments—observation, initial questions—but today we’re going to knit these two things together.
- Just read through, no elaborationWe’re going to talk about each and which might be best for you, given your setting.
- Polls are good for feedback throughout the session. They’re great if you have information that builds and is cumulative. So, you can force them to answer a few questions before you move on to the next topic. They’re also a fun active learning exercise and they give you immediate feedback.The questions have to be simpler, with no short answer or essay, but you can have more of them. You can set them up to be embedded on a site/LibGuide (if your students have computers), or use free tools to set them up as text messages. If you have a clicker system, you can use them as well. Not too expensive, but they do cost $$$. (1000 to 2000... Other?) You can take those polls and embed them into PowerPoints, and then the answers will show up there (PollEverywhere). They can also just raise their hands, of course!You can have students work together to submit a group answer, or have them work individually. Drawback—can be distracting, can have technical problems, can take up a lot of class time.
- Regardless of what method you choose to deliver the content (phone, embedded poll, clicker) you can have different question types. Multiple choice, yes/no.Substantive or friendly—can also help you to set the stage for what you’ll cover in the beginning. These can be great, because they are anonymous in the moment. So, if someone feels shy about raising a hand to say s/he is confused, this can help them to do it without drawing attention to themselves.
- So, we’ve all had quizzes—they are my weapon on choice. You can set them to happen at the end, or at the beginning and end to gauge what they knew versus what they know. If you want to measure long-term retention, you can give them the same quiz at a later date. If you are working with an instructor at the grade school or college level and you want to measure long term comprehension and retention, see if you can form relationships with any of them to get some extra time. With both quizzes and polls, you can easily convert your learning outcomes to question, if they’re carefully drafted. So, “You will know how to find a book on your topic” becomes “find a book on your topic.”Not more than five “meat” questions, maybe a couple of close out questions that are a bit more free-form.
- Some of these are open ended, some multiple choice. It’s a good idea to have a variety. Not too many easy ones, or you may not learn anything useful. I don’t include any “gimme” questions—things that are super duper easy. Make them think a little and engage their brains. You can also format these as quick assignments. Great for classes with one set task—evaluate and cite websites, for example. You can also give them an article citation and ask them to find it. Then describe briefly why they think it’s reputable. I love this question, but if I do it I keep the others fairly simple. More than one multi-step question can lead to dramatically decreased completion rates.
- You don’t need to be perfect your first time. If your questions aren’t perfect the first time or your instruction isnt perfect, that’s okay You will realize even as you get more accomplished, that you sometimes don’t say things perfectly! You can correct that on the spot. You can use answers as an opportunity to congratulate them for their awesomeness.
- What is a minute paper? You have them write or type for just one minute—only two real questions (and maybe some basic information).On paper or online is your choice. It’s easier to tabulate things online though. Drawbacks:It’s hard to evaluate these in the moment, but you can. It’s also very hard to tie these in with your objectives—however, they are useful. I like to do a combination of hand-raising polling, with a post quiz that has a minute paper attached.
- Gives them space to clarify and explain, if they want to. You can also invite them to leave contact info. You’ll see these final questions again in the minute paper, when we talk about that.
- With polls and quizzes, if they’re brief, it’s easy to tabulate and review responses immediately. I like to do this. Skim over the answers and address problems right there. “It seems like I was unclear on Point X. Can someone describe the answer? If they get things wrong, take the blame. Say—I didn’t explain this clearly enough, because a lot of you are confused. I’m going to run through it again quickly now to make sure you get it, because it’s very important. Use your drama words, but no threats. This way they feel comfortable asking you questions later, if they need to, or expressing questions right then in class. They don’t feel like you’re chastising them for an incorrect answer.HAVE YOUR ELEVATOR SPEECHES READY.This is the most important step. If you don’t integrate the information, it’s not useful to anyone. We’re not perfect—this kind of thing can be jarring at first. Change can relate to the way the questions are worded, ordered, or what is included, and also in the ways you teach.
- Warn the students that it’s coming—but you don’t have to treat it like a test. Just say—”I’m trying to make sure that I’m explaining everything you need, so if you felt something was unclear, please do say so--please help out people who come after you!”Do it ten minutes prior to the end, if at all possible. That way they have five minutes to take it, and you have five minutes to do a final recap and address anything that looks problematic. Have your elevator speeches ready on each of the outcomes—be able to talk about each one very quickly and clearly. It’s important that they know WHY this is important. You will use the information to make sure they know everything they need to know. You will use it to make your instruction more effective for future students.
- Most of these can be used for both. Which are embeddable in ppt or other?
- Basic: free10 questions per survey100 responses per surveySelect: $17 per month ($204 per year)Unlimited questionsUnlimited responsesCustom urlsSkip logic etc.
- Free account: No limits on respondents or number of surveysPRO account:$200 per yearNo PollDaddy branding
- Good for SMSFree version40 responses per pollPersonal$15 per month ($180 per year)50 responses per pollPresenter model $65 per month ($780 per year)250 responses per pollThey also have higher education plans and k-12 plans that are less, so those are worth taking a look at.
- Free, but somewhat clunky to use.Formerly Google Docs—some of their documentation still uses that older term.
- List of verbs:http://www.acu.edu/academics/adamscenter/course_design/syllabus/verbs.html