1. The Evaluation Test
EDTECH 505
Who likes tests? Right – almost no one. I’ve found that graduate students in particular (even if they are
teachers who gives tests frequently) get quite nervous about actually *taking* tests. It's normal for
anyone to be concerned about not knowing how to answer questions, receiving poor grades, etc. All this
results in a relatively high level of test anxiety (although this has a certain usefulness despite the
discomfort it causes).
For the “evaluation test,” for EDTECH 505, my goal is not to trick you or to see how many minute details
from the text you can recall. I am not particularly concerned, either, with your ability to correctly
identify dozens of definitions of glossary words. The assessment for your reading of the course text, The
ABCs of Evaluation (Boulmetis & Dutwin, 2005), could take many forms – ranging from multiple choice
items, to short-answer questions, to essay-length questions. While each of these types of assessments
has its place in the learning process, I have developed a more compelling way to assess your knowledge.
If you have taken a class with me previously, you will recognize the format. This is what I call a “visual
metaphor” exam. Let me explain.
By “visual metaphor,” I mean that you have to take certain concepts (identified by me) and find, for each
concept, some kind of visual representation of it (in this case, I mean a still picture or even something
you create). To be a true metaphor, the picture is not an exact representation of the topic, but instead a
representation of it that reminds you of the concept. For example, I recently gave a presentation and I
wanted to talk about trustworthiness of the study on a slide. Rather than fill the slide with text (blah!), I
used a picture of a wild bird eating out of a person’s hand (I love Flickr.com!!!)
The pictures, though, are not the only part of the test. There will be some actual words that you have to
write. These words are limited, though. I ask for the limited text for two reasons: 1) I do not want to
read essays and few, if any, want to write them, and 2) (more importantly) a student who writes a
compact, precise statement in her or his own words is much more likely to remember it.
The Nitty Gritty
Your visual exam has 33 precisely slides. These are listed in tabular format (below). Each one is based on
a things you have read or defined over the past few weeks.
On each “text” slide, you will have three areas of text:
1. Across the top will be the slide title (these are provided on the table below).
2. In the middle is your ~ 40 word (36-44 is the acceptable range) statement that explains the
concept.
3. At the bottom (in a slightly smaller font) is the URL that points to the picture you’ve used OR the
words “Created by author” if it is a picture from your own collection.
NO text will appear on the visual slide unless the text is somehow part of the graphic (as it might be if
you draw an evaluation model), and no pictures or distracting graphics should appear on the text slides.

2. Submission Logistics
You can create your visual exam in any way you would like, but I will only evaluate it by using
GoogleDocs. Whatever you create, then, must be something I can see by ONLY using GoogleDocs. If you
simply upload a presentation, which causes me to have to open PowerPoint or some other program,
then you will fail the exam on a technicality (which counts heavily toward any ‘do-over’).
If you are unsure how to use the presentation tool in GoogleDocs, then there’s no better time to learn
it! Actually, I am assuming you already know how to use one presentation software or another, and the
skills readily transfer. If you are still lost, then I encourage you to use either A) the Help documentation,
B) Tutorials on YouTube or on another site, or C) Your peer’s knowledge. Notice that I did not say,
“Contact me,” as I do not give tutorials on productivity software or websites (Why not? Because I figure
that our graduate students, who are studying educational technology, should be able to figure it out on
their own. The encouragement of these kind of skills is part of my ‘hidden curriculum”).
Some people create a presentation directly from within the GoogleDocs’s presentation tool, while
others use PowerPoint or the OpenOffice version of it, and then upload it to GoogleDocs. If you upload,
you have to allow GoogleDocs to convert the presentation into GoogleDoc format. If you choose this
option, be careful. If you make something too fancy on your laptop/desktop, as sometimes there is a
disconnect between advanced features and the web-based version.
After creating the GoogleDoc presentation, you must put it in the EDTECH 505 folder that you created at
the beginning of the course and shared with me. That way, I can simply go to your folder and see the
presentation there. You do NOT need to “publish” the presentation so that it can be viewed by anyone
on the web. Simply put it in the shared folder, and I’ll see it there. There is NOTHING to upload via
Moodle regarding this test.
The presentation will be in the folder that already bears your name, so the file name does not need your
own. Instead, it can simply be “505 Reading Test.”

3. Slide Content Table
Slides do not have to be numbered via text that you insert.
Slide # Slide Title Text Content
1 Title slide Student name, Semester, and Class Section (a picture can be
used here if you’d like – either of yourself or one that is
particularly interesting/attractive ).
2 Reflection Slide You can write enough text to fill this slide if you would like,
but I want you to respond to this prompt,
“If you were to to consider the most important things you've
learned about evaluation over the past few weeks, what
would those lessons be? Why do you find these aspects to
be important? Importantly, how are they related to you job
(or future job) as an educational technologist?”
There is no wrong or right answer here, but you should write
a thoughtful response that has some substance to it. Do not
simply repost a blog entry – though you are free to use
thoughts from your previous entries as inspiration. Single
sentences or incoherent thoughts will have points deducted.
PLEASE do not use a font face any smaller than “12”. Merci.
3 (picture slide) A visual representation of (AVRO) the idea, “Why we should
evaluate”
4 Rationale for Evaluation Your answer to the question “Why we should evaluate.”
5 (picture slide) AVRO the relationship of evaluation to educational
technology
6 Evaluation & EdTech Your explanation of the relationship between evaluation and
educational technology.
7 (picture slide) AVRO the difference between research and evaluation
8 Research & Evaluation Your explanation of the difference between evaluation and
research.
9 (picture slide) AVRO of efficiency, efficacy, and impact (or just one of these
is fine)
10 Efficiency, Efficacy, & Impact Your explanation of the difference between efficiency,
efficacy, and impact.
11 (picture slide) AVRO of the impact of evaluation on decision making
12 Impact of Evaluation Your explanation of the impact evaluation has on decision
making
13 (picture slide) AVRO one evaluation model (here, you should create a visual
representation of the model).
14 (Model #1 name) Your explanation of the purpose of the first selected model
15 (picture slide) AVRO a second evaluation model (here, you should create a

4. visual representation of the model).
16 (Model #2 name) Your explanation of the purpose of the second selected
model
17 (picture slide) AVRO quantitative vs. qualitative data
18 Quantitative vs. Qualitative Your explanation of the difference between qualitative and
quantitative data
19 (picture slide) AVRO nominal, ordinal, interval & ratio data (so, four
pictures).
20 Levels of Data Your explanation of the differences between these types of
data (you are allowed to use up to 36 - 60 words on this slide
only).
21 (picture slide) AVRO four different kinds of data collection instruments
(again, four pictures)
22 Data Instruments Your identification of the four instruments you placed on
slide #21. (this will be very simple).
23 (picture slide) AVRO formative and summative evaluations (two pictures)
24 Formative vs. Summative Your explanation of the difference between formative and
summative evaluation.
25 (picture slide) AVRO population vs. sample
26 Samples and Populations Your explanation of the difference between a population and
a sample.
27 (picture slide) AVRO validity vs. reliability
28 Validity and Reliability Your explanation of the difference between validity and
reliability
29 (picture slide) AVRO independent and dependent variables
30 Independent & Dependent Your explanation of the difference between independent
and dependent variables
31 (picture slide) AVRO criterion-referenced vs. norm-referenced tests
32 Criterion- vs. Norm- Your explanation of the difference between criterion-
referenced referenced and norm-referenced tests
33 Activity Evaluation Answer this question:
Please provide your feedback on this particular kind of
“test.” What is your opinion about the use of visual
metaphor to reinforce your learning?

5. Grading Rubric
CATEGORY OUTSTANDING SATISFACTORY BELOW
(9.3-10) (8 – 9.2) EXPECTATION
(< 8)
Submission of test In GoogleDocs (10) Not in GoogleDocs (0)
Slides in correct order Yes (10) No (0)
Reflection slide Solid writing; great Good writing, sufficient Contains either poor
depth; obvious depth. Does a decent writing, surface-level
connection to EdTech. job at reflection but reflection, lack of
No more than one slide. perhaps lacks a quality connections, incorrect
No “wasted words” (ex., of reflectiveness or formatting, etc.
“I think, I believe, I feel, connection that would
etc.) . Font is be “outstanding.” Font is
appropriate size. appropriate size.
Picture Slides Overall incredible Good pictures or Fuzzy pictures, pictures
pictures or graphics that graphics … the depth of or graphics that lack
make me say, WOW! metaphor may be meaning, overall
The visual metaphor is somehow lacking OR it pictures that are just
interesting. Not seems to be completely silly or random, etc.
necessarily obvious, but unrelated (tantamount
connected. to a “random” picture”)
Text slides The title appears at the Title and photo credit The slide title is missing
top. The 40 word (+/- 4 appear on all slides. The or incorrect; the text
words) explanation is 40 word (+/- 4 words) description of the idea
succinct and correct. explanation is good – or concept is incorrect
The photo credit perhaps something is or missing; the photo
appears at the bottom incorrect or phrased credit it missing on a
oddly. number of slides (-1 for
each slide where the
credit is missing)
Instructor's discretion Up to 5 points overall
55 points total possible

6. Frequent Questions
1. Can I use more than one picture on my picture slide?
a. Yes, but not more than four.
2. I would rather just write an essay. Can I?
a. No.
3. I would rather just take a 100 question multiple choice test. Can I?
a. If you design it, vet it, and provide the answer key, I’ll think about it.
4. Can my pictures simply be silly things that I find wherever and put into the middle of slide. I
will be sure that they are lossy and are of generally poor quality and hard to see.
a. NO!! The picture(s) you select for each idea should fill the slide. They should be of good
quality and should fill the slide itself (you do not need the highest resolution photo to do
this, necessarily). You should try to find pictures that have Creative Commons licensing.
5. If I upload something incorrectly or if you cannot open it, do I automatically get a zero?
a. While I'd be sorely tempted to do so, you will not get a zero. However, you will not be
eligible for full credit once things get corrected. If I find a submission error, I will let you
know, and you'll have 5 minutes to fix. Ok, longer than that.. but not as long as a week.
REFERENCES
Boulmetis, J., & Dutwin, P. (2005). The ABCs of evaluation (2nd Ed.). San Francisco, CA: Jossey-Bass