ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
LAK13: Epistemology, Pedagogy, Assessment and Learning Analytics
1. Epistemology, Pedagogy, Assessment –
Learning Analytics?
Simon Knight @sjgknight
Image from http://xkcd.com/903/ licensed under a Creative Commons Attribution-NonCommercial 2.5 License.
2. Acknowledgements
Thanks to my co-authors and supervisors Simon
Buckingham Shum and Karen Littleton for their
work on this and our workshop papers
Thanks to Cindy Kerawalla and anonymous
reviewers for their helpful suggestions
Images – mostly from Wellcome Images
http://wellcomeimages.org/ under CC licence
3. LÆAP-ing the divide
1. LA ‘buy in’ to ways of
thinking about
epistemology, assessment
and pedagogy
2. For theoretical, practical,
ethical reasons we should
engage in these debates
3. These considerations have
practical implications – the
middle ground
4. Talk Structure
1. Introduce epistemology
2. Discuss implications for
assessment (& LA) – the
Danish example
3. The other side of the coin –
application & understanding
student’s processes
4. An epistemology for
knowledge building (tools)
6. The Triad
Foreground relationships between:
• epistemology (the nature of knowledge)
• assessment (of learnt knowledge?)
• pedagogy (the nature of learning)
7. Key questions
• What does it mean to
know?
• How do we decide
(assess) if someone
knows or not?
• How do we get people
to come to know (to
learn)?
8. Why does epistemology matter?
“…assessment is one area where notions
of truth, accuracy and fairness have a
very practical purchase in everyday life”
(Williams, 1998, p. 221).
9. Why does epistemology matter?
“…assessment is one area where notions
of truth, accuracy and fairness have a
very practical purchase in everyday life”
(Williams, 1998, p. 221).
• LA ‘buy in’ to particular ways of
thinking about these issues, but they
might be flexible enough to move
beyond the current impasse
10. Learning Analytics
• Data mining
• Digital trace –
including linguistic
data
• Deployed for
pedagogic purposes
12. Danish exams with internet access
http://news.bbc.co.uk/1/hi/education/8341589.stm
Epistemological assumptions
• Allows testing of problem-solving and analysis - sifting
information
• "if you allow communication, discussions, searches and
so on, you eliminate cheating because it's not cheating
any more. That is the way we should think."
A role for LA?
• Potential for auto-grading – LA role, a new model?
http://www.timeshighereducation.co.uk/416090.article
14. Epistemological assumptions
- Knowledge is communicative, and discursive;
language both represents, and creates knowledge
- Separating nuggets of information from their
contextualised uses may not assess knowledge
16. Pedagogic assumptions
• Assessment is used in teaching (but doesn’t drive it)
• Pedagogy should involve knowledge practices – not
assessment practices
• Discourse is fundamental
18. The other side of the coin…
• What we ask students to do
• What they do
19. Epistemology: In action
• “What they do”
• Analytics give unprecedented(?) access
• Marks a shift, from standardised assessments
to knowledge in action
• From seeing in a vacuum, to seeing “for”
20. The Lens of Epistemic Beliefs Behaviours
“epistemological beliefs are a lens for a learner's views on
what is to be learnt” (Bromme, 2009)
• Certainty, simplicity, source, justification – for knowing
• Search allows us to probe; Collaborative Information
Seeking is particularly salient
21. No ‘unveiling’ of beliefs
• “In action” is key; language “to do”
• Psychometrics - many issues of standardised
assessments (& current measures are poor)
• They mask variance, subtleties and contexts
22. Examples of epistemic behaviour
Explicitly or implicitly (behaviour not belief…)
people:
• select sources of information
• have standards by which to judge credibility
• collate information in meaningful ways
• build arguments and make claims
• decide when to start and stop looking for
information
• decide (implicitly or explicitly) the breadth and
depth of information required
23. LA in Structured Knowledge Building
• Strong CSCL tradition to ‘make explicit’ in
structured environments (Knowledge Forum,
Belvedere, etc.)
25. Epistemic Behaviours & Tools
By sharing such maps we:
– Facilitate in-team information management by
making sensemaking explicit
– Can capture information about domain structures
for future users in distributed, asynchronous or
implicit information seeking
– Can feedback to users to support more advanced
epistemic perspectives (e.g. seeking evidence)
26. Next ‘class’ - LÆAP the divide
• LA built on well grounded EAP
• Importance of dialogue around knowledge
building
• The methods to analyse this meaningfully (see
also DCLA paper)
27. Dialogue as a frame
• Moving beyond ‘what do they access/click/add’
• Dialogue creates co-constructed frames for
knowledge building, and problem identification
• Pragmatic, sociocultural context
28. Natural Language Processing based LA
• Discursive context
• Discourse both as a context, and as a creator
of context – historic and fluid properties
• Importance of dialogue in knowledge creation
• Systems provide a target for chat, and a
means to mark sections of chat
29.
30.
31. Discourse Analysis
• Exploratory and accountable talk as social
mode of thinking or interthinking
• …with epistemic implications
• Foreground epistemic language used to
indicate the perspectives taken on content
32. Analytics questions addressed:
• How do users interthink to construct and solve
problems?
• How do users co-conceptualise their
information needs, and how good are they at
finding and mapping information to those
needs?
• How do users use each other, and information,
to build knowledge together
33. LÆAP into the middle space
1. LA implicate epistemologies
(Danish e.g.)
2. Use of data for particular
conversations/assessment/
AfL, is key
3. Design implications –
foreground particular facets
of data & activity
In particular my interest is in understanding how people think about information needs, find information, and how we can support them to do that better. As a corollary to that I’m interested in the sort of epistemological implications of search engines, etc. and how that relates to assessmentLower image from http://commons.wikimedia.org/wiki/File:LensFilter-001.jpg (cc-by)http://commons.wikimedia.org/wiki/File:Dichroic_filters.jpg public domain (by NASA)
Unveilings require statues, unmoving objects, and they require standardisation. But people aren’t like that, they exist in environments, they act, they enact, they wear different clothes in different contexts – whether analytics can address this is, I think, an open question; but it is a epistemological issue which should be considered as such.http://catalogue.wellcomelibrary.org/record=b1213589
Image under a CC licence http://www.flickr.com/photos/carabou/146530613/lightbox/ My thanks to GolnazArastoopur for a useful comment which lead to me writing this blog http://people.kmi.open.ac.uk/knight/2013/03/epistemic-information-architecture-panes-windows-frames-queries-sessions-discourse/