Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The 'T' in TEL
Software Development for TEL Research
Problems, Pitfalls, and Solutions
Roland Klemke
Technology Enhanced Learning
Technology
Computer Science
Engineering
...
Learning
Learning Science
Pedagogy
...
Enhanced
???
OVERLAP?
Problem Area 1:
The TEL Field Itself
Interdisciplinarity in TEL
http://blog.richardmillwood.net/2013/05/10/learning-theory/
https://voxy.com/blog/2010/12/four-...
Feasibility
Reliability
Interoperability
Scalability
Security
Learning Effects
Knowledge Gain
Learning Time
Mastery
Usabil...
Architectures
Proof of Concept
Standards
Performance
Stress Test
Pre-/Post Tests
Assessments
Observation
Interviews
Usabil...
What to do?
Know your enemy!
● What are the targets of the
research group / project you
are involved with?
● Which methods...
Problem Area 2:
Research Approaches
in TEL
3.
The software
engineering process
follows the
requirements given
by the learning
theory and
intervention design
1.
A sel...
Example for a theory-driven approach:
Gamification of Learning in a MOOC Context
1.
Problem & Goal identification
E.g.: Ad...
Gamification of Learning in a MOOC Context
Resulting Stimulated Planning Intervention
Typical Challenges:
Are you able to ...
2.
Experimental
developments lead
to testable
prototypes
3.
Theory selection
and experimental
design follow
technical
cons...
1.
Analysis of technological affordances
E.g.: Sensor-based Augmented Reality
2.
Hypotheses about learning aspects
E.g. Re...
Expertise development in the
context of the WEKIT project
Technical architecture and
prototype setup
Typical Challenges:
H...
Expertise development in the context of the WEKIT project
Resulting theoretical Framework
Typical Challenges:
How to deriv...
1.
An educational
service needs to be
delivered at a pre-
defined level of
quality
2.
The educational
services is
producti...
1.
Identification of requirements for
educational service
E.g.: Need for a MOOC offer
2.
Technical implementation of servi...
Gamification of Learning in a MOOC Context
Resulting Software Architecture
Typical Challenges:
Is the existing service
acc...
Gamification of Learning in a MOOC Context
Resulting Software Architecture
Typical Challenges:
Can you talk "architecture"...
1.
An educational
service is (partially)
in place
1.
Improvements for
Learners are
expected based on
some sound theory
1.
...
What to do?
Plan! Try! Re-plan!
● Don't expect your research to
follow a straight path - expect
deviations, obstacles, set...
Problem Area 3:
Project Constellations
Isolated PhD project
Advantages
● You lead the whole process
● The only requirements to meet are those
of your PhD researc...
PhD project embedded in research project
Advantages
● The project provides a research context
● The project team can suppo...
PhD project embedded in service project
Advantages
● A service project aims at providing a
stable, reliable, scalable serv...
What to do?
Talk, talk, talk!
● Identify the possible risks in
each of the constellations as
early as possible
● Talk to a...
Problem Area 4:
Software Engineering
in TEL
What is Software Engineering?
Software engineering
provides methods and
process models to tackle
the design and
developmen...
Why Software Engineering in TEL?
Artefacts of digital technology at the core of TEL research
Educational Purpose: key func...
Why can’t we just apply it?
Software development goal: provide a fully fledged system with a complete set
of functionality...
What to do?
Act!
● How complex are the
technological requirements of
your research?
○ Learn programming
(seriously!)
○ Tea...
Discussion
Discussion
TEL Research is complex, but doable!
Software
Engineering
T E L
Research
Approach
Project
Constellations
What?
This is a lecture, why do we
have to do something?
Let's discuss!
Reflect on your own research
situation in TEL alon...
Upcoming SlideShare
Loading in …5
×

The 't' in tel software development for tel research problems, pitfalls, and solutions

At the core of TEL research are artefacts of digital technology, their design, implementation, application, and evaluation. Usually, these artefacts aim to fulfil a specific educational purpose and need to satisfy a number of requirements with respect to functionality, usability, scalability, or interoperability.
Software engineering is the discipline that structures, organises, and documents all aspects of the software development process in manageable steps. It explains all relevant stakeholder roles involved in the process and defines process models to handle the complexity of the software development process.
In research oriented projects, software engineering goals and research goals often collide: Software engineering strives to provide a fully fledged system with a complete set of functionality and a broad coverage of use cases. Research aims for evaluating testable hypotheses based on specific aspects of a system. This leads to the problem that the complexity of the design steps, complexity of the derived/developed solution contradicts easy to measure results. Furthermore, project contexts and research contexts often collide, leading to the question how to develop technology that fulfills development needs and research needs.
The lecture looks at typical situations, which occur in technology-oriented research projects and tries to show approaches to handle the inherent complexity within these.

References
Tchounikine, P.: Computer Science and Educational Software Design. Springer Berlin Heidelberg, Berlin, Heidelberg (2011).
Goodyear, P., Retalis, S.: Technology-enhanced learning Design Patterns and Pattern Languages. Sense Publishers (2010).
Mor, Y., Winters, N.: Design approaches in technology-enhanced learning. Interact. Learn. Environ. 15, 61–75 (2007).
Bjork, S., Holopainen, J.: Patterns in Game Design (Game Development Series). Charles River Media (2004).
Calvo, R.A., Turani, A.: E - learning Frameworks = ( Design Patterns + Software Components ). In (Goodyear & Retalis, 2010).
Wang, F., Hannafin, M.J.: Design-Based Research and Technology-Enhanced Learning Environments. Source Educ. Technol. Res. Dev. 53, 5–23 (2005).
Kirkwood, A., Price, L.: Technology-enhanced learning and teaching in higher education: what is “enhanced” and how do we know? A critical literature review. Learn. Media Technol. 39, 6–36 (2014).
Ross, S.M., Morrison, G.R., Lowther, D.L.: Educational Technology Research Past and Present: Balancing Rigor and Relevance to Impact School Learning. Contemp. Educ. Technol. 1, 17–35 (2010).

  • Be the first to comment

The 't' in tel software development for tel research problems, pitfalls, and solutions

  1. 1. The 'T' in TEL Software Development for TEL Research Problems, Pitfalls, and Solutions Roland Klemke
  2. 2. Technology Enhanced Learning
  3. 3. Technology Computer Science Engineering ... Learning Learning Science Pedagogy ... Enhanced ??? OVERLAP?
  4. 4. Problem Area 1: The TEL Field Itself
  5. 5. Interdisciplinarity in TEL http://blog.richardmillwood.net/2013/05/10/learning-theory/ https://voxy.com/blog/2010/12/four-ways-in-which-technology-enhances-language-learning/
  6. 6. Feasibility Reliability Interoperability Scalability Security Learning Effects Knowledge Gain Learning Time Mastery Usability Acceptance Motivation Satisfaction GOALS
  7. 7. Architectures Proof of Concept Standards Performance Stress Test Pre-/Post Tests Assessments Observation Interviews Usability Study Questionnaires User Tracking Observation METHODS Software Engineering Learning Design ???
  8. 8. What to do? Know your enemy! ● What are the targets of the research group / project you are involved with? ● Which methods do apply? ● Do they match the expectations of your research aim? ● Are you able to perform these methods (e.g. required technical competencies available)?
  9. 9. Problem Area 2: Research Approaches in TEL
  10. 10. 3. The software engineering process follows the requirements given by the learning theory and intervention design 1. A selected Learning Theory requires specific procedures 2. We design interventions based on expected effects according to the theory THEORY
  11. 11. Example for a theory-driven approach: Gamification of Learning in a MOOC Context 1. Problem & Goal identification E.g.: Addressing high number of drop-out 2. Theory Selection E.g. Implementation Intention Theory 3. Intervention Design E.g. Stimulated Planning Game Element 4. Application Case Identification E.g. MOOC on IT Security 5. Implementation E.g. Widget development 6. Application and Data Collection E.g. Stimulated Planning Game Element L E T
  12. 12. Gamification of Learning in a MOOC Context Resulting Stimulated Planning Intervention Typical Challenges: Are you able to design the tools you need? Do you manage the technical requirements involved?
  13. 13. 2. Experimental developments lead to testable prototypes 3. Theory selection and experimental design follow technical constraints and possibilities 1. Emerging technologies offer new affordances. Learning related outcomes are fuzzy INNOVATION
  14. 14. 1. Analysis of technological affordances E.g.: Sensor-based Augmented Reality 2. Hypotheses about learning aspects E.g. Record and re-enact expert performance 3. Prototypical Development E.g. AR prototype with wearables 4. Pilot application and data collection E.g. Acceptance & usability data 5. Refinement towards learning framework E.g. WEKIT Framework 6. Evaluation E.g. Measurement of learning aspects E T L Example for an innovation-driven approach: Expertise development in the context of the WEKIT project
  15. 15. Expertise development in the context of the WEKIT project Technical architecture and prototype setup Typical Challenges: How to apply your research to this complex architecture? How to balance partner interests?
  16. 16. Expertise development in the context of the WEKIT project Resulting theoretical Framework Typical Challenges: How to derive a theoretical framework? How do framework and software prototype match?
  17. 17. 1. An educational service needs to be delivered at a pre- defined level of quality 2. The educational services is productive and needs to be evaluated 3. Research interventions can (potentially) be applied to the existing service SERVICE
  18. 18. 1. Identification of requirements for educational service E.g.: Need for a MOOC offer 2. Technical implementation of service E.g. productive setup of MOOC platform 3. Development of educational offer E.g. Course content development 4. Evaluation of service success E.g. usage data, feedback data 5. Identification of problems and improvement need E.g. Lack of student engagement 6. Research design E.g. Interventions to support sense of community to increase engagement T L E Example for a service-driven approach: Implementation of a MOOC offer
  19. 19. Gamification of Learning in a MOOC Context Resulting Software Architecture Typical Challenges: Is the existing service accessible for development? Are you allowed to develop for it? What support is available?
  20. 20. Gamification of Learning in a MOOC Context Resulting Software Architecture Typical Challenges: Can you talk "architecture"? Can you deliver components?
  21. 21. 1. An educational service is (partially) in place 1. Improvements for Learners are expected based on some sound theory 1. You already have a new technology to apply in mind (or at least your supervisor has) REALITY Typical Challenges: Where to start? How to progress? How to ensure results? Is this even sound science?
  22. 22. What to do? Plan! Try! Re-plan! ● Don't expect your research to follow a straight path - expect deviations, obstacles, setbacks ● Think in iterations! ● Identify your personal gaps and find solutions for these ● If your gaps are on the "T"- side: we'll get back to that later
  23. 23. Problem Area 3: Project Constellations
  24. 24. Isolated PhD project Advantages ● You lead the whole process ● The only requirements to meet are those of your PhD research ● You have the maximum amount of freedom Disadvantages ● You have to do everything yourself ● It may be hard to find participants for your experiments ● It may be hard to get support for aspects you are not the expert in Questions: ● Do you have the technical skills to run the project alone? ● Can you develop the required interventions yourself? ● Do you have access to participants for your experiments? ● What about pre-studies, beta-tests, exchange with colleagues?
  25. 25. PhD project embedded in research project Advantages ● The project provides a research context ● The project team can support you ● Ideally, you can focus on your specific aspects, while other aspects are covered by other participants Disadvantages ● Project requirements and PhD requirements may diverge over time ● Project deadlines and PhD schedule may collide ● Project collaboration may fall behind expectations Questions: ● Who helps you to align research goals and project goals? ● Do you know the research aims of the other participants? Are there conflicts? ● Are you allowed to publish about the results relevant to your research?
  26. 26. PhD project embedded in service project Advantages ● A service project aims at providing a stable, reliable, scalable service ● A service project has real users ● A service project can be expected to be professionally organised and performed Disadvantages ● Decisions in service projects follow project requirements and might ignore research ● Software used in a service project might not be tailorable (e.g. closed source) ● Schedules, deadlines, requirements may collide Questions: ● Is the service project open for research? ● Can you get the data? ● Do you get the technical support you need? ● Can you plan/perform/apply your interventions in the productive service environment?
  27. 27. What to do? Talk, talk, talk! ● Identify the possible risks in each of the constellations as early as possible ● Talk to all parties involved about these risks
  28. 28. Problem Area 4: Software Engineering in TEL
  29. 29. What is Software Engineering? Software engineering provides methods and process models to tackle the design and development process for the creation of software Components ● Problem statement ● Requirements ● Design and Specification ● Implementation ● Test ● Application
  30. 30. Why Software Engineering in TEL? Artefacts of digital technology at the core of TEL research Educational Purpose: key functional requirements Software Aspects: quality, scalability Software Engineering: discipline that structures these aspects
  31. 31. Why can’t we just apply it? Software development goal: provide a fully fledged system with a complete set of functionality and a broad coverage of use cases Research goal: evaluate testable hypotheses based on specific aspects of a system Problem: complexity of the design step, complexity of the derived/developed solution contradicts easy to measure results. Project context and stakeholder context often collide, the question is how to tweak technology also how to make it researchable
  32. 32. What to do? Act! ● How complex are the technological requirements of your research? ○ Learn programming (seriously!) ○ Team up with programmers ○ "Wizard of Oz" approach? ○ Use of ready-made tools?
  33. 33. Discussion
  34. 34. Discussion TEL Research is complex, but doable! Software Engineering T E L Research Approach Project Constellations
  35. 35. What? This is a lecture, why do we have to do something? Let's discuss! Reflect on your own research situation in TEL along the lines of this presentation ● What are your biggest gaps to close? ● Where are your highest risks? Can you become an "E" (if you are "L" or "T")?

×