Ralf Klamma, RWTH Aachen University ACIS Group BJET-Wiley Seminar Birmingham, March 11, 2014

1. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
1
Learning
Layers
This slide deck is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Do Mechanical Turks Dream
of Big Data?
Advanced Community Information Systems (ACIS)
RWTH Aachen University, Germany
klamma@dbis.rwth-aachen.de

2. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
2
Learning
Layers
Responsive
Open
Community
Information
Systems
Community
Visualization
and
Simulation
Community
Analytics
Community
Support
WebAnalytics
WebEngineering
Advanced Community Information
Systems (ACIS)
Requirements
Engineering

3. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
3
Learning
Layers
Abstract
With the advent of data collections on a planetary level, also the
role of researchers producing, processing and analysing such data
sets is debated as heated as in the early days of nuclear research.
It seems that the Dr. Strangelove image of scientists has turned
into a faceless mass of Mechanical Turks hiding behind agencies
and large research networks. So, it is time to peek behind the
curtain to disclose the network nature of modern science. A basic
ethical obligation is to get enough knowledge to make informed
decisions. So, we visit some recent incidents of big data debates
in higher education and mass surveillance. In particular, we are
questioning the role of computer science as producer of dual use
weapons of mass surveillance. Ironically, computer science is not
only part of the problem but also part of the solution. We discuss
some interesting socio-technical approaches of giving back the
power of data transparently into the hands of the owners.

4. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
4
Learning
Layers
Agenda
TheNetworked
Scientist
LearningAnalytics
LessonsLearnt
Conclusions&
Outlook

5. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
5
Learning
Layers
THE NETWORKED SCIENTIST

6. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
6
Learning
Layers
Iconographic Images of Science

7. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
7
Learning
Layers
Iconographic Images of Science

8. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
8
Learning
Layers
Iconographic Images of Science

9. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
9
Learning
Layers
Computer Science Knowledge Network

10. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
10
Learning
Layers
The Knowledge Map of Computer Science
Map of computer science in 2010
[Pham, Klamma & Jarke, SNAM 2010]
HCI
Networks and Communication
Software Engineering
Artificial Intelligence
Theory
Database
Computer Graphics
Computer Vision
Security and Privacy
Distributed and Parallel Computing
Machine Learning
Data Mining

12. Map of computer science in 1990 Map of computer science in 1995

13. Map of computer science in 2000 Map of computer science in 2005

14. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
14
Learning
Layers
Mechanical Turks

15. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
15
Learning
Layers
Consequences for Scientometrics
■ The great „iron fence“ has been replaced by many fences
around research communities
– Dr. Strangelove is a faceless community now
– The long tail of research communities
– Many research communities under public pressure (e.g.
environmental sciences - http://www.pangaea.de/)
– It will get worse! (open access/data, public funding cuts)
■ Big Data Research for Understanding Science
– Social Network Analysis, Machine Learning
– Mechanical Turks?
■ Where is the research ethics?
– Menlo Report (2012)

16. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
16
Learning
Layers
LEARNING ANALYTICS

17. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
17
Learning
Layers
What is Learning Analytics?
"Field associated with deciphering trends and patterns
from educational big data, or huge sets of student-
related data, to further the advancement of a
personalized, supportive system of higher
education." (2013 Horizon Report)

18. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
18
Learning
Layers
Leaking

19. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
19
Learning
Layers
Recent News about
Lea(rn|k)ing Analytics L
■ BIFIE-Leak - 400.000 confidential tests of pupils and 37.000
E- mail addresses of Austrian teachers have been found on
Romanian servers accessible from the Internet (Die Presse)
■ UMD-Leak - 300.000 personal record data were
compromised by a hack at the University of Maryland (UMD)
■ FSU-Leak: 47.000 teachers in training data leaked at Florida
State University (FSU)
■ Oxford-Leak: University of Oxford Leaks List of Its 50 Worst-
Performing Students (The Chronicle of Higher Education)
This list is really endless

20. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
20
Learning
Layers
TeLLNet - SNA for European
Teachers‘ Life Long Learning
■ How to manage and handle large scale data on
social networks?
■ How to analyse social network data in order to
develop teachers’ competence, e.g. to facilitate
a better project collaboration?
■ How to make the network visualization useful
for teachers’ lifelong learning?
Song, Petrushyna, Cao, Klamma:
Learning Analytics at Large: The Lifelong Learning Network of 160, 000
European Teachers. EC-TEL 2011

21. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
21
Learning
Layers
Analysis and Visualization of
Lifelong Learner Data

22. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
22
Learning
Layers
Ethical Concerns During the Project
■ The eTwinning platform data should be protected as
much as possible
– No live access, access only in anonymous dumps
– Better: Privacy preserving technologies
■ Teacher Workshops
– Identification of teachers only with consent
■ Learning Analytics Tools
– Tool was available on the Web
– Data accessible only for teacher in the networks

23. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
23
Learning
Layers
Still, Technology is a Powerful Tool

24. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
24
Learning
Layers
LESSONS LEARNT

25. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
25
Learning
Layers
Datability
■ A clear identification of benefits, risks and harms for
collecting ICT data
■ Ethical guidelines, approval routines and best
practices for data sharing in science and education
■ Transparency and accountability without the loss of
privacy
■ Academic freedom

26. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
26
Learning
Layers
What we get
■ Will happen J Big Data by Digital Eco Systems (Quantitative Analysis)
– A plethora of targets (Small Birds)
– Professional Communities are distributed in a long tail
– Professional Communities use a digital eco system
– An arsenal of weapons (Big Guns)
– A growing number of community learning analytics methods
– Combined methods from machine intelligence and knowledge representation
■ May not happen L Deep Involvment with community
(Qualitative Analysis)
– Domain knowledge for sense making
– Passion for community and sense of belonging
– Community learns as a whole
→ Community Learning Analytics for the Community by the Community

27. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
27
Learning
Layers
Learning Analytics vs. Community
Learning Analytics
Formal Learning Learning Analytics Community
Regulated
Learning
Community
Learning Analytics
Environment LMS EDM/VA CIS/ROLE DM/VA/SNA/Role
Mining
Tools Fixed LMS Specific Eco-System Tool Recommender
Activities Fixed Content
Recommender
Dynamic Content
Recommender /
Expert
Recommender
Goals Fixed Progress Dynamic Progess / Goal
Mining / Refinement
Communities Fixed Not applicable Dynamic (Overlapping)
Community
Detection
Use Cases Courses Learning Paths Peer Production /
Scaffolding
Semantic Networks
of Learners /
Annotations

28. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
28
Learning
Layers
Reflective Open
Community Information Systems
• Network
Models
• Network
Analysis
• Actor Network
Theory
• Communities
of Practice
• Expert
Identification
• Community
Detection
• Web Mining
• Recommender
Systems
• Multi Agent
Simulation
WebAnalytics
• Advanced
Web &
Multimedia
Technologies
• XMPP
• HTML5
• MPEG-7
• Web
Services
• REST
• LAS
• Cloud
Computing
• Mobile
Computing
WebEngineering
• MediaBase
• MobSOS
• D-VITA
• Requirements Bazaar
• Direwolf
• AERCS/CAMRS
• yFiles
• Repast
• AERCS
• LAS & LAS2peer
• youTell
• SeViAnno 2.0
Responsive
Open
Community
Information
Systems
Community
Visualization &
Simulation
Community
Analytics
Community
Support
Requirements Engineering
• Large-Scale Web-Based Social Requirements Engineering
• Agent and Goal Oriented i* Modeling
• Participatory Community Design