Presentation to our research paper "IS Success Awareness in Community-Oriented Design Science Research" held at DESRIST 2015, May 21, 2015 in Dublin, Ireland.

1. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-DR-1505-1 This work by Dominik Renzel is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
IS Success Awareness in
Community-Oriented Design Science Research
Dominik Renzel, Ralf Klamma, Matthias Jarke
Advanced Community Information Systems (ACIS) Group
Chair of Computer Science 5 (Databases & Information Systems)
RWTH Aachen University, Aachen, Germany
renzel@dbis.rwth-aachen.de
DESRIST 2015
May 21, 2015
Clontarf Castle, Dublin, Ireland

2. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-DR-1505-2
Design Science Research meets
Community of Practice
Design Science Research
(Hevner et al., 2004)
Community-Oriented
Design Science Research
“Learn how to do better“ as innovation driver;
requires self-reflection & success awareness
Design + evaluation of IT artifacts;
emphasizes relevance, innovation & rigor
Community of Practice
(Wenger, 1998)

3. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-DR-1505-3
Long-Tail Communities of Practice
Community of Practice
(Wenger, 1998)
 Joint enterprise, mutual engagement,
shared repertoire, informal structures,
permeable boundaries
 Goal: “learn how to do better“
 Agency with IT artifact ensembles
(cf. Sein et al., 2011)  CIS
Long-Tail
(Anderson, 2004)
 Specialized, dynamic, heterogeneous
 Digital/ecological niche
 Innovation driver!
organized in

4. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-DR-1505-4
CIS Success Awareness in
Community-Oriented DSR
 Central: context-specific CIS success model(s)
 Context: CIS, community, stakeholder, time/version
 CIS success models guide design & practice
 Ongoing practical CIS evaluation drives CIS success modeling
CISSuccess
Model
CISSuccess
Model
CISSuccess
Model

5. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-DR-1505-5
CISSuccess
Criterion Measure
OCS
CIS Success Modeling
1. Think community!
2. Avoid obtrusiveness!
3. Balance parsimony & completeness (Burton-Jones
et al., 2006)
4. Prefer established instruments (Gable et al., 2008)
5. Prefer formal validation
 Foundation: IS-Impact Model (Gable et al., 2006)
 Multi-dimensional formative index construction
(Diamantopoulos & Winklhofer, 2001; Petter et al., 2007)
 A-priori models: community experts & requirements
 Data sources:
– Automatically collected & enriched usage data
– Online micro-surveys (incl. criterion measure)
 Validation: multiple regression & expert panels

6. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-DR-1505-6
Case Study – Design & Evaluation of
Aphasia Patient Chat Tool
 Community
– 200 aphasia patients distributed across Germany
– Inter-disciplinary therapist/researcher team
 CIS (meta-)artifacts:
– SOCRATES (chat tool incl. dictionary-based word completion)
– Design goal: therapy tool to train & lost skills; to mitigate social isolation
– SOCRATES CIS success model
– Evaluation goal: awareness of most relevant SOCRATES success factors
 Data sources:
– Usage data: rich low-level chat protocol logs (XMPP)
– Survey response: tailored to aphasia patients‘ short attention span

7. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-DR-1505-7
Case Study Findings
 Standard chat UIs cause pathological lock-states
– Visible from usage data-based a-priori CIS success metric
– Initiated UI simplification as design improvement
 Immediate word completion causes typing pauses
– Detected drop in usage data-based CIS success metric
– Introduction of 5-second delay as design improvement
 Word completion: training help, not augmentation
– Detected from usage data (action after word completion list)
– Exaptation case; no intervention necessary however
 Usage data-driven metrics allow real-time awareness
 Presence of CIS success model influences design

8. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-DR-1505-8
Influence of Design on
Success Model
 Reduced feature set reduction in candidate CIS success metrics
 Design influences CIS success model (here: simplification)

9. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-DR-1505-9
SOCRATES CIS Success Model –
Final Version Patients
N=|Patients|
N=|Patients|
N=|Patients|
N=6
N=6
N=6
Usage data-based metric Survey data-based metric
 SOCRATES success model constructed from usage & survey
data with strong internal validity

10. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-DR-1505-10
Conclusions & Future Work
 Productive meeting of Community of Practice theory with
Design Science Research
– Communities of Practice offer rich IT artifact innovation contexts
– Communities of Practice lack IT artifact design & evaluation
methods
 Aphasia case study demonstrates CIS success awareness
 Future work: addressing the Long-Tail with IT meta-artifacts
– Requirements Bazaar
– CIS Success Awareness Platform