2018 IEEE Harlan Mills Award Keynote at the Automated Software Engineering Conference that discusses reducing the accidental complexity in software development so that the tools software professionals use amplify human intelligence.

1. The Need for Context
in So0ware Engineering
Gail C. Murphy
University of BriCsh Columbia
except for images where noted
@gail_murphy

2. I want to ignite your imaginaCon…
2
Image a)ribu.on: commons.wikimedia.org/wiki/File:Toy_balloons_red_and_blue.jpg

3. Knowledge Navigator Video, Apple 1987
Apple’s Knowledge Navigator
https://www.youtube.com/watch?v=QRH8eimU_20

4. Thanks to Jaxsun McCarthy Huggan, Tasktop, 2018
So0ware development today

5. Thanks to Jaxsun McCarthy Huggan, Tasktop, 2018
So0ware development today

6. Time [minutes]
Subject
0 30 60 90 120 150 180
T3
T2
T1
S4
S3
S2
S1
R4
R3
R2
R1
●
●
●
●
●
●
●●
●●●●
● ● ● ●
●
● ●●●●●
●
●
●●●
●●●
●●
●
51 activity switches
10 task switches, 3 distinct tasks
●
●
●●
●
●●●●
●
●●
●
●
●●
●●
●
●●
●●●
●●●●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
● ● ●
● ● ●●
●●
●
●
●
●
●●
●
●
●
●
●
●●●
●
● ●
●
● ●
●
●●
●
●
●●
●
166 activity switches
36 task switches, 3 distinct tasks
●
●●●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
●
●
●
●
●●●
●
●●
●
●
●
● ●
●
●
●
●●●●
●
●
●●●
●
●
●●
●
●
●●●
●●●
●●●
●●●●●
●
●
●●●●●●●●
●
●●●
●
● ●● ●●
●
●●
● ●●
●
●
●
●
●
●
●
●●
●
●●●●
●●●
●
●●
●
●●●●●●
●
●●
● ●●●
●
●
● 230 activity switches
79 task switches, 4 distinct tasks
●●
●● ●
●●
●
●
●●●
●
●●
●●●●●
●●
●●
●
●●●●●
●● ●
●●●●
●
●
●
● ●●
●●
● ●●
● ● ●
85 activity switches
13 task switches, 4 distinct tasks
● ●●
●●
●●
●
●
●
●●●
●●
●●●● ●
●
●
●●●
●
●
●
●
●
●
●
● ●● ● ●
●
●
●
●● ●
●
59 activity switches
20 task switches, 5 distinct tasks
● ●
●●●
●
●
●●●●
●●●
● ● ●
● ●
● ●
●●●
●●● ●
●
●
● ●
88 activity switches
17 task switches, 5 distinct tasks
●
●
●●●
●●●●●●●
●●
●●
●
●●●●
●
●
●
●
●●●●●
●
●
●
●●
●
●
●
●
●●
●●
●●●●●●●
●●
●
●●
●
●
●●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●●
●●
●●●
●●
●
●
●
●
● ●●●
●●
●
●
●
●●●
●
●
148 activity switches
27 task switches, 4 distinct tasks
●
●●● ●● ●●
●
●
●
●
●● ●●●●●
●●● ●●
●
●●●● ●●●
●
●
●
●●● ●
●●
●● ●
108 activity switches
16 task switches, 5 distinct tasks
●
●
●
●
●
●
●●
●
●●
●
●
●● ● ●
●
● ● ●
● ●
●
●
● ●
●
66 activity switches
25 task switches
4 distinct tasks
●
●
●
● ●
●●●
●
●
●
●
●
●●●
● ●
● ●
●
●
●
●
● ●●
●
● ●
●
● ●●
●●
●
●
102 activity switches
61 task switches
6 distinct tasks
●
●
●
●●●
●
●●●●●●●●●●
●
●●
●
● ●
●
●
● ●
●
●●●●●●●
●
● ●●●●●
●●●● ● ●
●
●●● ●
●
●●
●●
●●
96 activity switches
28 task switches, 4 distinct tasks
●
●
●
●
●
●
Dev:VC
Dev:Debug
Dev:Code
Dev:Review
Dev:TestApp
Dev:Other
BrowsingRel
BrowsingUnrel
MeetInformal
MeetPlanned
Email
Planning
ReadWriteDoc
Other
What do
developers do?
[Meyer, Fritz, Murphy,
Zimmermann, 2014]
Task Switches:
13.3 (± 8.5) .mes per hour
Ac.vity Switches:
47 (± 19.8) .mes per hour
[“SoZware Developers’ Percep.ons of Produc.vity”, Meyer , Fritz, Murphy and Zimmermann, FSE 2014]

7. SoZware development
tools are not amplifying
human intelligence
Study, definiCon and use
of context can improve
the flow of so0ware
development work

8. 01
10
11
MISSING
EXISTS
NEED

9. 01
10
11
MISSING
EXISTS
NEED

10. What’s
MISSING
in
So0ware
Engineering?
01

11. 10
Knowledge
Navigator Demo
View the demo as a set
of tools connected by
context

12. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
(interrupts by touching screen)
surprise birthday party next Sunday
P:
Image recogni.on: www.goodfreephotos.com

13. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
(interrupts by touching screen)
surprise birthday party next Sunday
P:
Image recogni.on: www.goodfreephotos.com

14. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
(interrupts by touching screen)
surprise birthday party next Sunday
P:
Image recogni.on: www.goodfreephotos.com

15. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
(interrupts by touching screen)
surprise birthday party next Sunday
P:
Image recogni.on: www.goodfreephotos.com

16. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
Tools
(interrupts by touching screen)
surprise birthday party next Sunday
P:
Image recogni.on: www.goodfreephotos.com

17. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
Tools
Context
(interrupts by touching screen)
surprise birthday party next Sunday
P:
Image recogni.on: www.goodfreephotos.com

18. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
Person
Event
Tools
Context
(interrupts by touching screen)
surprise birthday party next Sunday
P:
Image recogni.on: www.goodfreephotos.com

19. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
Person
Event
Role
Tools
Context
(interrupts by touching screen)
surprise birthday party next Sunday
P:
Image recogni.on: www.goodfreephotos.com

20. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
Person
Event
Role
Tools
Context
(interrupts by touching screen)
surprise birthday party next Sunday
P:
Image recogni.on: www.goodfreephotos.com

21. P = Professor
C = Computer
Today you have a faculty luncheon at
12:00; you need to take Kathy to the
airport by 2; you have a lecture at
4:15 on deforesta.on in the Amazon
rain forest
C:
Person
Event
Role
Tools
Context
right, let me see the lecture notes
from last semester
P:
Image recogni.on: www.goodfreephotos.com

22. P = Professor
C = Computer
Today you have a faculty luncheon at
12:00; you need to take Kathy to the
airport by 2; you have a lecture at
4:15 on deforesta.on in the Amazon
rain forest
C:
Person
Event
Role
Tools
Context
right, let me see the lecture notes
from last semester
P:
Image recogni.on: www.goodfreephotos.com

23. P = Professor
C = Computer
Today you have a faculty luncheon at
12:00; you need to take Kathy to the
airport by 2; you have a lecture at
4:15 on deforesta.on in the Amazon
rain forest
C:
Person
Event
Role
Tools
Context
right, let me see the lecture notes
from last semester
P:
Course
Lecture
Image recogni.on: www.goodfreephotos.com

24. P = Professor
C = Computer
(shows notes)C:
Person
Event
Role
Tools
Context
No that’s not enough. I need to
review more recent literature. Pull up
the new ar.cles I haven’t read yet.
P:
Course
Lecture
Journal ar.cles only?C:
Hmmmm. Fine.P:
Image recogni.on: www.goodfreephotos.com

25. P = Professor
C = Computer
(shows notes)C:
Person
Event
Role
Tools
Context
No that’s not enough. I need to
review more recent literature. Pull up
the new ar.cles I haven’t read yet.
P:
Course
Lecture
Journal ar.cles only?C:
Hmmmm. Fine.P:
Image recogni.on: www.goodfreephotos.com

26. P = Professor
C = Computer
(shows notes)C:
Person
Event
Role
Tools
Context
No that’s not enough. I need to
review more recent literature. Pull up
the new ar.cles I haven’t read yet.
P:
Course
Lecture
Journal ar.cles only?C:
Hmmmm. Fine.P:
Paper
Image recogni.on: www.goodfreephotos.com

27. P = Professor
C = Computer
Your friend Jill Gilbert has published
an ar.cle about deforesta.on in the
Amazon and its effects on rainfall in
the sub-Saharan. It also covers the
drought’s effect on food produc.on
in Africa and increasing imports of
foods.
C:
Person
Event
Role
Tools
Context
Course
Lecture
Contact Jill.P:
Paper
Image recogni.on: www.goodfreephotos.com

28. P = Professor
C = Computer
Your friend Jill Gilbert has published
an ar.cle about deforesta.on in the
Amazon and its effects on rainfall in
the sub-Saharan. It also covers the
drought’s effect on food produc.on
in Africa and increasing imports of
foods.
C:
Person
Event
Role
Tools
Context
Course
Lecture
Contact Jill.P:
Paper
Image recogni.on: www.goodfreephotos.com

29. “Context is any informa.on that can be used
to characterize the situa.on of an en.ty.”
— Dey and Abowd, CHI 2000 Workshop
[“Towards a be)er understanding of context and context-awareness”, Dey and Abowd, CHI Workshop on The What, Who, Where, When dn How of Context-Awareness 2000]

30. For this talk, think about context as a
seman.c plane that tools (components,
widgets) can consult when ac.ng

31. 17
Knowledge
Navigator Demo
What is the effect of
context?

32. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
(interrupts by touching screen)
surprise birthday party next Sunday
P:
What is the professor
focusing on?

33. P = Professor
C = Computer
You have three messages:
your graduate research team in Guatemala
just checking in
Robert Jordan, a second semester junior
reques.ng a second extension on his term
paper
and your mother reminding you about your
father’s (cut off)
C:
(interrupts by touching screen)
surprise birthday party next Sunday
P:
What is the professor
focusing on?
the events
(not accessing them)

34. P = Professor
C = Computer
Today you have a faculty luncheon at
12:00; you need to take Kathy to the
airport by 2; you have a lecture at
4:15 on deforesta.on in the Amazon
rain forest
C:
right, let me see the lecture notes
from last semester
P:
What is the professor
focusing on?

35. P = Professor
C = Computer
Today you have a faculty luncheon at
12:00; you need to take Kathy to the
airport by 2; you have a lecture at
4:15 on deforesta.on in the Amazon
rain forest
C:
right, let me see the lecture notes
from last semester
P:
What is the professor
focusing on?
issue of current interest
(not on how to access history)

36. P = Professor
C = Computer
(shows notes)C:
No that’s not enough. I need to
review more recent literature. Pull up
the new ar.cles I haven’t read yet.
P:
Journal ar.cles only?C:
Hmmmm. Fine.P:
What is the professor
focusing on?

37. P = Professor
C = Computer
(shows notes)C:
No that’s not enough. I need to
review more recent literature. Pull up
the new ar.cles I haven’t read yet.
P:
Journal ar.cles only?C:
Hmmmm. Fine.P:
What is the professor
focusing on?
the ques.on of interest
(not on how to answer it)

38. P = Professor
C = Computer
Your friend Jill Gilbert has published
an ar.cle about deforesta.on in the
Amazon and its effects on rainfall in
the sub-Saharan. It also covers the
drought’s effect on food produc.on
in Africa and increasing imports of
foods.
C:
Contact Jill.P:
What is the professor
focusing on?

39. P = Professor
C = Computer
Your friend Jill Gilbert has published
an ar.cle about deforesta.on in the
Amazon and its effects on rainfall in
the sub-Saharan. It also covers the
drought’s effect on food produc.on
in Africa and increasing imports of
foods.
C:
Contact Jill.P:
What is the professor
focusing on?
the topic
(not recall, re-reading or lookup)

40. Context can support the flow of work…
by reducing the accidental complexity
and enabling a focus on the essen/al complexity
of the problem at hand

41. Where is the
context
in SoZware Engineering?

42. 01
10
11
MISSING
EXISTS
NEED

43. 01
10
11
MISSING
EXISTS
NEED

44. 10
What
context
EXISTS
in SE?

45. Focus
on tools
where there is human interacCon

46. Draw
largely on examples
from my research group

47. Kinds of Context
StaCc
ArCfacts
History
Dynamic
ExecuCon
Individual
AcCvity
Team
AcCvity
representa.ve
no claims of completeness

48. Context as
StaCc ArCfacts
Development Environment
30
File
Project
Class
Development
Environment
Context
Fragment

49. Context as
StaCc ArCfacts
Development Environment
30
File
Project
Class
Development
Environment
Find Similar
Classes
Context
Fragment

50. Context as History
Hipikat [Čubraniç & Murphy, 2003]
Development
Environment
How have similar
bugs been solved?

51. Context as History
Hipikat [Čubraniç & Murphy, 2003]
Development
Environment
How have similar
bugs been solved?
Commit
Issue
Versioned
File
Context
Fragment

52. Context as History
Hipikat [Čubraniç & Murphy, 2003]
Development
Environment
How have similar
bugs been solved?
Commit
Issue
Versioned
File
Context
Fragment
[“Hipikat: Recommending Per.nent SoZware Ar.facts”, Čubraniç and Murphy, ICSE 2003]

53. 32
Context as
Dynamic ExecuCon
Development
Environment
Run.me
Object
Code
Context
Fragment

54. 32
Context as
Dynamic ExecuCon
Development
Environment
Why didn’t
Pacman move?
Run.me
Object
Code
Context
Fragment
WhyLine [Ko & Myers, 2004]

55. Kinds of Context
StaCc
ArCfacts
History
Dynamic
ExecuCon
Individual
AcCvity
Team
AcCvity
representa.ve
no claims of completeness

56. Context as
Individual AcCvity
Mylyn [Kersten & Murphy 2006]
File
Task
Development
Environment
Context
Fragment
Class
content assist based on interest
derived from ac.vity
[“Using Task Context to Improve Programmer Produc.vity, Kersten and Murphy, FSE 2006]

57. Context as
Individual AcCvity
Mylyn [Kersten & Murphy 2006]
File
Task
Development
Environment
What files was I
ediCng for Task
A?
Context
Fragment
Class
content assist based on interest
derived from ac.vity
[“Using Task Context to Improve Programmer Produc.vity, Kersten and Murphy, FSE 2006]

58. 35Context as
Team AcCvity
Tasktop
Tool
Feature
Value Stream
Dashboard
Context
Fragment
Story
© Tasktop Technologies, Inc. 2017-2018. All rights reserved.
Task

59. 35Context as
Team AcCvity
Tasktop
Tool
Feature
Value Stream
Dashboard
What is the
flow Cme for a
feature?
Context
Fragment
Story
© Tasktop Technologies, Inc. 2017-2018. All rights reserved.
Task

60. Kinds of Context
StaCc
ArCfacts
History
Dynamic
ExecuCon
Individual
AcCvity
Team
AcCvity
representa.ve
no claims of completeness

61. What is needed to effec.vely use
context
in SoZware Engineering?

62. 01
10
11
MISSING
EXISTS
NEED

63. 01
10
11
Missing
Have
Need

64. 11
What do we
NEED
in
SE to make
context
effecCve?

65. The fundamental
need is to augment
human intelligence
not…

66. The fundamental
need is to augment
human intelligence
not…

67. Four Steps
Recognize Explain Experiment Build

68. Context exists
in other
communiCes…
Recognize
Icons made by: Tracy Tam, Vectors Market, catkuro, Freepik, Those Icons, and Zlatko Najdenovski under a Crea.ve Commons BY 3.0 license

69. Paper Analysis
ICSE 2018 ASE 2018
"Context"
Context per this talk
0 25 50 75 100
Recognize
Approximate analysis of recent
papers in major SE conferences
Most uses of “context” are in
program analysis sense
Use of “context” as in this talk
are rare
Recognize

70. What if…
45
any paper with a tool
includes a sec.on
explicitly describing
context?
Explain

71. Inputs:
Recent issue history (~>300)
Outputs:
Developer recommenda.ons
Requires:
Ac.ve developers
Provides:
<N/A>
Context
Explain

72. Human: Devy, I’m done
Devy: You have uncommitted changes.
Should I commit them?
Human: Ok
Devy: OK, I’m about to open a pull request,
should I assign Alice?
Human: OK
Explain

73. Explain

74. Are there other kinds of context?
RecognizeExperiment

75. Determining what context makers to a tool…
50RecognizeExperiment

76. 51
What tool/system
architecture will best
accommodate
context?
Build

77. 52
How does
context relate
to ontologies?
Build

78. So there is lots to do….

79. 01 MISSING
10 HAVE
11 NEED

80. 01 MISSING
10 HAVE
11 NEED

81. Thanks
to my fantas.c academic and industrial colleagues, post-doctoral
fellows, graduate and undergraduate students, to Mik Kersten
and Robert Elves (my co-founders at Tasktop Technologies) and to
the fantas.c team at Tasktop from whom I have learned so much.
Icons made by Freepik under a Crea.ve Commons BY 3.0 license

82. Current tools oZen cause soZware
professionals to do as much work for the
tools as the tools do work for them.

83. We are passing too much of the accidental
complexity to the humans decreasing the
cogni.ve power they have for the real problems.

84. We can do more to enable the humans
to focus on the essen/al complexity…
58
Image a)ribu.on: commons.wikimedia.org/wiki/File:Toy_balloons_red_and_blue.jpg

85. One possible
future…
Thanks to Jason Murphy and Kieran Murphy Vancouver 2018
h)ps://www.youtube.com/watch?v=m0aaaAEE6Fc

86. One possible
future…
Thanks to Jason Murphy and Kieran Murphy Vancouver 2018
h)ps://www.youtube.com/watch?v=m0aaaAEE6Fc

87. One possible
future…
What do you
imagine?
Thanks to Jason Murphy and Kieran Murphy Vancouver 2018
h)ps://www.youtube.com/watch?v=m0aaaAEE6Fc

88. SoZware development
tools are not amplifying
human intelligence
Study, definiCon and use
of context can improve
the flow of so0ware
development work
@gail_murphy