An invited talk I gave at WBVS 2011(The 1st Brazilian Workshop on Software Visualization), in Sao Paulo, Brazil on Sep 27 2011

1. Software Visualization
Promises & Perils
Michele Lanza
REVEAL @ Faculty of Informatics
University of Lugano, Switzerland

2. The use of the crafts of typography,
graphic design, animation, and
cinematography with modern human-
computer interaction and computer
graphics technology to facilitate both the
human understanding and effective use of
computer software.
John Stasko et al.
Software Visualization
Programming as a Multimedia Experience
MIT University Press, 1998

3. algorithm visualization
software visualization
program visualization

4. the visualization of
t h e h i g h e r- l e v e l
algorithm visualization
abstractions which
describe software
software visualization
program visualization

5. the visualization of
t h e h i g h e r- l e v e l
algorithm visualization
abstractions which
describe software
software visualization
the visualization of
the actual program
program visualization code or data
structures in static or
dynamic form

6. program visualization

7. static code/data
visualization
program visualization
dynamic code/data
visualization

16. what is
software?
Soft...

17. Software: The
intangible objects inside
the computer.
where?

18. Programming is a kind of writing.
Gerald M. Weinberg
Psychology of Computer
Programming
Dorset House, 1971

19. programming ~ writing

20. programming ~ writing
software ~ text

21. Deciduous shrub, glabrous or nearly so, with
weak, trailing sub-glaucous, often purple-
tinted stems, either decumbent and forming
low bushes 50 - 100 cm high, or climbing
over other shrubs, rarely more erect
and reaching 2 m. Prickles hooked,
all +- equal. L’flets 2-3 pairs, 1-3.5
cm, ovate or ovate-elliptic, simply,
rarely double serrate, glabrous on
both sides or pubescent on the
veins (rarely all over) beneath,
rather thin; petiole usually
with some stalked glands;
stipules narrow, auricles
straight. Flowers 1-6, white
3-5 cm diam.; pedicels 2-4 cm
with stalked glands, rarely
smooth; buds short... etc. etc.

22. Deciduous shrub, glabrous or nearly so, with
weak, trailing sub-glaucous, often purple-
tinted stems, either decumbent and forming
low bushes 50 - 100 cm high, or climbing
over other shrubs, rarely more erect
and reaching 2 m. Prickles hooked,
all +- equal. L’flets 2-3 pairs, 1-3.5
cm, ovate or ovate-elliptic, simply,
rarely double serrate, glabrous on
both sides or pubescent on the
veins (rarely all over) beneath,
rather thin; petiole usually
with some stalked glands;
stipules narrow, auricles
straight. Flowers 1-6, white
3-5 cm diam.; pedicels 2-4 cm
with stalked glands, rarely
smooth; buds short... etc. etc.

23. It would be a great mistake to suppose
that the dominant role of visual
observation has now been overthrown by
more ‘analytical’ methods.
John Ziman
Reliable Knowledge
An Exploration of the Grounds
for Belief in Science
Cambridge University Press, 1978

24. old habits
die hard

25. /***************************************************************************/ if(p<3&y==E)H=y^16; /* shift capt.sqr. H if e.p.*/
/* micro-Max, */ t=b[H];if(t&k|p<3&!(r&7)!=!t)break; /* capt. own, bad pawn mode */
/* A chess program smaller than 2KB (of non-blank source), by H.G. Muller */ i=99*w[t&7]; /* value of capt. piece t */
/***************************************************************************/ if(i<0||E-S&&b[E]&&y-E<2&E-y<2)m=I; /* K capt. or bad castling */
/* version 3.2 (2000 characters) features: */ if(m>=l)goto C; /* abort on fail high */
/* - recursive negamax search */ if(h=d-(y!=z)) /* remaining depth(-recapt.)*/
/* - quiescence search with recaptures */ {v=p<6?b[x+8]-b[y+8]:0; /* center positional pts. */
/* - recapture extensions */ b[G]=b[H]=b[x]=0;b[y]=u&31; /* do move, strip virgin-bit*/
/* - (internal) iterative deepening */ if(!(G&M)){b[F]=k+6;v+=30;} /* castling: put R & score */
/* - best-move-first 'sorting' */ if(p<3) /* pawns: */
/* - a hash table storing score and best move */ {v-=9*(((x-2)&M||b[x-2]!=u)+ /* structure, undefended */
/* - full FIDE rules (expt minor ptomotion) and move-legality checking */ ((x+2)&M||b[x+2]!=u)-1); /* squares plus bias */
if(y+r+1&S){b[y]|=7;i+=C;} /* promote p to Q, add score*/
#define F(I,S,N) for(I=S;I<N;I++) }
#define W(A) while(A) v=-D(24-k,-l-(l>e),m>q?-m:-q,-e-v-i, /* recursive eval. of reply */
#define K(A,B) *(int*)(T+A+(B&8)+S*(B&7)) J+J(0),Z+J(8)+G-S,F,y,h); /* J,Z: hash keys */
#define J(A) K(y+A,b[y])-K(x+A,u)-K(H+A,t) v-=v>e; /* delayed-gain penalty */
if(z==9) /* called as move-legality */
#define U 16777224 {if(v!=-I&x==K&y==L) /* checker: if move found */
struct _ {int K,V;char X,Y,D;} A[U]; /* hash table, 16M+8 entries*/ {Q=-e-i;O=F;return l;} /* & not in check, signal */
v=m; /* (prevent fail-lows on */
int V=112,M=136,S=128,I=8e4,C=799,Q,N,i; /* V=0x70=rank mask, M=0x88 */ } /* K-capt. replies) */
b[G]=k+38;b[F]=b[y]=0;b[x]=u;b[H]=t; /* undo move,G can be dummy */
char O,K,L, if(Y&8){m=v;Y&=~8;goto A;} /* best=1st done,redo normal*/
w[]={0,1,1,3,-1,3,5,9}, /* relative piece values */ if(v>m){m=v;X=x;Y=y|S&G;} /* update max, mark with S */
o[]={-16,-15,-17,0,1,16,0,1,16,15,17,0,14,18,31,33,0, /* step-vector lists */ } /* if non castling */
7,-1,11,6,8,3,6, /* 1st dir. in o[] per piece*/ t+=p<5; /* fake capt. for nonsliding*/
6,3,5,7,4,5,3,6}, /* initial piece setup */ if(p<3&6*k+(y&V)==S /* pawn on 3rd/6th, or */
b[129], /* board: half of 16x8+dummy*/ ||(u&~24)==36&j==7&& /* virgin K moving sideways,*/
T[1035], /* hash translation table */ G&M&&b[G=(x|7)-(r>>1&7)]&32 /* 1st, virgin R in corner G*/
&&!(b[G^1]|b[G^2]) /* 2 empty sqrs. next to R */
n[]=".?+nkbrq?*?NKBRQ"; /* piece symbols on printout*/ ){F=y;t--;} /* unfake capt., enable e.p.*/
}W(!t); /* if not capt. continue ray*/
D(k,q,l,e,J,Z,E,z,n) /* recursive minimax search, k=moving side, n=depth*/ }}}W((x=x+9&~M)-B); /* next sqr. of board, wrap */
int k,q,l,e,J,Z,E,z,n; /* (q,l)=window, e=current eval. score, E=e.p. sqr.*/ C:if(m>I/4|m<-I/4)d=99; /* mate is indep. of depth */
{ /* e=score, z=prev.dest; J,Z=hashkeys; return score*/ m=m+I?m:-D(24-k,-I,I,0,J,Z,S,S,1)/2; /* best loses K: (stale)mate*/
int j,r,m,v,d,h,i=9,F,G; if(!a->K|(a->X&M)!=M|a->D<=d) /* if new/better type/depth:*/
char t,p,u,x,y,X,Y,H,B; {a->K=Z;a->V=m;a->D=d;A->K=0; /* store in hash,dummy stays*/
struct _*a=A; a->X=X|8*(m>q)|S*(m<l);a->Y=Y; /* empty, type (limit/exact)*/
/* lookup pos. in hash table*/ } /* encoded in X S,8 bits */
j=(k*E^J)&U-9; /* try 8 consec. locations */ /*if(z==8)printf("%2d ply, %9d searched, %6d by (%2x,%2x)n",d-1,N,m,X,Y&0x77);*/
W((h=A[++j].K)&&h-Z&&--i); /* first empty or match */ }
a+=i?j:0; /* dummy A[0] if miss & full*/ if(z&8){K=X;L=Y&~M;}
if(a->K) /* hit: pos. is in hash tab */ return m;
{d=a->D;v=a->V;X=a->X; /* examine stored data */ }
if(d>=n) /* if depth sufficient: */
{if(v>=l|X&S&&v<=q|X&8)return v; /* use if window compatible */ main()
d=n-1; /* or use as iter. start */ {
}X&=~M;Y=a->Y; /* with best-move hint */ int j,k=8,*p,c[9];
Y=d?Y:0; /* don't try best at d=0 */
}else d=X=Y=0; /* start iter., no best yet */ F(i,0,8)
N++; /* node count (for timing) */ {b[i]=(b[i+V]=o[i+24]+40)+8;b[i+16]=18;b[i+96]=9; /* initial board setup*/
W(d++<n|z==8&N<1e7&d<98) /* iterative deepening loop */ F(j,0,8)b[16*j+i+8]=(i-4)*(i-4)+(j-3.5)*(j-3.5); /* center-pts table */
{x=B=X; /* start scan at prev. best */ } /*(in unused half b[])*/
Y|=8&Y>>4; /* request try noncastl. 1st*/ F(i,M,1035)T[i]=random()>>9;
m=d>1?-I:e; /* unconsidered:static eval */
do{u=b[x]; /* scan board looking for */ W(1) /* play loop */
if(u&k) /* own piece (inefficient!)*/ {F(i,0,121)printf(" %c",i&8&&(i+=7)?10:n[b[i]&15]); /* print board */
{r=p=u&7; /* p = piece type (set r>0) */ p=c;W((*p++=getchar())>10); /* read input line */
j=o[p+16]; /* first step vector f.piece*/ N=0;
W(r=p>2&r<0?-r:-o[++j]) /* loop over directions o[] */ if(*c-10){K=c[0]-16*c[1]+C;L=c[2]-16*c[3]+C;}else /* parse entered move */
{A: /* resume normal after best */ D(k,-I,I,Q,1,1,O,8,0); /* or think up one */
y=x;F=G=S; /* (x,y)=move, (F,G)=castl.R*/ F(i,0,U)A[i].K=0; /* clear hash table */
do{H=y+=r; /* y traverses ray */ if(D(k,-I,I,Q,1,1,O,9,2)==I)k^=24; /* check legality & do*/
if(Y&8)H=y=Y&~M; /* sneak in prev. best move */ }
if(y&M)break; /* board edge hit */ }

27. free your mind

28. Software Visualization
The use of the crafts of typography,
graphic design, animation, and
cinematography with modern human-
computer interaction and computer
graphics technology to facilitate both the
human understanding and effective use of
computer software.
Lanza, 2010

29. (software)
visualization myths

30. a picture is worth
a thousand words

31. n g
a picture is worth
ro words
w
a thousand

32. UML is software
visualization

33. g
UML is software
n
r o
w
visualization

34. visual programming is
software visualization

35. n g
visual programming is
r o
w
software visualization

36. static
visualization
software visualization
dynamic
visualization

37. ng
ro ng
w static
visualization
ro
w
software visualization
ng
o
dynamic
wr visualization

39. about software of
software software
visualization

40. about
software
software of
software
information
visualization
visualization

41. William Playfair
1759 - 1823

42. Jacques Bertin
Semiology of Graphics,1967

43. visualization ~ story

47. It was the twenty-eight of November. An
immense confused mass of men, horses,
vehicles besieged the narrow entrances to
the bridges and began to flow over them.

48. It was the twenty-eight of November. An
immense confused mass of men, horses,
vehicles besieged the narrow entrances to
the bridges and began to flow over them.
Those in front, pushed by the weight of those behind were
crushed, trampled on, or forced into the ice-filled water of the
Berezina.
The confusion was so great that when Napoleon himself wished
to cross, it was necessary to use force to clear a passage.
Some there were who, determined to pass at all costs, cut a
horrible way for themselves with their swords. Others opened
an even crueler road for their carriages, driving them pitilessly
through the helpless crowd, crushing men and women, in their
odious greed sacrificing their companions in misery.

49. seeing ~ understanding

50. We acquire more information
through vision than through
all the other senses combined.
Colin Ware
Information Visualization
Perception for Design
Morgan Kaufmann, 2004

51. Sensation Perception
(Physical Process) (Cognitive Process)
Stimulus Sensory Organ Perceptual Organ
Brain

52. The Eye
A camera with a maximum
resolution of 625 dpi in the foveal
zone of the retina.

53. The Eye
A camera with a maximum
resolution of 625 dpi in the foveal
zone of the retina.
The retina is coated with rods (for
dim light) and cones (for bright
light and colors).

54. The Eye
A camera with a maximum
resolution of 625 dpi in the foveal
zone of the retina.
The retina is coated with rods (for
dim light) and cones (for bright
light and colors).
Three types of cones, specialized
on the blue, green, and red range
of the color spectrum.

55. Breaking News
We do not see
with the eyes

56. we see with
the brain

57. iconic
memory

58. short-term
memory

59. long-term
memory

60. Pre-attentive
processing

61. Category Form Color Spatial Motion Motion
Orientation Hue 2D position Flicker
Line length Intensity Direction
Line width
Size
Attribute
Shape
Curvature
Added marks
Enclosure

62. Orientation

64. Line Length

66. Line Width

68. 2D Position / Proximity

70. Enclosure

72. Shape / Similarity

74. Connectedness

76. Categorical vs Quantitative Encoding
of Pre-attentive Attributes

77. Type Attribute Quantitative?
Orientation No
Line Length Yes
Line Width Partially
Size Partially
Form
Shape No
Curvature No
Added Marks No
Enclosure No
Hue No
Color
Intensity Partially
Position 2D Position Yes

78. about software of
software software
visualization

79. Stephen Few Edward Tufte
Show Me the Numbers The Visual Display of Quantitative Information
Analytics Press, 2004 Morgan Kaufmann, 2004

81. about software of
software
visualization software

82. Computer graphics ✔

83. Computer graphics ✔
Gestalt principles ✔

88. simple
..but useless

89. Computer graphics ✔
Gestalt principles ✔
Semantics ✘

90. Interlude
Metrics

91. Interlude II
Software
Analysis

94. WCRE 1999, TSE 2003

97. visualization ~ interaction

98. visualization ~ interaction
interaction ~ tool

99. visualization ~ interaction
interaction ~ tool
tool ~ cool

100. CodeCrawler (1998 - 2005)

101. X-Ray Eclipse Plug-in xray.inf.usi.ch

102. the class blueprint

105. Initialize Interface Internal Accessor Attribute

106. Initialize Interface Internal Accessor Attribute

107. Regular
Initialize Interface Internal Accessor Attribute
Overriding
Extending
Abstract
Constant
Delegating
Setter
Getter
OOPSLA 2001, TSE 2005

108. Regular
Overriding
Extending
Abstract
Constant
Delegating
Setter
Getter

109. Regular
Overriding
Extending
Abstract
Constant
Delegating
Setter
Getter

110. Regular
Overriding
Extending
Abstract
Constant
Delegating
Setter
Getter

111. OK! But, is this useful?

112. the unbearable
lightness of being
visual

113. Solution 1: Anecdotal Evidence

114. Solution 2: Pattern Language

115. Software Systems as Cities

116. City Metaphor
VISSOFT 2007

117. City Metaphor
class building
package district
VISSOFT 2007

118. City Metaphor
class building
package district
VISSOFT 2007

119. City Metaphor
class building
package district
nesting level color
VISSOFT 2007

120. City Metaphor
number of methods (NOM) height
number of attributes (NOA) base size
number of lines of code (LOC) color
class building
package district
nesting level color
VISSOFT 2007

121. Program Comprehension
ArgoUML
LOC 136,325
ICPC 2007

122. Program Comprehension

124. FacadeMDRImpl
NOA 3
skyscraper
NOM 349
LOC 3,413

127. CPPParser
NOA 85
office building
NOM 204
LOC 9,111

130. JavaTokenTypes
NOA 173
parking lot
NOM 0
LOC 0

133. house PropPanelEvent
NOA 2
NOM 3
LOC 37

135. Program Comprehension
ICPC 2007

136. Design Quality Assessment
disharmony map
ArgoUML classes
brain class 8
god class 30
god + brain 6
data class 17
unaffected 1,715
SoftVis 2008

137. System Evolution Analysis
time traveling
WCRE 2008

138. System Evolution Analysis
time
ArgoUML
time traveling 8 major releases
6 years
WCRE 2008

139. System Evolution Analysis
time
ArgoUML
time traveling 8 major releases
6 years
WCRE 2008

140. http://codecity.inf.usi.ch
implemented in Smalltalk
ICSE 2008 tool demo

141. Is it
useful ?

142. A Controlled Experiment

143. Design

144. technical report 2010
State of the art?

145. Design desiderata
1 Avoid comparing using a technique against not using it.
2 Involve participants from the industry.
3 Provide a not-so-short tutorial of the experimental tool to the participants.
4 Avoid, whenever possible, giving the tutorial right before the experiment.
5 Use the tutorial to cover both the research behind the approach and the tool.
6 Find a set of relevant tasks.
7 Choose real object systems that are relevant for the tasks.
8 Include more than one object system in the design.
9 Provide the same data to all participants.
10 Limit the amount of time allowed for solving each task.
11 Provide all the details needed to make the experiment replicable.
12 Report results on individual tasks.
13 Include tasks on which the expected result is not always to the advantage of the
tool being evaluated.
14 Take into account the possible wide range of experience level of the participants.

146. Design desiderata
1 Avoid comparing using a technique against not using it.
2 Involve participants from the industry.
3 Provide a not-so-short tutorial of the experimental tool to the participants.
4 Avoid, whenever possible, giving the tutorial right before the experiment.
5 Use the tutorial to cover both the research behind the approach and the tool.
6 Find a set of relevant tasks.
7 Choose real object systems that are relevant for the tasks.
8 Include more than one object system in the design.
9 Provide the same data to all participants.
10 Limit the amount of time allowed for solving each task.
11 Provide all the details needed to make the experiment replicable.
12 Report results on individual tasks.
13 Include tasks on which the expected result is not always to the advantage of the
tool being evaluated.
14 Take into account the possible wide range of experience level of the participants.

147. Finding a baseline
1. program comprehension
2. design quality assessment
3. system evolution analysis

148. Finding a baseline
1. program comprehension
2. design quality assessment
3. system evolution analysis

149. Finding a baseline
1. program comprehension
2. design quality assessment
3. system evolution analysis

150. Finding a baseline
1. program comprehension
2. design quality assessment

151. Tasks

152. Tasks
program comprehension 6
A1 Identity the convention used in the system to organize unit tests.
A2.1& What is the spread of term T in the name of the classes, their attributes and
A2.2 methods?
A3 Evaluate the change impact of class C, in terms of intensity and dispersion.
A4.1 Find the three classes with the highest number of methods.
Find the three classes with the highest average number of lines of code per
A4.2
method.

153. Tasks
program comprehension 6
A1 Identity the convention used in the system to organize unit tests.
A2.1& What is the spread of term T in the name of the classes, their attributes and
A2.2 methods?
A3 Evaluate the change impact of class C, in terms of intensity and dispersion.
A4.1 Find the three classes with the highest number of methods.
Find the three classes with the highest average number of lines of code per
A4.2
method.
B1.1 Identify the package with the highest percentage of god classes.
B1.2 Identify the god class with the largest number of methods.
Identify the dominant (affecting the highest number of classes) class-level
B2.1
design problem.
B2.2 Write an overview of the class-level design problems in the system.
design quality assessment 4

154. Tasks
program comprehension 6
5
A1 Identity the convention used in the system to organize unit tests.
A2.1& What is the spread of term T in the name of the classes, their attributes and
A2.2 methods?
A3 Evaluate the change impact of class C, in terms of intensity and dispersion.
A4.1 Find the three classes with the highest number of methods.
Find the three classes with the highest average number of lines of code per
A4.2
method.
B1.1 Identify the package with the highest percentage of god classes.
B1.2 Identify the god class with the largest number of methods.
Identify the dominant (affecting the highest number of classes) class-level
B2.1
design problem.
B2.2 Write an overview of the class-level design problems in the system.
design quality assessment 4

155. Tasks
quantitative 9
8
A1 Identity the convention used in the system to organize unit tests.
A2.1& What is the spread of term T in the name of the classes, their attributes and
A2.2 methods?
A3 Evaluate the change impact of class C, in terms of intensity and dispersion.
A4.1 Find the three classes with the highest number of methods.
Find the three classes with the highest average number of lines of code per
A4.2
method.
B1.1 Identify the package with the highest percentage of god classes.
B1.2 Identify the god class with the largest number of methods.
Identify the dominant (affecting the highest number of classes) class-level
B2.1
design problem.
B2.2 Write an overview of the class-level design problems in the system.
qualitative 1

156. Main research questions

157. Main research questions
1
Does the use of CodeCity increase the correctness
of the solutions to program comprehension tasks,
compared to non-visual exploration tools, regardless
of the object system size?

158. Main research questions
1
Does the use of CodeCity increase the correctness
of the solutions to program comprehension tasks,
compared to non-visual exploration tools, regardless
of the object system size?
2
Does the use of CodeCity reduce the time needed to
solve program comprehension tasks, compared to
non-visual exploration tools, regardless of the object
system size?

159. Variables of the experiment

160. Variables of the experiment
correctness
dependent
completion time
CodeCity
tool Eclipse + Excel
independent medium
object system size large
beginner
experience level advanced
controlled academia
background industry

161. Variables of the experiment
correctness
dependent
completion time
CodeCity
tool Eclipse + Excel
independent medium
object system size large
beginner
experience level advanced
controlled academia
background industry

162. Variables of the experiment
correctness
dependent
completion time
CodeCity
tool Eclipse + Excel
independent medium
object system size large
beginner
experience level advanced
controlled academia
background industry

163. Variables of the experiment
correctness
dependent FindBugs
completion time 1,320 classes
93,310 LOC
CodeCity
tool Eclipse + Excel
independent medium
object system size large
beginner
experience level advanced Azureus
controlled academia 4,656 classes
background industry 454,387 LOC

164. Variables of the experiment
correctness
dependent
completion time
CodeCity
tool Eclipse + Excel
independent medium
object system size large
beginner
experience level advanced
controlled academia
background industry

165. Variables of the experiment
correctness
dependent
completion time
CodeCity
tool Eclipse + Excel
independent medium
object system size large
beginner
experience level advanced
controlled academia
background industry

166. The experiment’s design
between-subjects
randomized-block

167. The experiment’s design
between-subjects
randomized-block
CodeCity
T1 large
Tool
T2 medium
Size
Ecl+Excl
T3 large
T4 medium

168. The experiment’s design
background academia industry
experience beginner advanced beginner advanced
B1 B2 B3 B4
between-subjects
randomized-block
CodeCity
T1 large
Tool
T2 medium
Size
Ecl+Excl
T3 large
T4 medium

169. The experiment’s design
background academia industry
experience beginner advanced beginner advanced
B1 B2 B3 B4
between-subjects
randomized-block
CodeCity
T1 large
Tool
T2 medium
Size
Ecl+Excl
T3 large
T4 medium

170. Execution

171. Experimental runs

172. Experimental runs
day 1
time
training session
(1 hour)

173. Experimental runs
day 1
time
training session
(1 hour)
e1
experiment session
(2 hours)
c1

174. Experimental runs
day 1 day 2
time
training session
(1 hour)
e1 e2
experiment session
(2 hours)
c1 c2

175. Experimental runs
day 1 day 2 day 3
time
training session
(1 hour)
e1 e2 e3
experiment session
(2 hours)
c1 c2

176. Experimental runs
day 1 day 2 day 3 day 4
time
training session
(1 hour)
e1 e2 e3
experiment session
(2 hours)
c1 c2 c4

177. Testing the waters
2009
November December
18 24 25 2 9
Lugano
1 3 1
1 1 1 1

178. Timeline of the experiment
2009 2010
November December January February ... April
18 24 25 2 9 21 28 5 8 14 28 18 22 24 25 14
Lugano
1 3 1 1 1 1
1 1 1 1 3
Bologna
2 1 6
1 1 1 1
Antwerp
5
6
Bern
4 1
6

179. Timeline of the experiment
2009 2010
November December January February ... April
18 24 25 2 9 21 28 5 8 14 28 18 22 24 25 14
Lugano
1 3 1 1 1 1
1 1 1 1 3
remote
sessions
Bologna
2 1 6
1 1 1 1
Antwerp
5
6
remote
session
Bern
4 1
6

180. Treatments and subjects
academia industry
beginner advanced advanced
large 2 2 6 10
CodeCity
medium 3 2 7 12
large 2 3 3 8
Ecl+Excl
medium 2 5 4 11
9 12 20 41

181. Collecting raw data

182. Collecting raw data
solution

183. Collecting raw data
completion time

184. Controlling time

185. Controlling time
common time

186. Controlling time
common time
info on subjects
Name (Task): Remaining time

187. Assessing correctness
1 T2: Findbugs, analyzed with CodeCity A3: Impact Analysis B1.2
A1 Multiple locations. The god class containing the largest number of methods in the system is
There are 40/41 [0.5pts] classes class MainFrame [0.8pts]
Dispersed. [1pt] defined in the following 3 packages [1/6pts for each]: defined in package edu.umd.cs.findbugs.gui2 [0.1pts]
which contains 119 [0.1pts] methods.
• edu.umd.cs.findbugs
A2.1
• edu.umd.cs.findbugs.bcel B2.1
Localized [0.5pts]
in package edu.umd.cs.findbugs.detect [0.5pts]. • edu.umd.cs.findbugs.detect The dominant class-level design problem is
DataClass [0.5pts]
A2.2 A4.1 which affects a number of 67 [0.5pts] classes.
Dispersed The 3 classes with the highest number of methods are [ 1 pts each correctly placed and 1 pts each misplaced]:
3 6
in the following (max. 5) packages [0.2pts for each]: 1. class AbstractFrameModelingVisitor
• edu.umd.cs.findbugs defined in package edu.umd.cs.findbugs.ba
contains 195 methods;
• edu.umd.cs.findbugs.anttask
2. class MainFrame
• edu.umd.cs.findbugs.ba defined in package edu.umd.cs.findbugs.gui2
contains 119 methods;
• edu.umd.cs.findbugs.ba.deref
3. class BugInstance
• edu.umd.cs.findbugs.ba.jsr305 defined in package edu.umd.cs.findbugs
• edu.umd.cs.findbugs.ba.npe contains 118 methods
or
• edu.umd.cs.findbugs.ba.vna class TypeFrameModelingVisitor
defined in package edu.umd.cs.findbugs.ba.type
• edu.umd.cs.findbugs.bcel contains 118 methods.
• edu.umd.cs.findbugs.classfile
A4.2
• edu.umd.cs.findbugs.classfile.analysis
The 3 classes with the highest average number of lines of code per method are [ 1 pts each correctly placed and 1 pts each
• edu.umd.cs.findbugs.classfile.engine 3 6
misplaced]:
• edu.umd.cs.findbugs.classfile.impl
1. class DefaultNullnessAnnotations
• edu.umd.cs.findbugs.cloud defined in package edu.umd.cs.findbugs.ba
has an average of 124 lines of code per method;
• edu.umd.cs.findbugs.cloud.db
2. class DBCloud.PopulateBugs
• edu.umd.cs.findbugs.detect defined in package edu.umd.cs.findbugs.cloud.db
has an average of 114.5 lines of code per method;
• edu.umd.cs.findbugs.gui
3. class BytecodeScanner
• edu.umd.cs.findbugs.gui2
defined in package edu.umd.cs.findbugs.ba
• edu.umd.cs.findbugs.jaif has an average of 80.75 lines of code per method.
• edu.umd.cs.findbugs.model B1.1
• edu.umd.cs.findbugs.visitclass
oracles
The package with the highest percentage of god classes in the system is
• edu.umd.cs.findbugs.workflow edu.umd.cs.findbugs.ba.deref [0.8pts]
which contains 1 [0.1pts] god classes
out of a total of 3 [0.1pts] classes.

188. Assessing correctness
blinding 1 T2: Findbugs, analyzed with CodeCity A3: Impact Analysis B1.2
A1 Multiple locations. The god class containing the largest number of methods in the system is
There are 40/41 [0.5pts] classes class MainFrame [0.8pts]
Dispersed. [1pt] defined in the following 3 packages [1/6pts for each]: defined in package edu.umd.cs.findbugs.gui2 [0.1pts]
which contains 119 [0.1pts] methods.
• edu.umd.cs.findbugs
A2.1
• edu.umd.cs.findbugs.bcel B2.1
Localized [0.5pts]
in package edu.umd.cs.findbugs.detect [0.5pts]. • edu.umd.cs.findbugs.detect The dominant class-level design problem is
DataClass [0.5pts]
A2.2 A4.1 which affects a number of 67 [0.5pts] classes.
Dispersed The 3 classes with the highest number of methods are [ 1 pts each correctly placed and 1 pts each misplaced]:
3 6
in the following (max. 5) packages [0.2pts for each]: 1. class AbstractFrameModelingVisitor
• edu.umd.cs.findbugs defined in package edu.umd.cs.findbugs.ba
contains 195 methods;
• edu.umd.cs.findbugs.anttask
2. class MainFrame
• edu.umd.cs.findbugs.ba defined in package edu.umd.cs.findbugs.gui2
contains 119 methods;
• edu.umd.cs.findbugs.ba.deref
3. class BugInstance
• edu.umd.cs.findbugs.ba.jsr305 defined in package edu.umd.cs.findbugs
• edu.umd.cs.findbugs.ba.npe contains 118 methods
or
• edu.umd.cs.findbugs.ba.vna class TypeFrameModelingVisitor
defined in package edu.umd.cs.findbugs.ba.type
• edu.umd.cs.findbugs.bcel contains 118 methods.
• edu.umd.cs.findbugs.classfile
A4.2
• edu.umd.cs.findbugs.classfile.analysis
The 3 classes with the highest average number of lines of code per method are [ 1 pts each correctly placed and 1 pts each
• edu.umd.cs.findbugs.classfile.engine 3 6
misplaced]:
• edu.umd.cs.findbugs.classfile.impl
1. class DefaultNullnessAnnotations
• edu.umd.cs.findbugs.cloud defined in package edu.umd.cs.findbugs.ba
has an average of 124 lines of code per method;
• edu.umd.cs.findbugs.cloud.db
2. class DBCloud.PopulateBugs
• edu.umd.cs.findbugs.detect defined in package edu.umd.cs.findbugs.cloud.db
has an average of 114.5 lines of code per method;
• edu.umd.cs.findbugs.gui
3. class BytecodeScanner
• edu.umd.cs.findbugs.gui2
defined in package edu.umd.cs.findbugs.ba
• edu.umd.cs.findbugs.jaif has an average of 80.75 lines of code per method.
• edu.umd.cs.findbugs.model B1.1
• edu.umd.cs.findbugs.visitclass
oracles
The package with the highest percentage of god classes in the system is
• edu.umd.cs.findbugs.workflow edu.umd.cs.findbugs.ba.deref [0.8pts]
which contains 1 [0.1pts] god classes
out of a total of 3 [0.1pts] classes.

189. Results

190. Statistical test
two-way analysis of variance (ANOVA)
95% confidence interval

191. Correctness
Ecl+Excl
CodeCity
Does the use of CodeCity increase the 8
correctness of the solutions to program
comprehension tasks, compared to non- 7
visual exploration tools, regardless of the 6
object system size?
5
4
3
2
1
0
medium large

192. Correctness
Ecl+Excl
CodeCity
Does the use of CodeCity increase the 8
correctness of the solutions to program
comprehension tasks, compared to non- 7
visual exploration tools, regardless of the 6
object system size?
5
24.26%
4
3
2
more correct with CodeCity
1
large effect size (d=0.89)
0
medium large

193. Correctness
Ecl+Excl
CodeCity
Does the use of CodeCity increase the 8
correctness of the solutions to program
comprehension tasks, compared to non- 7
visual exploration tools, regardless of the
object system size?
6
24.26%
5
4
more correct with CodeCity 3
large effect size (d=0.89)
2
medium large

194. Completion time
Ecl+Excl
CodeCity
Does the use of CodeCity reduce the time 60
needed to solve program comprehension
tasks, compared to non-visual exploration 50
tools, regardless of the object system size?
40
30
20
10
0
medium large

195. Completion time
Ecl+Excl
CodeCity
Does the use of CodeCity reduce the time 60
needed to solve program comprehension
tasks, compared to non-visual exploration 50
tools, regardless of the object system size?
40
12.01%
30
20
less time with CodeCity 10
moderate effect size (d=0.63)
0
medium large

196. Completion time
Ecl+Excl
CodeCity
Does the use of CodeCity reduce the time 60
needed to solve program comprehension
tasks, compared to non-visual exploration
tools, regardless of the object system size?
50
12.01%
40
30
less time with CodeCity
moderate effect size (d=0.63)
20
medium large

197. after the first round
CodeCity
vs
Ecl+Excl
+24% correctness
-12% completion time