When a software development company couldn’t understand why attorneys weren’t happier with their new document submission software researchers were called in to figure it out. Using a combination of in-context research and a lab-based Kano Method study we identified key misunderstandings and shortfalls in the offering, ultimately inspiring the development team to call for more exposure to users in future projects.

1. Kelly Moran, Projekt202
@Kel_Moran
Using Research to Identify Features
for Attorney Software
TECHNICALLY
LEGAL

2. OUR STORY TODAY

3. Problem
H.E.R.O.E.S. (Humble
Experience Researchers Or
Experience Strategists)
Adventure (Fieldwork)
Science (Lab Studies)
Resolution

4. TheProblem
A predicted budget shortfall

5. Several Assumptions
The “old way”
of filing court
documents
manually is time
consuming

6. Several Assumptions
Filing
electronically is
easier and faster

7. Several Assumptions
Legal personnel
are primarily
unhappy with the
product because
they lack technology
experience

8. Okay… and
MAYBE our
interface could
use a little TLC

9. TheH.E.R.O.E.S.
(Humble Experience Researchers Or Experience Strategists)

10. KELLY MORAN
PRINCIPAL EXPERIENCE RESEARCHER
RYAN BREAULT
UX DESIGNER
JESSIE WEBSTER
RESEARCH INTERN

11. To understand the people we are building experiences for
Whydowedoresearch?

12. Our proven methodology is driven
by a deep understanding of people,
creating a focused path for innovation
and true market leadership.

13. Data
Experience Driven Application Design & Development

14. 75%-95%ofnewproductlaunches
FAIL

15. “We haven’t done the research yet”

16. TheAdventure
Research saves the day (again)

17. PHASE 1: Fieldwork
In-context
observation
complemented by
semi-structured
interviews

18. PHASE 1: Fieldwork
Population:
Small-medium
sized law firms
Attorneys
Paralegals
Legal Secretaries

19. “Never theorize before you have data.
Invariably you end up twisting facts to suit
theories instead of theories to suit facts.”
- Sherlock Holmes
(Arthur Conan Doyle)

20. Obtain INFORMED CONSENT

21. Who did we speak with?

22. Who did we speak with?
KEY FINDING
A mixed data set allowed us insight into multiple areas
of the filing and law practice. This provided us with a
multifaceted perspective on the needs of lawyers and
their staff alike.
Attorney
Paralegal
Legal Secretary
Other

23. We need more focus…
VARIATION IN AT TORN EYS
Observing a broad scope at a shallow depth provided us
with a wide-angle lens of the domain, but prevents us from
extracting the type of focused insights that lead to highly
targeted solutions
Family Law
Transactional
Defense

24. SUMMARY OF FINDINGS

25. UI needed improvements
Some users lack experience
Other users were quite adept
Other assumptions were
unsubstantiated

26. DETAILED FINDINGS

27. Manual filing was not a waste of time as assumed.

28. The Attorney (P2)
“Maybe someday I’ll get to the point where I’m paperless…
That just scares the hell out of me.”

29. “I’m just a tactile person…I can't go paperless.”
The Paralegal (P1)

30. The Legal Secretary
“I try not to keep everything in the paper file” but
“Old school old habit; can’t let go of it”
(P9)

31. Submitting electronically is not faster.

32. REPORTED FLOW:
Paper-based
1. Gather all materials
2. Drive to courthouse
3. Stand in line
4. Wait for clerk to review,
approve, and make a copy
5. Received approved docs
6. Drive back to office

33. ASSUMED FLOW:
eFile
1. Log in
2. Submit documents
online
3. Resume previous work

34. ACTUAL FLOW:
eFile
1. Convert all my paper
files into soft copy
2. Find where “the
scanner put them”
3. Check that it did it right
4. Log in
5. Check off all the right boxes
6. Attach files
7. Submit

35. ACTUAL FLOW:
eFile
::: STILL NOT THE END OF IT :::

36. ACTUAL FLOW:
eFile
STILL HAVE TO:
8. Wait for response
9. Call the court and ask
what went wrong

37. Identifying eFiling Pain Points
LevelofObservedDifficulty
Process in eFiling
Sign
into
eFile
Find
Case
Attach
File
SelectFileType
SelectPaym
ent
Subm
it
Receive
Em
ail
LessDifficultMoreDifficult

38. “Sometimes you’re under pressure
and you’ve gotta file and it’s gotta
happen”
P1
Identifying eFiling Pain Points
LevelofObservedDifficulty
Process in eFiling
Sign
into
eFile
Find
Case
Attach
File
SelectFileType
SelectPaym
ent
Subm
it
Receive
Em
ail
LessDifficultMoreDifficult

39. “Pulldowns and drop downs are not
conditioned enough for the casual filer…
it’s not like chatting with the courts”
P8
Identifying eFiling Pain Points
LevelofObservedDifficulty
Process in eFiling
Sign
into
eFile
Find
Case
Attach
File
SelectFileType
SelectPaym
ent
Subm
it
Receive
Em
ail
LessDifficultMoreDifficult

40. “Just play with it see what it wants”P6
Identifying eFiling Pain Points
LevelofObservedDifficulty
Process in eFiling
Sign
into
eFile
Find
Case
Attach
File
SelectFileType
SelectPaym
ent
Subm
it
Receive
Em
ail
LessDifficultMoreDifficult

41. The loss of human interaction was
seen as a hinderance.

44. Relationships & law
have always been intertwined.

45. Technology & law have not.

46. A GOOD LAWYER
KNOWS THE LAW.
PRO TIP:

47. A GREAT LAWYER
KNOWS THE
JUDGE.
PRO TIP:

48. BRIDGING THE GAP
We need to move attorneys
and their staff away from
the feeling that paper is
the only way to track
and organize their work,
and that face-to-face is the
only way to build connection.

49. BRIDGING THE GAP
eFile and other services
need to be:
Transparent
Easily organized to match
internal processes

50. TheScience
Lab studies - where the truth sets us all free

51. PHASE 2:
The KANO Method
What’s a Kano?

52. The Babysitter
Phenomenon
People quickly
become overwhelmed
and frustrated when
presented with too
many options,
features or tasks.

53. ASSUMPTION:
More features =
better software
Starting with a list of
dozens of possible
features
Some are identified as
necessary for sales
purposes or to hold up
against competitors
This still leaves too many
features to know what to put
the most effort into, or when

54. Developed by Japanese
quality management
consultant and customer
loyalty expert Noriaki Kano,
in the 1970’s and 80’s
The KANO
Method

55. Distinguishes between
essential and
differentiating attributes
The KANO
Method

61. To understand the people we are building experiences for
Justasking“Doyoulikethis?”
isn’tenough
In a Kano Method study each feature uses a pair of questions
to determine the feature category type and its coefficients

62. How the experience would be
with this feature
Functional Dysfunctional
How the experience would be
without this feature

63. Asking functional and dysfunctional questions
allows us to plot features on an evaluation matrix.
Product
Requirement/Feature 1. Like 2. Expect 3. Neutral 4. Tolerate 5. Dislike
1. Like Q A A A O
2. Expect R I I I M
3. Neutral
R I I I M
4. Tolerate R I I I M
5. Dislike R R R R Q
Functional
Dysfunctional

64. Applying KANO
The team identified a
number of solutions

65. Applying KANO
The firm selected
15 features to test

66. Applying KANO
The designer created
mockups

67. Applying KANO
Kano method testing
was blended with
qualitative interviewing
for 3 days of user
testing

68. Session Specifics One-on-one sessions
lasting 40-60 minutes
previewed 15 unique
concepts

69. Session Specifics The concepts were
logically grouped and
ordered, simulating
sequential steps
taken within the eFiling
system

70. Session Specifics
The Kano Method
for Feature
Categorization
provided the bulk of
the session structure,
but significant amounts
of qualitative data
supplement the key
findings

71. To understand the people we are building experiences for
Discussionformat
For this study we arranged questions in groups of three…

72. Ask how the participant feels if a feature
is present

73. Ask how the participant feels if that feature
was not present

74. Ask how the feature would influence the
participant’s decision to purchase the
software

75. How would you feel if your car
got 200 miles to the gallon?
I like It
I expect it
I am neutral
I can tolerate it
I dislike it
FunctionalExample:

76. How would you feel if your car DID
NOT get 200 miles to the gallon?
DysfunctionalExample:
I like It
I expect it
I am neutral
I can tolerate it
I dislike it

77. How does this feature impact your decision to buy this product?
Pay more
I’d pay more for
a service with
this feature
Choose over
another
I’d choose a service
with this feature over
a same-priced service
Rather not
I’d rather use a
service without
this feature
No impact
This feature
doesn’t impact
my decision

78. We created basic mockups to better help participants
visualize the questions and features.
*A Note About Kano Mockups

79. Exact design and functionality was not important here.
The goal was to provide a visual aid to better
communicate proposed features.
*A Note About Kano Mockups

81. Slight bias toward female with all
of the support staff being female,
and a slight majority of attorneys
being male.
Male
KEY FINDING
Demographics
13
Female
7
Male
Gender

82. Age was fairly evenly distributed
between the mid-twenties and
mid-fifties.
KEY FINDING
Demographics
Male
9
35-44
7
25-34
Age
4
45-54

83. There was a strict 50/50 split
between practicing attorneys and
support staff.
KEY FINDING
Demographics
6
Paralegals
10
AttorneysPosition
4
Legal
Secretaries

84. Demographics
The firm sizes were almost equally
distributed between small (solo and
small) and large (medium and
large).
KEY FINDING
Firm Size
8
Solo
3
Small
4
Medium
5
Large

85. [Dashboard] “I love it!
Would it go to my phone?
I rely heavily on my
phone?”
[Status Tracking] “That
was the coolest”

87. Three features performed strongly
any way they were analyzed

88. Multi-Database Search
Category: Strong Attractive
-0.10 0.95
Percentage Willing to Pay More for
Feature
85%
“Top 3” Mentions: 8
“[It] would be worth $100 a month just for the
ability to search cross county.”
- P4
PULL QUOTE

89. Dashboard
Category: Rising Attractive
-0.25 0.90
Percentage Willing to Pay More for
Feature
80%
“Top 3” Mentions: 10
“That’s perfect. I love it.”
- P19
PULL QUOTES
“Would it go to my
phone?…I rely heavily
on my phone.”
- P19

90. Case Indexing
Category: Strong Attractive
-0.10 0.95
Percentage Willing to Pay More for
Feature
90%
“Top 3” Mentions: 7
“The way it is right now is terrible.”
- P20
PULL QUOTES
“That was the
coolest”
- P19

91. “When’s this coming out!?”
- P18

92. Findings &
Recommendations
Results of the Kano
study were shared
in a workshop allowing
stakeholders to combine
user feedback and
preferences with
organizational abilities

93. Findings &
Recommendations
This led naturally to a
detailed product roadmap

94. Feature cards displayed the
Kano data in simple format

95. Cards were mapped by
organizational ability and
then placed in the roadmap

96. PHASE I: January 1, 2016
The six features slated for January 1 of 2016 will create a
product offering that lays the foundation for a full
product suite. Increasing efficiency will be a major benefit
to users in this first release. Transparency and
information management will also be common themes.
05 W E A K LY AT T R AC T I V E
Should be easy to hide/ignore by the experienced user.
Preferably an action is required to see definitions.
UNMET NEEDS: Efficiency, Clarity
Definitions of Filing Codes
01 ST R O N G LY AT T R AC T I V E
UNMET NEED: Conveinence
Multi-Database Search
08 ST R O N G LY AT T R AC T I V E
UNMET NEEDS: Vigilance, Peace of Mind
Set Alerts
10 R I S I N G AT T R AC T I V E
UNMET NEEDS: Financial Records, Efficiency
Monthly Summary
While Kano participants had a difficult time
understanding the value of this feature,
competitors offer it as a "Strategic Intelligence"
tool. Tyler also has this feature already built.
For these reasons it will be included in the Jan
'16 release, although it is highly recommended
that Tyler initiate an effort to better
understand how this feature might be
positioned and utilized.
13 I N D I F F E R E N T
UNMET NEEDS: Competitive Advantage
Market Research
Services will be offered on a subscription basis.
projekt202 recommends a tiered or packaged
approach with a month-to-month payment option.
14 O N E- D I M E N S I O N A L
UNMET NEED: Unlimited Use
Subscription
06 ST R O N G LY AT T R AC T I V E
UNMET NEEDS: Accountability, Comfort,
Transparency
Status Tracking
This first version of the Dashboard will be limited
to eFile data, but will acclimate users to the
concept of using a dashboard.
11.1
R I S I N G AT T R AC T I V E
UNMET NEEDS: Alignment, Information Visibility
Dashboard V1
12 ST R O N G LY AT T R AC T I V E
UNMET NEEDS: Tactile Feel, Personalization,
Organization
Case Indexing
15 ST R O N G LY AT T R AC T I V E
UNMET NEEDS: Research, Time, Convenience
Court Recordings
Team discussions determined that the
"Notifications" feature might best be implemented
as part of the Dashboard, rather than as a
separate, attention demaninding alert appearing
un-prompted on the screen.
07 I N D I F F E R E N T
UNMET NEED: Immediacy
Notifications
PHASE II: Q2
Transparency will be the dominant theme of Q2. Tyler
will also release features that will help alleviate the
strain and anxiety many users feel from the move away
from paper. New organizational systems and document
tracking will simulate the tactile and visible nature of
paper processes.
Should be implemented in a way that does
not hinder the experienced user.
04 B AS I C AT T R A C T I V E
UNMET NEED: Error Prevention
Smart Suggestions
.2
This enhanced Dashboard will include
court-related data such as scheduled hearings.
11 R I S I N G AT T R A C T I V E
UNMET NEEDS: Alignment, Information Visibility
Dashboard V2
There was no clear benefit to the study
participants in paying court fees at the end of
the month. ** However, it is anticipated that
there may be some perceieved value in paying
other costs (such as the cost to set an alert) on a
monthly basis. This could be wrapped into a
subscription fee, or could potentially be a fee for
service. Additional information gathered in late
2015/early 2016 would guide implementation.
09 R E V E R S E **
UNMET NEEDS: Parity with Competition
Consolidated Monthly Billing
PHASE III: Q4
The three features estimated for a 4th quarter release
either require an intense development effort, siginficant
organizational preparation, or further research. From
reaching parity with competition to a phase two, these
features act in a support role for features previously
developed and released by Tyler for Attorney Services.
02 B AS I C AT T R A C T I V E
UNMET NEED: Time
Motion Templates
03 R I S I N G AT T R A C T I V E
UNMET NEED: Time
Step-by-Step Walkthrough
PHASE IV: TBD
Motion Templates and the Step-by-Step walkthrough
must be developed together to best meet the needs of
the user and to most efficiently utilize Tyler's
development capacity. Both features are highly
dependant on external work to research current
market standards. Uncertainties regarding execution
of such a feature push this beyond the forseeable part
of a roadmap.
<< CUSTOMER ACQUISITION TACTICS Analytics and tactics are defined and aligned to a phased launch >>
Q3 2015
Identify and Define Roles of:
Customer Journey Mapping
Determine Story Arc
Message Map Creation
Go to Market Plan
PreLaunch
Q4 2015
D E C I S I O N M A K E R & I N F L U E N C E R
Define the benefit (ie. the problem and the solution) for
each audience
Phase engagement plan for either the eFile tool or outside
the eFile tool
How/when do they realize they need you?
Features and benefits, both short and medium form
Product Roadmap

97. TheResolution
How much did we win? Let us count the ways…

98. New appreciation for end-user input

99. Increased consideration of experience as a design factor

100. The work was presented internally at Tyler in September 2015

101. It was previewed to users
at the 2016 Odyssey User
Conference

102. and…

105. Kelly Moran, Projekt202
@Kel_Moran
slideshare.net/KellyMoran
Using Research to Identify Features
for Attorney Software
TECHNICALLY
LEGAL

106. FRIDAY 9/9
Big Data + Human Data
Aliza Gold
VP of Experience Strategy
& Insights
3 – 4 p.m.
Technically Legal: Using
Research to Identify Features
Kelly Moran
Principal Experience Researcher
FRIDAY 9/9
1– 2 p.m.
SATURDAY 9/10
1– 2 p.m.
SATURDAY 9/10
2– 3 p.m.
UX = ROI: It’s Not Just a Myth
Jeremy Johnson
VP of Customer Experience
The UX of Your Day Job: How to
Remove Organizational Friction
Charlie Trotter
Sr. UX Designer
@