This document summarizes a large-scale research project involving 500 postcards and interviews with 147 customers to understand their experiences and mental models of banking. The research involved branch observations, cultural probes, deeper story interviews, and home interviews. The project generated a large amount of data, including 390 postcards, 23 interviews, and 868 stories total. Analyzing and synthesizing this large volume of data into meaningful insights for stakeholders presented numerous challenges. The document outlines lessons learned around the research method, stakeholder communication, analysis process, and logistics of the large project.
1. Wish you were here!
500 Postcards & we still need the full story
A case study of the trials and tribulations
of a large scale research project
Louise Long (NAB) @longlou
John Murphy (Design4Use) @JohnMurph_
Overall
Gain a deep understanding the Customer eXperience for all aspects of Personal Banking i.e. What are customers current experiences?
What is their ideal or target experience? (or at least what aspects are important to an ideal experience)
Context
To be used throughout the bank as a support to Net Promoter Score (NPS) providing clues as to WHY
To be used by people and teams NOT especially familiar with UX (e.g. 7000 people in building alone)
Lots of research budget and a reasonable time frame
Already done for origination – straightforward insofar as
Well defined stepwise process
Lots of interaction with bank
Problems
Personal banking is PERSONAL (use example of technology now being in our personal lives)
Servicing a personal account is much more about a bunch of ad-hoc interactions
Several different methods
Objective to use different methods to investigate different aspects of servicing
e.g. transactions versus complaints or closing accounts
AND triangulate the different methods
Final goal – to get good context rich stories.
Techniques & Expectation of each
- Branch observations > immediate transactions
- Cultural probe > honest accounts of complaints, disputes and interactions with accounts of context
Interviews in-context (home) > strong understanding of influence of context, less ‘stuffy’ than in bank
Interviews at NAB > complex interaction stories such as variations to home loans
Anticipated Issues
How we get Team of 4 full time + 2 part time across all the data?
How will this work with the analysis?
Several different methods
Objective to use different methods to investigate different aspects of servicing
e.g. transactions versus complaints or closing accounts
AND triangulate the different methods
Final goal – to get good context rich stories.
Techniques & Expectation of each
- Branch observations > immediate transactions
- Cultural probe > honest accounts of complaints, disputes and interactions with accounts of context
Interviews in-context (home) > strong understanding of influence of context, less ‘stuffy’ than in bank
Interviews at NAB > complex interaction stories such as variations to home loans
Anticipated Issues
How we get Team of 4 full time + 2 part time across all the data?
How will this work with the analysis?
Context
2 rounds 6 people
City, suburban
Public area and net to tellers
Goods
Some good stories
Easy to model teller workflow
Bads
Very little context
Most people did not want to talk in detail or at all (i.e. Who the hell are you?)
Analysis – lots of fragments and hard to get stories with context
Context
50 participants
Probe set up as postcards to identify individual data points
(describe idea of postcards versus diary)
Recruited participants (some for specific types of interactions)
Offered both electronic and hard copy
Idea for participant to write and post stamped and addressed card into mail each time they interacted with bank over a 2 week period.
Context
Idea also was to use interaction to prompt customer for another deeper/more significant story that left a good or bad impression
Goods
Lots and lots of stories
Bads
Uncontrolled deeper stories – somewhat hit and miss
Often missing context for deeper understanding
Analysis – lots of work to analyse the hardcopy postcards
Big box almost lost in the post and then arrived v.late holding up schedule – don’t do this around Christmas!!!
Context
23 interviews
Many out of hours
recruited participants (some for specific types of interactions)
Expected to get some stories but not necessarily frank information
Goods
Participants were able to relax and tell good stories aided by a technique of mental maps…
Goods (cont’)
Technique of mental map
Zoom
Maintain context
Make sense in analysis
Simplify complexity
Demo customer understanding and perspective
Bads
Almost none really apart form the late hours!
Context
3 people
City, suburban
in context of personal banking
Goods
Good stories with context of real life
e.g. small travel agent office (split between working and personal banking)
e.g. retired couple with highly detailed paper ledger
Bads
Time and effort to arrange and travel logistics
Some interviews were held at arms length within home context
More data in the postcards than we could use
Refining the data into stories i.e. raw data into mini stories
More data in the postcards than we could use
Refining the data into stories i.e. raw data into mini stories
Role playing and main stories
Persona adoption and sharing sessions
Aretefacts
Mental model diagram
Louise
Rich understanding of customer experiences
Guidelines for what is important to customers in servicing
Practical integration of emotion into banking UX
Link with NPS scoring as a feeling indicating WHAT is wrong
Tool to help explain WHY it is wrong
Louise
½ of room for showing off
½ of room for analysis
Better if could have partition
Louise
Method
Challenge constraints – in-depth interviews can bring out personal stories with care in how they are organised and run
would use this method if only time and budget for one method
Communication with stakeholders
Balance between keeping stakeholders informed and maintaining project time
Analysis
First pass over data to create mini-stories
Important to create and maintain profiles with large images and display publically to maintain empathy
Role play
Logistics
Very important to plan cultural probe well – Christmas episode