The good, bad and the ugly. Mobile banking in NZ. Designing for mobility.
•Download as PPTX, PDF•
5 likes•2,464 views
The document summarizes a study evaluating the mobile banking experiences of major New Zealand banks and one large US bank. It assessed the banks based on 5 criteria: platforms supported, features, user ratings, usability, and interaction design. Based on weighted scores across all criteria, ASB and the large US bank JP Morgan Chase had the top mobile banking experiences, while ANZ had the lowest score due to few features, platforms, and poor user ratings. The study found that good mobile usability and features can drive customer switching between banks.
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to The good, bad and the ugly. Mobile banking in NZ. Designing for mobility.
Similar to The good, bad and the ugly. Mobile banking in NZ. Designing for mobility. (20)
More from Optimal Usability
More from Optimal Usability (20)
Recently uploaded
Recently uploaded (20)
The good, bad and the ugly. Mobile banking in NZ. Designing for mobility.
- 1. The good, the bad and the ugly: Mobile Banking in New Zealand Kris Nygren & Michael Dutton Designing for Mobility 2013 Melbourne
- 2. Some of us…20 UX consultants and designers in Auckland and Wellington
- 3. Disclaimer (and credentials!) – we have worked with most New Zealand banks on more than 100 projects
- 4. Mobile and mobile banking were hot trends in NZ in 2012 (penetration:) http://services.google.com/fh/files/blogs/our_mob ile_planet_new_zealand_en.pdf TNS Mobile Life study (New Zealand) 2012
- 5. Google search trends told the same story: Mobile banking (NZ) Banking (NZ) Google Analytics
- 6. By mid-2012 it was like a mad race with New Zealand banks announcing new mobile banking offerings on a weekly basis!
- 7. But lets step back and look at the bigger picture…
- 8. Global mobile stats in 2012… 6 billion mobile connections 1.2 billion mobile web users 1 billion smartphone handsets http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats/a#subscribers http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats/a#smartphonepenetration
- 9. Rank Country Subscribers (millions) 1 China 1,100 2 India 699 3 USA 322 4 ? 260 Where is mobile 5 Brazil 259 growing? 6 Russia 227 7 Japan 128 8 ? 120 9 Germany 114 http://mobithinking.com/mobile- marketing-tools/latest-mobile- stats/a#chinasubs 10 ? 107
- 10. Rank Country Subscribers (millions) 1 China 1,100 2 India 699 3 USA 322 4 Indonesia 260 Where is mobile 5 Brazil 259 growing? 6 Russia 227 7 Japan 128 8 Pakistan 120 9 Germany 114 http://mobithinking.com/mobile- marketing-tools/latest-mobile- stats/a#chinasubs 10 Nigeria 107
- 11. What about growth in +2000% mobile banking? 1.1bn 550m 55m 2009 2013 2015 http://www.berginsi http://www.ababj.com/tech- http://www.prweb.co ght.com/News.aspx? topics-plus/m-banking-530- m/releases/2010/02/pr m_m=6&s_m=1 million-users-by-2013-2695.html web3553494.htm
- 12. China 2012: >1 billion mobile subs http://mobithinking. >400m mobile web m/blog/china-top- mobile-market users
- 13. The global trend is for “un-banked” and “under-banked” consumer segments driving the growth in mobile banking http://mobithinking.com/banking-the-unbanked
- 14. Q: This country has 29 million mobile subscribers. 65% (19 million) use mobile money services. Where am I? A: http://www.cck.go.ke/resc/statcs.html
- 15. Q: This country has 29 million mobile subscribers. 65% (19 million) use mobile money services. Where am I? A: Kenya http://www.cck.go.ke/resc/statcs.html
- 16. And it turns out it’s the same trend in the US as in China, India and Kenya….
- 17. The “unbanked” and “underbanked” are key to growth – adoption among younger, minorities, lower income segments are driving growth in the US
- 18. But the question we really wanted to answer…who provides the BEST mobile banking experience in New Zealand?
- 19. So, UX consultancies regularly study the usability of internet and mobile banking…
- 20. …and there is a plethora of mobile banking “awards”
- 21. …and sometimes its just one guy’s opinion!
- 22. So we asked ourselves - as UX researchers and designers - if we were to evaluate mobile banking offerings as objectively and scientifically as possible, …how would we do it?
- 23. This is how. Evaluate each offering against 5 core criteria:
- 24. We assessed all New Zealand banks (and US giant) and plotted them on a pentagon spider-diagram Usability Interaction Platforms design User Features ratings
- 25. Big graph area = good Usability Interaction Platforms design User Features ratings
- 26. Small graph area = bad Usability Interaction Platforms design User Features ratings
- 27. Before we start, however, it’s important to understand the relative size and resources of the banks we evaluated… TSB Bank ~$200m revenue JP Morgan Chase $100bn+ revenue
- 28. We started with some secondary research – looking at publically available data Platforms User ratings Features
- 29. It turned into this heat-map… iPhone Platform Y Y Y Y Y Y y y iPad N Y N N N N y y Android Y Y Y Y Y N y y Windows Y N N N N N y N Open accounts N N Y N N Y N N No log-in balances Y Y Y N N N N N 4/5 digit log-in Y N Y Y Y N N Y View account balances Y Y Y Y Y Y Y Y View transaction lists Y Y Y Y Y Y Y Y Transfer between accounts Y Y Y Y Y Y Y Y Bill payment Y Y Y N Y Y Y N Set up payees - From Mobile N N N N N Y Y N View automatic payments N Y Y N Y Y N Y Change upcoming payments N N Y N Y y N N View term deposits Y Y Y N Y y N N View mortgages & loans Y Y N Y Y Y Y Y Functionality View KiwiSaver Y N N N N N N N Pay tax N N N N Y Y N N Alerts: Text N N N N N Y Y N Push N N N N N Y Y Y Email N N N N N Y Y N Foreign exchange, rates etc. Y N N N Y Y N N Share trading N N N N N N N y NFC N N N N N y N N Location based services Y N Y N N N Y Y Pay to mobile Y N N Y N Y N Y Pay TradeMe sellers Y N Y N N N N N Pay to Facebook friends Y N N N N N N N Personalise account N N N N N Y N N Personalised service N N N N Y N N N Picture cheque deposits N N N N N N Y N Loyality scheme points Y n N N N N Y N Secure Messaging Y N N N Y Y N N App store - latest version 4.53 3.33 4.55 1.82 4.35 4.45 2.34 2.19 Rating App store - all versions 3.31 1.87 4.16 1.82 4.35 4.45 3.49 3.00 Google play 3.94 2.93 4.50 3.93 4.35 4.24 3.92
- 30. We looked closely at the features provided – both core and non-core Features
- 32. Here is how they ranked on features…tiny TSB at the top!
- 33. Next we looked at the user rating in the app stores (which must be taken with a grain of salt…) User ratings
- 34. And there were some clear winners and losers
- 35. Finally, we looked at how many platforms banks make their mobile offering available on Platforms
- 36. Revealing a wide spread with TSB and ANZ potentially excluding a large part of their customer base
- 37. As UX consultants, we believe that comprehensive usability testing was critical to a robust evaluation Usability
- 38. …so we recruited real bank customers and tested all the mobile banking apps in our specialist mobile user testing lab in Auckland and Wellington
- 40. Big gap between the best and the worst…and as expected, a strong (inverse) relationship between usability issues and SUS (system usability score)
- 41. On balance, Westpac and ASB provided the most ‘usable’ UI
- 42. Finally, we asked five UX consultants and interaction designers to assess how well designed each mobile banking app was Interaction design
- 43. We created a robust evaluation framework, which enabled independent assessment and scoring of each banking app
- 44. And yet again, it revealed a big gap between the top and bottom performers
- 45. We didn’t feel all criteria were equally important, so we weighted them to emphasise the most important aspects
- 46. So, how did they all stack up?
- 47. This should start to give you a picture…
- 48. In the end, ANZ achieved the lowest score due to a lack of features, few platforms and poor user ratings Usability 56.01 Interaction Platforms design User ratings Features
- 49. ASB, by contrast achieved strong scores across all criteria Usability 77.83 Interaction Platforms design User ratings Features
- 50. In fact, giant US Bank JP Morgan Chase (often held up as a benchmark in mobile banking) came in second to ASB Usability 74.86 Interaction Platforms design User ratings Features
- 51. Scores: 74.86 77.83 68.83 67.35
- 52. Finally, some key insights from this study… • Is “feature bloat” a problem in terms of mobile banking UX? Not if the app is well designed. • It’s really hard to measure good design! • Good mobile usability and features can make people switch banks! • Americans still use cheques! • And, these findings are already out of date…
Editor's Notes
- This is some of us…We have 20 guys who do user experience strategy, research and design in Auckland and Wellington. Some of us.We’ve completed more than 1,000 UX projects for 200 clients spanning 20 industries since 2002 [CLICK]
- Banking is our single biggest client segment – so this is both credentials and disclaimer!In fact, we’ve completed more than 80 UX projects between Westpac and ASB (CBA’s Kiwi subsidiary) alone – they are both Top 5 clients for usBut over time we’ve worked with all the Big 4 NZ banks and some of the smaller ones as well
- Mobile banking was the hot trend in NZ in 2012. Here are some numbers:According to Google, smartphone penetration in NZ reached 44%And a global TNS Mobile Life study claimed that 28% of Kiwi’s already use mobile banking
- This is evident in Google search trends as well
- In fact, mid-way through the year it was like a mad race with banks announcing new mobile banking apps and features on a weekly basis!And as user experience consultancy with a big banking practice we needed by make sure we stayed ahead of the pack and had a clear grasp of the latest trends and intel from the world of mobile bankingSo we invested a lot of time looking over the horizon – and we found out some fascinating stuff!
- Lets start with the really big pictureThere are 6 billion mobile subscriptions globally. Growth is in India and China – already 30% of global subscribers. Mobile subscribers outnumber fixed line connections 5:11.2 mobile user access the internet on their phone1 billion people globally have a smartphone
- Lets start with the really big pictureThere are 6 billion mobile subscriptions globally. Growth is in India and China – already 30% of global subscribers. Mobile subscribers outnumber fixed line connections 5:11.2 mobile user access the internet on their phone1 billion people globally have a smartphone
- Ok, time for a quick pop quiz: this is the Top 10 list of countries by number of mobile subscribers in 2012.Any guesses as to who is 4th on the list?
- Nigeria
- So lets get back to the global picture for mobile banking:Bergh Insights estimated 55 million users in 2009Juniper Research forecasts 550 users by 2013Global Industry Analysts predict 1.1bn users by 2015That’s a healthy 2000% growth over 6 years. So who’s driving this growth? [CLICK]
- Its there guys:In 2012, the number of mobile subscribers in China passed 1 billion…and 400m Chinese accessed the internet on their mobile [click]
- And its these guys. This is a picture of an actual “bank branch” in India. The global trend is for “un-banked” and “under-banked” consumer segments driving the growth in mobile banking.Makes a lot of sense, since here are more than twice as many people globally with mobile phones than with bank accounts in the world [CLICK]
- Time for another Pop Quiz:Question [CLICK] AnswerSo what about developed economies? What is a solid source of truth about mobile banking here? [CLICK]
- Time for another Pop Quiz:Question [CLICK] AnswerSo what about developed economies? What is a solid source of truth about mobile banking here? [CLICK]
- How about these guys – the US Federal Reserve! They research and monitor the mobile banking space closely. This is a research report from March 2012.One of their key findings was around who is driving the growth in mobile banking in the US [CLICK]
- Turns out its these guys:And this effectively the same trend as in China, India and Kenya. It’s “unbanked” and “underbanked” driving the growth – adoption among younger, minorities, lower income segments are driving growth in the US. The Fed goes so far believes that mobile is “closing the digital divide” in the US
- One option is to look at what other user experience companies think. We are member of the UX Alliance – who regularly evaluates internet and mobile banking offeringsBut these studies don’t really point to who provids the BEST mobile banking experience
- There are a bunch of organisations and associations handing out mobile banking awards. Keynote is one of the more prominent ones. They announced that Chase Bank was the global leader in mobile banking in 2012But they didn’t really focus on the mobile banking experience as such…
- We were quite excited to this – the 2012 Mobile Banking Awards! – but it seems this is just one guys opinion, based mostly on other people’s options!Worse still – Michael Nuciforo actually worked on the project to deliver the ANZ goMoney app that he has given second place in his own awards.This stuff goes against our religion. As a UX consultancy – everything we do is as scientific as we can make it – based on primary research. Wherever possible we make inferences based on observed behaviours and objective data
- So we asked ourselves – if we were evaluating mobile banking offerings as objectively and scientifically as possible – bearing in mind that our domain is the user experience - how would we do it?KN: At this point I’m going to hand over to my own mad scientist - Michael Dutton. Thanks Michael…MD: Thanks Kris, let me talk you through how we went about doing this – and what we found [CLICK]
- We settled on five evaluation aspects that combined gave us a robust, objective user experience rating:Firstly, what users can actually do on the mobile app [clicks]What platforms can the app be used on – iPhone, Android, iPad and Windows [clicks]We looked at the user ratings for all the apps in the Apple App Store and Google Play markets[clicks]We asked eight (?) UX consultants and designers to individually evaluate each banking app against a set of interaction design and usability criteria[clicks]But most importantly, we user tested all the apps in our testing labs in Auckland and Wellington[clicks]Maria, heath, sharon, KA, natalie, alex, mark, mdd
- We assessed every New Zealand mobile banking app and plotted them on a five-sided spider diagram. Some of the New Zealand apps are basically poor cousins of the Australian “parent” versions.We also made the call to evaluate [one] mobile banking app that is recognised as one of the best in the world – [JP Morgan Chase] - to make sure we didn’t get to Kiwi-introspective.I am a customer of Chase bank, which made it easy to get access.
- Better mobile banking offering = bigger diagram area
- Worse mobile banking offering = smaller area
- Lets quickly put into context the relative resources of the banks we’ve looked at…ANZ = $7bn revenue. Westpac, BNZ and ASB are roughly the same size at $4bn-ish. KiwiBank is about $1bn. And TSB is “only” $200 millionCompare this with Chase bank which turns over $100 billion +
- We ended up with this sort-of “uber-spread-sheet-heat-map“ across these three criteriaLet me break this down a bit for easier consumption
- In 2012, Verve Mobile conducted a survey asking users whatare the most valuable services for a financial app. 63% transfers62% bill pay (Two parts to this, setting up a payee and actually paying a bill)40% Alerts (Three parts to this; text, email, push options – can you do it via mobile?)33% Person to Person payments29% Branch ATM LocatorWe took those five features and added a sixth one; finding a past transaction. Sounds simple and it was, but our participants wanted to know more about the transaction and what their running balance was at that point in time. Why would they want this? More details = greater priming to aid in memory recall. Running balance = reduces cognitive burden.
- As you can see, implementing those features comes in many different concepts, most interestingly is Kiwi Banks Pay and Transfer icon that looks like Pie for some reason.Most banks had the pay/transfer features explicit and upfront like ASB, but when we look at Chase, a lot of participants didn’t know what the marketing speak “Chase Quickpay” meant. Why isn’t paying bills or transfering money, quick? Top right you’ll see Planned Payments. Essentailly it’s pending payments that you requested a future based date on. Of course one of the greatest assests to having a smartphone device is the GPS integration allowing you to find anything anywhere. And of coursse on the bottom right here we have the ability to deposit a check via photography. Tell a Kiwi about this neat feature and they look at you with astonishment; “you use checks in america?”We established in total 26 features that mobile banking could have. This is based on the core six identified by Verve, along with 20 features banks had already introduced in some form. So how many is too many?
- We’ll answer that in a bit. As you can see, TSB has the greatest number of features, followed by ASB. At the bottom, we have WestPac and ANZ with only 3 core features each. Westpac has 5 non-core and ANZ has 3 non-core. Remember the 2006 HBR study on the number of product features and it’s impact on usability? They found that 66% of people will go for the feature rich item, but after using the item, only 44% would actually use it. It will be interesting to see if that held true to our research, six years later!
- What do ratings tell us? For the app maker it’s a quick and easy way to view the direction of your latest release. For the consumer it’s really difficult to know what a rating means. It’s self reported information and sometimes not even real reporting information; the defunked YourAppReviewed dot com use to post hundreds of five star reviews for money. That being noted, real reviews do provide some interesting feedback. [click]
- One bad app release can seriously impact an app’s rating. Greater market penetration, greater the impact. Look at Chase. All versions, 3.5 stars, current version had two bugs according to user comments and was rated 2.3. Out of the 507 reviews of their current app, 269 rated it 1 star!!!!!There is no starndard for user ratings: “Support iPhone 5 please”, “iPhone 5 support ASAP, other than that is good”. Which one was rated three Stars and the other was rated only 1 Star!Ratings also vary by iTunes store, NZ versus USA.
- There’s not much to say about platforms, we know the four major: iOS, android, windows and tablet (adroid/ios). We this round of research we simply stuck with iPad for tablet.
- Random chat about this. Speaks for its self.
- What:[CLICK] In a 2012 survey of users regarding most desired mobile banking features, Transfers (63), Transactions, Bill Pay (62), Alerts (40), Person to Person payments (33), and ATM/Branch locator (29), were the rated the most desired features. We took this as a sign of what to test! Locked apps to whatever was published in the app stores on January 15th, 2013. Who: [CLICK]Millenials, why? According to Deliotte those born from 1979 – 1995 are the young adult consumers that represent the fastest growing segment of today’s workforce and 25% of the global population. Most active mobile banking users. Expected annual income is to reach 3.4 trillion by 2018. Where? [CLICK] We did the testing in Auckland and Wellington. Method: [CLICK]Traditional lab testing, iPhone 4 device with all the apps installed, counter balanced design of course, single individual capturing all data for consistency. Participants could not test their own bank, captured SUS and Stack Ranking of favorite to least favorite app tested.
- [What we found]
- [What we found]
- Poorly implemented features, definitely impact SUS scores and appear to correlate to average # of usability issues identified by participant. Define SUS. [CLICK]Remember the HBR study on feature and it’s impact on usability We found a similar effect here, TSB had the most features and the most issues, yet it ranked dead last as the app that people would choose out of those they tested! What does this tell us? There is no magic number on the limit of features, but the greater the number of features the more important good design is in making those features usable.
- While you look at this graph, Let’s just be clear, this is just a data point, it doesn’t mean they won or have the best user experience!We didn’t want to just take a SUS score and show it to you, so we captured the average issue score which is the average number of usability issues (regardless of severity), and we also captured users stacked ranking of preference which when combined with the SUS score is determining our Uber Usability score. TSB: 19 featuresWestPac: 8 featuresASB: 16 featuresWestPac-3.33 ---- 85 ---- 5.3 = 90.30ASB-1.22 ---- 83.44 ---- 6.9 = 90.34ANZ-2.00 ---- 81.25 ---- 6 = 87.25BNZ-3.13 ---- 80 ---- 6 = 86KiwiBank-5.10 ---- 70.31 ---- 7.5 = 77.81Chase (BM)-6.22 ---- 58.53 ---- 4.2 = 62.73TSB-13.38 ---- 44.38 ---- 1 = 45.38
- IxD experts. We all fall into the trap, it’s better because of X. Translate that from tacit knowledge to objective and evaluative process. Heuristics become dated, quickly. An underlined link on a mobile app, just isn’t relevant. Relevant Framework: What’s important in mobile banking: Security, aesthetic effects – we are dealing with multi-million dollar banks, affective computer matters, efficiency, bringing up relevant keyboards or custom keyboards for numeric entry points. Be objective, but don’t limit your IxD experts. Their design knowledge translates well into psychological principles that can be objective. Be patient, comparison and consistency takes diligence.Our secret sauce: Well we are only showing part of it, but in all its consists of:Areas of focus: 10Criteria: 50
- Starting and Finishing: from logging in the first time to logging outNavigation and IA: labeling, structure, navigation modelErrors: if you encountered any, messaging, clarity, understanding.Accessibility: Let’s face it, for most of us, everyday our eyes are deterorating just a bit. Therefore these apps should adhere to established Accessibility principlesComprehension: as Kris pointed out mobile banking is taking off in under-developed countries. Simple and meaningful words, clarity of iconography, reduce comprehension and the gateway to usage.Efficiency of use: isn’t it the bankers who like to say, Time is money?Layout: Lack of clutter, clean, intuitive, efficientVisual Design: aesthetics are important as it establishes brand, but also aids in navigation, efficiency, IA, etc. General: technical issues, security (auto-matic logout), basic help and support, confirmation messaging as appropriate.
- So we have these five areas that we created to measure the apps against, but we realized it’s not fair to weight them equally. Not deploying to a platform can be a business decision, especially if your customers aren’t there. The user ratings, they can be helpful for finding problems, but they can also be falsified in today’s world. Interaction design, that’s definitely important, but how do you assess that interaction design in the real world; through usability testing. So we weighted our Uber Usability score a bit higher and we also looked at the features. Just because you have 19 features doesn’t mean you are meeting the basic feature needs of your market. So we put a weighting on core features, the ones that the users asked for.
- So, how did it all come out in the wash? [CLICK]
- The worst performing of all the NZ banking apps was ANZ with an over-all score of 56.01 out of 100 [CLICK]While ANZ score well in both Usability and Interaction Design, the ANZ offering is well behind in basic features, platform coverage and app store user ratings. User RatingUber UsabilityCore FeatureIxDPlatformsANZ36.487.2533.3375.9240
- The best performing of all the NZ banking apps was ASB with an over-all score of 77.83 out of 100 [CLICK]While ANZ score well in both Usability and Interaction Design, the ANZ offering is well behind in basic features, platform coverage and app store user ratings. User RatingUber UsabilityCore FeatureIxDPlatformsASB90.690.3455.5680.7880
- For comparison, Chase bank came in 2:nd over-all
- So here is the winners’ podium – the Top 3 performers in our study [CLICK]With an honorouble mention for Westpac as the 3rd best New Zealand mobile banking app
- Feature bloat is acceptable as long as it doesn’t impair usabilityThat creating a robust and objective framework for IxD is challengingThat good usability along with offering core features is an incentive for individuals to switch banksKiwi’s hand out account numbers like business cards. American’s still hand out check’s . . . I don’t know why Chase bank, perhaps next year you’ll do better