IAC 2024 - IA Fast Track to Search Focused AI Solutions
Conversion Rate Optimisation and the art of Testing
1. Conversion Rate Optimisation and the
art of testing
Amy McInnes - User Experience Consultant
Laura Fealy – Online Marketing Consultant
May 2013
2. What’s in it for you?
• Conversion Rate Optimisation (CRO)
• What it is
• Conversions and their killers
• Our process
• Testing
• Understand different methods
• When to use which
• Example case studies
3. What CRO is not…
It’s not…a single exercise or a one off project to assess how well a site is
performing at a given time.
therefore, it’s not…a single session of user testing, an annual survey, a
round of A/B testing, or the implementation of an analytics solution.
CRO is not a one trick pony…It is not a one off
process.
4. What CRO is…
It is…an on-going cycle; a way of combining different UX elements to
produce a continual testing and analysis process.
the key…constantly benchmarking performance.
CRO is an iterative process – lather, rinse, repeat,
lather, rinse, repeat…
5. CRO – the nitty gritty…
It’s about…the art of increasing the percentage of visitors to your website
that take action and convert into customers or leads.
you get…a thorough analysis of your sites performance against your users
behavior, highlighting key action points, followed by a detailed testing
plan…hence benchmarking!
Start creating a website that’s customer-driven and
realise your sites true revenue potential.
6. Essentially, in order to drive traffic
and improve conversion rates you
need to optimise your site…but what
do you optimise?
7. 7 Conversion Optimisation - 30/05/2013
Optimise everything!
Firstly, define what the aim of your site is?
To make MONEY!
• Break it down - Everything on a site will be there to contribute
towards the sites aims such as conversion
• Without identifying these aims and testing separate elements how
can you say what effects conversion and to what degree?
8. 8 Conversion Optimisation - 30/05/2013
CRO involves
• Maximising existing traffic - the process of converting visitors into leads or
sales
• Identifying barriers - increasing the number of leads by reducing ‘bounce
rates’ by identifying ‘barriers’ to conversion – this includes on and offline, a
users entire journey
• Widening the conversion funnel - creating more sales but also nurturing
the quality of customer experience impacting on average spend
• Reducing churn – keeping customers
CRO involves improving a website to provide a positive
experience for the visitor, increasing the chances of those
visitors converting into a sale or a lead.
9. What counts as a
conversion?
• Purchase
• Download
• Sign up
• Register
• Logins
• Sharing on social
media sites
• A ‘like’ or retweet
• Viewing a video
• Request a call back
10. 10 Conversion Optimisation - 30/05/2013
Conversion killers
• Unclear value proposition
• Lack of trust
• Lack of interest
• Confusion
• Usability issues
• Product specific objections
• Users get bored or distracted
• Users don’t believe your company or product is any good
• Competition
• Perceived risk
• Prospect is just window shopping
• Bad prior experience
11. How do we do it?
With an amalgamation of user research,
analytics data and expert intuition
coupled with a scientific approach
15. 15 Conversion Optimisation - 30/05/2013
1. Analyse - What and Where to Optimise
Analyse
Design & Test
Optimise
KPI / Benchmark Assessment
Content Analysis
User Segment Analysis
Funnel Analysis
Analytics Analysis
Test planning
Success/failure Criteria
Scamps
Interim analysis
Realization
Targeting
Communication
User
Research
Usability Tests
Expert Evaluation
Competitor Benchmarking
Persona Development
Surveys
Wireframes
MVT/AB Testing
Analytics
1
3
4 2
Hypothesis
Solution Design
Test Prioritization
Analytics:
• Right tools = measure and test changes
• Highlights issues
• Doesn’t tell you why or suggest solutions
• Pick what are your main goals of the site
• Understand what drives these goals (assess all aspects of the site
and understand how they aid the goals)
• Use this understanding to build metrics that will allow you to
measure success or failure - benchmarking
16. 16 Conversion Optimisation - 30/05/2013
Utilise Quantitative Data
Identify who your
users are?
How they interact
with your site.
Understand where
and why they
might leave.
Pinpoint seasonal fluctuations,
are they campaign related or
are there further
issues/potential.
Identify where and why
people leave the conversion
process to highlight barriers.
However, despite how much web analytics can tell you
‘what’ the problems are on your site’, it will never tell
you why the problems are there or suggest solutions
17. 17 Conversion Optimisation - 30/05/2013
Incorporating the Qualitative Data
• Statistics and numbers are a starting point to brainstorm
• Qualitative data brings the real understanding to achieving your maximum
profit potential
• Learning more about the customer will not only help you better position
your product for success, it will also help identify areas of friction that are
holding you back
• Their wants and needs
• How they interact with your site
22. 22 Conversion Optimisation - 30/05/2013
3. Design & Test
Analyse
Design & Test
Optimise
KPI / Benchmark Assessment
Content Analysis
User Segment Analysis
Funnel Analysis
Analytics Analysis
Test planning
Success/failure Criteria
Scamps
Interim analysis
Realization
Targeting
Communication
User
Research
Usability Tests
Expert Evaluation
Competitor Benchmarking
Persona Development
Surveys
Wireframes
MVT/AB Testing
Analytics
1
3
4 2
Hypothesis
Solution Design
Test Prioritization
23. 23 Conversion Optimisation - 30/05/2013
Design & Test Techniques
Creative Development &
Wireframes
A/B & Multivariate Test Plans
50% 50%
Version A
20% Conversion Rate
Version B
38% Conversion Rate
Traffic
24. 24 Conversion Optimisation - 30/05/2013
Highest Paid Person’s Opinion
HiPPOs… why rely on the
highest paid person’s opinion?
Say NO to HiPPOs and use
your user’s preferences to
shape the development of
your website. This will
ultimately lead to higher
customer satisfaction and
conversion levels.
25. 25 Conversion Optimisation - 30/05/2013
An introduction to testing
AB Testing
MVT Testing
Segmentation
Behavioural Targeting
ConversionImprovementPotential
Higher
Lower
Testing
Targeting
26. 26 Conversion Optimisation - 30/05/2013
AB testing
What is it: Testing one page variant against another, or number of other pages.
Also known as split testing, you tend to establish results quickly.
When to use it: AB testing works well if you want to test the effectiveness of
different layouts or designs, or if you have just one hypothesis you want to
test.
27. 27 Conversion Optimisation - 30/05/2013
AB testing – an example
The hypothesis: Our lead form takes longer to complete than it should and could
cause some confusion because it’s laid out over two columns.
The solution: Run an AB test using the original as a baseline and test a linear format
with unnecessary headings removed.
The result: A 14% improvement in form completions.
28. 28 Conversion Optimisation - 30/05/2013
Multivariate Testing
What is it: Testing a number of variants on a page
simultaneously in order to determine which
combination of content works well. Also known as
MVT, it can take longer than an AB test to establish
a statistically viable result.
When to use it: MVT should be used when you have a
number of elements you want to test on ONE page.
It will help you establish how the elements work
together and indentify an optimal mix for
conversion.
29. 29 Conversion Optimisation - 30/05/2013
MVT testing – an example
The hypothesis: There are too many
elements on the product page that
distract from the CTA.
The solution: Run a multivariate test
hiding and showing various page
elements as well as experimenting with
different styles.
The result: No improvement or decline…...
30. 30 Conversion Optimisation - 30/05/2013
Segmentation
What is it: Targeting a specific group of users based on
their shared behaviour and preferences.
Segmentation can be an automated or a manual
process.
When to use it: Segmentation should be used when
you have distinct groups of users who share similar
behaviors/preferences. You should use
segmentation when you have a suitable amount of
data on your customer groups. This allows you to
tailor content to their particular needs. .
31. 31 Conversion Optimisation - 30/05/2013
Segmentation– an example
The hypothesis: Different segments of customers will look for different information
on a product page.
The solution: Run a segmentation campaign based on the analysis of the multivariate
test. Target new, returning and previously purchased customers with their
preferred content.
The result: A 3% improvement in revenue.
Full
Mid
Min
32. 32 Conversion Optimisation - 30/05/2013
Behavioural Targeting
What is it: Targeting a specific individual based on
their interactions and preferences. Also known as
BT, this is an automated process.
When to use it: BT can only be effectively
implemented when you have enough data to
confidently provide a user with a tailored
experience. If you have a high traffic site, strong
analytics and the ability to tie in your CRM data,
then BT can help drive incremental growth
33. 33 Conversion Optimisation - 30/05/2013
Behavioural Targeting – An underutilised tactic
Why is BT not used?: The majority of
BT is currently run offsite. Few sites
effectively leverage BT for
conversion. This is often down to a
lack of data or simply not knowing
where to start.
An Example: Amazon is one of the few
examples that use targeting and
tailor their main page for the
individual preferences of the
customer.
34. 34 Conversion Optimisation - 30/05/2013
Tools and Resources
No Budget Some Budget/ Lower site
traffic
High budget/high site
traffic
Google Content
Experiments (free)
Visual Website Optimiser
($49+ per month)
Maxymiser
(usually 4 figure sum)
Optimizely
($19+ per month)
Test and Target
(usually 4 figure sum)
http://www.conversion-rate-experts.com/split-testing-software/
Useful comparative tool
35. 35 Conversion Optimisation - 30/05/2013
Testing hints and tips – Build
• Pick the correct test based on your hypothesis.
• Cross browser check the variants. Be cautious of
WYSIWYG solutions.
• Ensure all conversion pages are correctly tagged.
• Avoid using a container tag solution.
• Most solutions cause a flicker and can negatively affect the user
experience.
• Know the cost of running the test and monitor as it
progresses.
36. 36 Conversion Optimisation - 30/05/2013
Testing hints and tips – Running
• Start of with a low percentage of traffic and if appropriate build to 100%
• Monitor performance of variants if using standard testing models.
• Remove poor performing variants where and when appropriate.
• For arm bandit solutions, monitor to ensure that all variants do get traffic
driven to them at the beginning of the test.
• Consolidate variants in analytics to ensure
accurate reporting.
• Avoid running simultaneous tests
• Try not to look at the results too often....
37. 37 Conversion Optimisation - 30/05/2013
Testing hints and tips – Analysis
• Ensure a statistical confidence has been met.
• Ensure a suitable percentage of traffic has been exposed to the test.
• Do not be disheartened by a negative result.
• Use segmentation to identify any targeting opportunities.
• Push winning variant live all site visitors.
39. 39 Conversion Optimisation - 30/05/2013
4. Optimise
Analyse
Design & Test
Optimise
KPI / Benchmark Assessment
Content Analysis
User Segment Analysis
Funnel Analysis
Analytics Analysis
Test planning
Success/failure Criteria
Scamps
Interim analysis
Realization
Targeting
Communication
User
Research
Usability Tests
Expert Evaluation
Competitor Benchmarking
Persona Development
Surveys
Wireframes
MVT/AB Testing
Analytics
1
3
4 2
Hypothesis
Solution Design
Test Prioritization
This stage is more about
briefly congratulating
yourself on the work you
have done and remembering
you now need to repeat on
an on-going basis so you are
continually improving
conversion over time.
40. By measuring the changes in
Analytics we can create the next
phase of testing…benchmarking
41. 41 Conversion Optimisation - 30/05/2013
The Benefits
• Convert more visitors and increase repeat business
• Stay ahead of competition
• Be a leader rather than a follower
• Increasing conversion rates = increase overall profit
• Increased brand perception
• Testing gives statistical evidence to back up changes
Remember, no matter how good your SEO campaign
is at attracting more visitors, a poor site design will
not increase your conversion rates.
42. 42 Conversion Optimisation - 30/05/2013
Only for Ecommerce Sites? No!
Any Ecommerce website or brand concerned with lead generation can
benefit from CRO, however there are a couple requirements:
– Be able to provide access to web analytics
– Have specific measurable goals in place
– Have the ability to implement changes based on the resulting
recommendations
43. When?
CRO can start at any time during your
development cycle from pre-launch to
re-development. There’s no time like the
present.
46. 46 Conversion Optimisation - 30/05/2013
To summarise
Conversion rate optimisation is:
• NOT a one off exercise
• It is NOT a single expensive tool
• It is a continual process of improvement combining a number of basic
weapons available to everyone:
• Benchmark using analytics, surveys and user feedback
• Analyse the information to highlight areas for improvement
• Test ways to improve these areas and see what effect this has
47. 47 Conversion Optimisation - 30/05/2013
To summarise
Conversion rate optimisation benefits:
• Increase users
• Improve the user experience
• Maintain a strong / positive brand image
• First-mover advantage - Stay one step ahead of the competition
• Get to the root of problems and solve them
• Eliminate risk and costly guess-work
• Increase profit
48. 48 Conversion Optimisation - 30/05/2013
To summarise
Testing methods
• AB testing - single hypothesis testing
• MVT - multiple hypothesis testing
• Segmentation - targets a group of users
• Behavioural targeting - is for individual users
Hints
• Don't rush into spending huge volumes on a testing solution
• When analysing, try to segment the data to identify trends
Its about - Essentially, in order to drive traffic and optimise conversion rates, every visitor will have a problem which your website needs to solve.You get – Actionable insights that highlight improvements through a monthly report. It an iterative development process that’s repeated to increase your ROI
Pretty obvious I agree, but you do need to think about aims more specific than ’to make money’. Remember, everything on the site is geared towards your goals, so everything on the site must have their own goals that contribute towards these.
Could ask audience what sort of questions they should be asking their users?E.g.To help you structure your content to better relate to your customers and better frame your purchase triggers, you want to learn:Why do your customers need your product?What are they missing out on by not having it?What opportunities will the use of your product bring them?What is their main purchase motivation?Have they tried to use a similar product before?After having used your product, what do they think the benefits will be to them?What are they expecting the outcome of using your product to be?How do they plan to use the product?To get a better understanding of what has worked for you and to learn sometimes surprising info about what actually ended up closing the sale, ask existing customers:How did they find out about you?What information on your page were they surprised to learn?What did they like about your ads and landing page?If there were one thing they would change about the ads or landing page, what would it be?Critically, for people who came to your website but didn’t purchase, you’d like to know:Did they find everything they were looking for?Why did they decide not to purchase?After visiting your site, what did they do next?Did they still want the product and choose to buy from a competitor?Or did they decide not to purchase the product at all?What was something you could have done to persuade them to purchase?
WireframesMVT/AB TestingAnalytics Test planningSuccess/failure CriteriaScampsHypothesisSolution DesignTest Prioritization
