When IBM started using Optimizely in late 2015, they were running less than a hundred experiments a year. Fast forward to today and they’re on track to run well over 3,000 a/b tests and personalization campaigns this year. In this webinar you’ll get a behind the scenes look at how the testing Center of Excellence (COE) at IBM empowers one of the world’s largest marketing organizations to achieve digital excellence.
6. Revenue, share of wallet, funnel conversion, risk
mitigation, ops efficiency
What
We Do Next gen “Test and Learn” platform for enterprise-
wide digital experience optimization
Over 1B impressions daily
Replaces Digital Guesswork
With Evidence-based Optimization
Digital Experimentation
SaaS Platform
Apply the scientific method to “at-scale”
business decision making.
20X increase in Yield
7. 1MEXPERIMENTS RUN
1BEXPERIENCES PER DAY EVENTS HANDLED PER DAY
6B
24OF FORTUNE 100
80NPS FOR TECH.
SUPPORT
OF THE INTERBRAND
40%
Optimizely is the World’s Leading Experimentation
& Personalization Platform
11. 11
A/B Testing
Find the average-best experience
Personalization
Deliver the best experience for each
visitor
12. 12
“In the online world, businesses have the opportunity to
develop very deep relationships with customers...so that
you can use that individualized knowledge of the
customer to accelerate their discovery process.”
Jeff Bezos
“Accurately predicting the movies Netflix members
will love is a key component of our service”
Neil Hunt, Chief Product Officer
The value of personalization is in the trillions of dollars
Personalization at scale
has the potential to create
$1.7 trillion to $3 trillion
in new value.
Based on projections by McKinsey Global Institute
13.
14. Chapter 1 How do you begin?
14
Experiments Created per Year
2015 97
Program Roll-Out
Pre-Marketing
A/B Tests Personalization
15. Scaling a mindset of web experimentation at
IBM has required significant cultural shift
15
PAST
The challenge that was blocking adoption in 2015?
It was a centralized program owned by one person –
resulting in an undemocratic process for a practice
meant to be collaborative.
Hey there, Ari!
How are you doing?
16. Chapter 2 How do you scale?
16
2017
2018
679
2,615
Experiments Created per Year
2016 512
2015 97
Program Roll-Out
Pre-Marketing
Program Roll-Out
Post-Marketing
A/B Tests Personalization
17. 17
CMO Support
IBM CMO Michelle Peluso kicked off
the Quarter of Client-Centricity in Q2
2018.
Making the Marketer
We kicked off a worldwide tour to
bring best practices from all
disciplines, including
experimentation, to every marketer.
2018 CHQ Contest
2018 saw immediate adoption of
personalization, yielding many novel
ideas worldwide, with support of
newly launched CoE.
2018: Review of a year in experimentation
18. A decentralized program allowed marketers to
embrace their successes and reframe failures
18
Make testing a part of the campaign planning
& content optimization process.
Testing & Personalization Center of
Excellence guides marketing teams at all
levels of proficiency.
Understand the drivers of performance and
the areas of opportunity. Iterate on
experiments & always learn from previous
results.
19. Chapter 3 Go from counting to culture
19
2017
2018
679
2,615
Experiments Created per Year
2016 512
2015 97
Program Roll-Out
Pre-Marketing
Program Roll-Out
Post-Marketing
A/B Tests Personalization
2019
YTD
2,179 ?
20. A cultural shift from optimizing web pages in silos
to connecting the digital journey
20
Dynamic
Ad Creative
Ad
Lorem
ipsum
ad finis
CTA
Image
Ad Ad Ad
Personalized
Page Nurture
21. Optimizing not only one web page but connecting the digital journey
21
Full creative execution of
Cloud Narrative included
both prospecting and
retargeting with
differentiated creative
Application Innovation
Data & AI
Multi Cloud
Display banner1
Prospecting Retargeting
On-site personalization2
https://www.ibm.com/cloud/yourcloud/
23. Chapter 4 How do you make this matter
for business outcomes?
23
24. We are always re-assessing the impact of our web presence
24
Personalization
Design Language
Enterprise Architecture
SEOJourney
Analytics & Data
Product Marketing
Multiple go-to market strategies
and supporting platforms
Web
Simplification
Governance
Standards
Integrated platform
Centralized web strategy
25. Working towards 2020 web innovations like
always-on personalization built into the CMS
25
Always-On Personalization
Customizing next best action for visitor to take
based on journey and behavior
Recommended
for you
The IBM z14 family of servers is the
core of trusted digital experiences.
Technology challenge?Ask a
bona fide expert at no cost.
Hear real-world successes with the
secure and scalable and IBM Z
1
2
3
Page experience
Authoring experience
Platform foundations
26. 26
“If you want to build a ship,
don’t drum up the people
to gather wood, divide the
work, and give orders.
Instead, teach them to yearn
for the vast and endless sea.”
– Antoine de Saint-Exupéry, the author of The Little Prince
Hi Everyone, thanks so much for joining us for today’s Digital Change Agents session. My name is Alex Lee and I am a Customer Success Manager here at Optimizely.
We are thrilled to have Anne Reints who will be talking about how Mailchimp’s experimentation program has evolved from a marketing focused activity to become a key influencer on the product side.
How are you this morning Anne?
Let her say hi then carry on with your slides
I suggest running through that with Anne
A few housekeeping items for today:
We are recording this webinar
You’ll receive the recording and slides in a follow-up email
and we want today’s session to be interactive. So please ask questions. You can do so by entering them in the chat box of Go to webinar console.
This is part of a series of webinars called Digital Change Agents. This webinar series is designed to share stories from our customers about how they have built a testing program, are focusing on experimentation and have able to use experimentation to drive progress against the metrics that matter the most to them . Previous DCA session recordings can be accessed at the link shown on this slide
optimizely.com/digital-change-agents/
Optimizely is the market leader in a category called “Digital Experience Optimization.” Optimizely’s solutions help you optimize your digital products, digital commerce and also digital campaigns.
Our solution is a Digital Experimentation Platform. If you are not familiar with the concept of experimentation, it is a next generation “test and learn” system that enables you to continuously test new features and user experiences in order to determine which variations will provide the best business outcomes.
This is a sampling of the business impact Optimizely yields for some of the top brands in the world.
By using Optimizely to improve conversion and acquisition funnels, cart performance, and the in-product experience, these customers are replacing digital guesswork with real-time information about how their users perform, and rolling out the winning experiments.
Peter Gray, VP of Optimization at The Wall Street Journal, put it best when he said that “the discipline of experience testing is an exponentially more powerful way of making business decisions”
These examples drive home a simple concept: the more you experiment, the faster you learn - both what works and doesn’t work. Experimentation completely de-risks business decision-making, and enables you invest more in ideas you know will work.
Let’s talk a bit more about experiments. In its simplest form, an experiment is showing different users a different variation of an experience, measuring the results, and deploying the variation that wins.
By doing this, you can double-down on the ideas that work. And you can eliminate wasting resources on bad ideas that fail with real users. Experimentation has a compound effect, much like compound interest. The more you experiment, the faster you create a better experience upon which you can optimize again.
Let me show you what I mean by an experience...
Let’s talk a bit more about experiments. In its simplest form, an experiment is showing different users a different variation of an experience, measuring the results, and deploying the variation that wins.
By doing this, you can double-down on the ideas that work. And you can eliminate wasting resources on bad ideas that fail with real users. Experimentation has a compound effect, much like compound interest. The more you experiment, the faster you create a better experience upon which you can optimize again.
Let me show you what I mean by an experience...
So let’s dig into personalization a bit more.
According to McKinset, the potential value of Personalization is in the trillions of dollars across core industries. That is incredible. This is still an opportunity that hasn’t been fully realized by many brands today. Amazon and Netflix are notorious for being the pioneers in this area, and are leading the way in delivering personalized experiences for their prospects and customers. This in turn creates customer loyalty as well as revenue opportunities.
Here is what we have in store for today:
I’m going to introduce today’s session here in a minute
And then spend some time talking about who Optimizely is and how we help companies like yours.
After that we have the main event where we’ll hear from the Intuit team on why they’ve built experimentation into their business.
And lastly I will kick off a Q&A period before taking questions from you all. So, please ask questions — you can do so by submitting them in the chat box of Go To Webinar console.