Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (9)
Similar to Split Testing for SEO - 9 Months of Learning
Similar to Split Testing for SEO - 9 Months of Learning (20)
More from Dominic Woodman
More from Dominic Woodman (7)
Recently uploaded
Recently uploaded (20)
Split Testing for SEO - 9 Months of Learning
- 1. @dom_woodman
- 2. This talk is about split testing
- 3. Money.
- 4. Talked ourselves out of money How to get the money back
- 5. Talked ourselves out of money How to get the money back
- 6. How did we talk ourselves out of the money?
- 7. Part 1
- 8. Not knowing things is scary
- 11. ?
- 15. We’re paid experts, of course we should know
- 16. 3 Part 2 Fear
- 17. SEO is hard to measure
- 18. Before
- 19. Before After
- 21. Not money. At least we’re on sessions
- 22. Still not money. Now we’re on clicks.
- 23. What on earth is search visibility?
- 24. Seasonality
- 28. Ranks
- 31. No-one cares when the going is good
- 33. Part 3: SEO has become more individual
- 34. More ranking factors Which interact in more complex ways
- 36. So how does this all fit together
- 37. So how does this all fit together
- 38. So how does this all fit together
- 39. So how does this all fit together
- 40. So how does this all fit together
- 41. So how does this all fit together
- 42. So how does this all fit together
- 43. So how does this all fit together
- 44. So how does this all fit together
- 45. So how does this all fit together
- 46. So how does this all fit together
- 47. So how does this all fit together
- 48. So how does this all fit together
- 49. So how does this all fit together
- 50. So how does this all fit together
- 51. So how does this all fit together
- 52. So how does this all fit together
- 53. So how does this all fit together
- 54. SEO Manager: Jan 2016 - Jan 2017 ● SEO Manager at T-Shirts and Shorts. ● Completed 4 major projects. Reduced Dev backlog. ● Removed a penguin penalty. Regularly audited backlinks. ● Organic traffic went up from 100K ~ 150K What does my CV look like?
- 55. SEO Manager: Jan 2016 - Jan 2017 ● SEO Manager at T-Shirts and Shorts. ● Completed 4 major projects. Reduced Dev backlog. ● Removed a penguin penalty. Regularly audited backlinks. What does my CV look like? -- If I’m unlucky
- 56. Talked ourselves out of money How to get the money back
- 57. Stop making guesses about our results
- 58. CRO and growth hacking are doing it right
- 61. Learning from split tests
- 62. What does an SEO split test look like? Control Variant
- 63. This is a traditional CRO test Control Variant
- 64. I’m about to show you 2 graphs a lot. This is the first.
- 65. Black is the variant, treat blue as the control
- 66. In this case the blue line is higher, it’s a negative test
- 67. This is the start date
- 68. The second graph shows the total additional sessions
- 69. The fans show margin of error
- 70. The test is significant when the fan crosses 0
- 71. In this case it happened here:
- 72. Here they are together.
- 73. 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data
- 74. 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data
- 75. Established wisdom would say ALT tags on images are good for SEO
- 76. We found adding ALT tags to images had no effect
- 77. 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data
- 78. How does SEO text on category pages perform?
- 79. E-commerce site number 1 - No effect/negative effect
- 80. E-commerce site number 2 ~ 3.1% increase in organic sessions
- 81. 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data
- 82. We can render Javascript!
- 83. What happens if your content is only visible with Javascript? Javascript EnabledJavascript Disabled
- 84. Making it visible increased organic sessions by ~ 6.2%
- 85. 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data
- 86. Title tag before: Which TV should I buy? - Argos Title tag after: Which TV to buy? - Argos What happens when you match title tags to the greatest search volume?
- 87. Organic sessions decreased by an average of 8%
- 88. 1.Adding ALT tags 2.Removing SEO category text 3.Using JS to show content 4.Setting exact match title tags 5.Adding structured data
- 89. Category pages have lots of images and not much text
- 90. Adding structured data to category pages
- 91. Organic sessions increased by 11%
- 92. Running your own tests
- 93. 1.How much traffic do you need? 2.How long should you run a test for 3.What kind of things are easy to test? 4.How can you do all the maths?
- 94. 1.How much traffic do you need? 2.How long should you run a test for 3.What kind of things are easy to test? 4.How can you do all the maths?
- 95. Average of 1,000 organic sessions per day to your set of pages
- 96. 1.How much traffic do you need? 2.How long should you run a test for 3.What kind of things are easy to test? 4.How can you do all the maths?
- 97. Typically takes 2 - 3 weeks to reach significance.
- 98. 1.How much traffic do you need? 2.How long should you run a test for 3.What kind of things are easy to test? 4.How can you do all the maths?
- 99. Anything where you’re making changes to a page
- 100. 1.How much traffic do you need? 2.How long should you run a test for 3.What kind of things are easy to test? 4.How can you do all the maths?
- 104. Thanks! @dom_woodman
Editor's Notes
- I’m sure you read it on the programme
- We made our jobs harder and made ourselves easier to fire. So how did we talk ourselves out of it?
- So how did we talk ourselves out of it?
- So how did we talk ourselves out of it?
- This isn’t the same as when you’re kid. When your a kid it’s the imagination that gets you
- You know what's scary:
- That doesnt happen as a kid, When a teacher asks, what’s going to happen as they pour vinegar in a volcano and you have no idea. But it’s amazing and you learn One of my favourite days in primary school -- we’re in primary school -- we’re making dioramas for our project on hawaii (st lucia?), someones down the houses, the beach, the trees. -- our teacher she comes out with this big paper mache volcano, -- holds up a bottle of vinegar and asks us whats about to happen, -- we’ve not seen it we have no idea. David asks if she’ll drink it. -- pours half a bottle of vinegar into the top and it explodes with lava -- it was so cool -- all the unknown - we had no idea how it worked, it was like magic -- what made that happen? None of us had any idea but we wanted to know
- As an adult standing in a board room when someone hands you a new set of numbers asks you a question you don’t know, it’s terrifying
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- Boy am i jealous of PPC everytime there is a quarterly meeting and they say, Last quarter we said we’d add buy more stuff into all our titles. you gave us £16,000 we gave you 800 leads last quater. This quarter we gave you 900 leads.
- Sadly that’s how for most of us, things change as we get older. It gets harder and harder to ask questions and admit we don’t know things. Particularly when we’re all meant to be experts on it. And this is where we struggle in business. Here’s the problem -- seo is hard to measure -- but i can’t measure my changes against that, i have to control for seasonality and google updates, so we talk about ranks -- even then i’m hand wavy. What if I rank for new keywords? Russ talks about search console data. But businesses speak the language of traffic and revenue, they need numbers -- we sit as consultants and seo managers as the experts -- people ask us about this black box -- they want those numbers and we can’t get them accurately. But we can’t admit that, we’re not kids we’re experts and so begins bad numbers and hand waving. SEO has always had these problems, but some things have changed: -- it’s got harder, the system has got harder to game and more complicated. It’s no longer the case The solution -- leave our egos at the door and accept what has been accepted in CRO & growth hacking that, the role is not pick and choose and never measure -- we have to make an evaluation of the maths of case studies -- we have to learn from rigourous tests -- we have to test things ourselves
- What have we learnt from tests -- structured data is a good thing -- we’ve seen uplifts consistently from adding structured data to schema -- render experiment from icanvas -- block changes to title tags have been unsuccessful -- removing seo text from category pages has helped and hindered -- removing ads has not had any consistent effects -- title change to exact match keyword was negative -- schema for upwork -- entities for next web -- combining pages for realtruck
- What have we learnt from tests -- structured data is a good thing -- we’ve seen uplifts consistently from adding structured data to schema -- render experiment from icanvas -- block changes to title tags have been unsuccessful -- removing seo text from category pages has helped and hindered -- removing ads has not had any consistent effects -- title change to exact match keyword was negative -- schema for upwork -- entities for next web -- combining pages for realtruck
- What have we learnt from tests -- structured data is a good thing -- we’ve seen uplifts consistently from adding structured data to schema -- render experiment from icanvas -- block changes to title tags have been unsuccessful -- removing seo text from category pages has helped and hindered -- removing ads has not had any consistent effects -- title change to exact match keyword was negative -- schema for upwork -- entities for next web -- combining pages for realtruck
- Boy am i jealous of PPC everytime there is a quarterly meeting and they say, Last quarter we said we’d add buy more stuff into all our titles. you gave us £16,000 we gave you 800 leads last quater. This quarter we gave you 900 leads.
- We can’t just cut and paste other people’s ideas & results anymore
- but people want to know what our changes will do
- so we have to estimate the effect of our changes beforehand
- and measure it after the fact.
- But we’re meant to be experts people expect us to know
- and we want that money
- Measuring SEO is hard
- Which means we use all these data sources, we use ranks and we calculate CTR estimates to estimate traffic.
- But this is all smoke, mirrors and hand waving,
- but it does mean we get our money in the short term
- But it also means our estamates are more likely to be wrong and make no sense
- Nobody cares when everything is going well
- But as soon as things run into trouble everyone begins to question your estimates and measuring and point out they’re awful
- Once you’ve been burned by this a couple times
- You might give it a try with just ranks & clicks
- But then you’re unconvincing
- And you don’t get the money, because the other departments make better justifications so nothing happens
- So how did we talk ourselves out of it?
- Split your pages into 2 buckets, make changes to one of them and measure the difference in traffic from google
- Split your pages into 2 buckets, make changes to one of them and measure the difference in traffic from google
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
- -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong