The document discusses Google's weighted sort algorithm, also known as expected true value (ETV), which provides a more accurate way to sort and analyze metrics with small sample sizes compared to standard sorting. It recommends using weighted sorting to analyze metrics like conversion rates, bounce rates, and click-through rates to gain better insights into high performing pages, keywords, and other dimensions. The document provides links to spreadsheets from Dr. Pete that allow applying different weighted sorting formulas to one's own analytics data to ask questions like which keywords or pages are most convertible, valuable, or have organic search opportunity.

1. GETTINGYOUNOTICEDONLINE
September 2019
Weighted Sort: Google's not-so-hidden
Algorithm that could Transform your Marketing

2. matthew.kay@aira.net
linkedin.com/in/matthewkay87
twitter.com/matthew_jkay

3. Last year I spoke in Belfast...

4. And we played a game called...

5. ...spot the SEO/marketer by what he’s wearing.

6. s/o: Barry Adams to for pointing this out

7. If you’re not in a checkered shirt,
are you really a marketer?
Takeaway 1

8. Over the last 7 years I’ve worked with lots of clients.

10. Things have, generally, gone pretty well.

12. In spite of these successes...

13. I’ve failed.

14. Lots.

15. Client A

16. Client B

17. Failure p*sses me off.

18. But I believe that
perfection is something to strive for
but never achieved.

19. Part of that process (for me) is using data,
not gut instinct, to improve results.

20. Marketing has a challenge though.

21. Where do we start?

22. There are many questions we ask.

23. Which keywords have the best conversion rate?

24. What time should we post on social media?

25. Which landing pages are bouncing?

26. This is easy. Just sort the data.

28. But there’s a problem.

30. This is not accurate.

31. This is not accurate.
(or useful)

32. Why?

33. Let’s talk about sort.

34. Let’s talk about sort.
Quickly.

35. Let’s talk about sort.
Quickly.
Cause it’s pretty f*cking boring.

36. 123 ABC
Lexicographical order
A -> Z
Z -> A
Numerical order
Ascending
Descending

37. If we go back to to our dataset, we see this issue.

40. We don’t have enough data to be confident they are our
highest bouncing pages.

41. Or as Google put it...

42. The true value of a metric (e.g. bounce rate, conversion
rate, time on site etc) for dimensions (e.g. channel, page,
etc) with small participants will be imprecise.”
Avinash Kaushik
“
”

43. Why?

44. If one person visits a page and bounces, the bounce rate
for that page is 100%.

45. If the next 99 people visit don’t bounce, then the bounce
rate is 1%.

46. The initial dataset is too small to make judgements on.

47. So Google’s engineers were given the
following challenge...

48. Compute the "expected true value" for
each row on a table.
“
”

49. The solution?

50. Weighted Sort

51. Weighted Sort
a.k.a. ETV

52. How did they calculate expected true value (ETV)?

53. Imagine a scale.

55. On one end are zero site visits, and the other,
there are many.

56. Zero Many

57. Values to the left (closer to zero) are, arguably, not very
accurate because we don’t have enough data to be
confident in its output.

58. So they are deemed to be very
close to the dataset’s average.

59. Zero Many
Close to dataset’s average

60. Values to the right (closer to many) are more accurate,
so they are deemed to be very close to the figure
shown.

61. Zero Many
Close to actual

62. Values in the middle are deemed to be a combination
of the two.

63. Zero Many
Combination

65. An example

66. Back to our dataset...

71. A page with a bounce rate of 100% is sorted underneath
other pages with a lower bounce rate.

73. This isn’t just limited to the bounce rate metric.

75. Or the page dimension.

77. But weighted sort is only available for
percentage metrics in GA.

79. And it’s not available in other Google products.

81. Enter Dr. Pete.

82. Enter Dr. Pete

83. https://moz.com/blog/build-your-own-weighted-sort

84. He created a few versions of weighted sort that you can
download for free.

85. Basic ETV formula
Google's ETV sort
Weighted ETV formula
Log-based ETV formula

86. I’d recommend you look at the bottom two as they are a
more accurate calculation.

87. N.B. Don’t expect the same results from each formula.
E.g. The log model is more appropriate when you have
larger range values (e.g. 1-100,000).

88. Let’s take a quick look at Dr Pete’s spreadsheet

89. On the ‘Weighted ETV’ sheet, you’ll see five columns.

90. The dimension.

92. The metric you’re measuring against.

94. The metric you want sort by.

96. A multiple of those two metrics.

98. The ETV.

100. You’ll see an additional column on the log ETV sheet.

102. The log formula uses the Log (base 2) of visits instead of
the raw visit value when working out ETV.

103. This increases accuracy of the ETV calculation for
large data ranges.

104. What are the applications?

105. Weighted sort in GA covers bouncing pages.

106. What other questions can we ask?

107. What time should we post on social media?

109. Apply ETV.

111. Which keywords have the best conversion rate?

113. Add some weighted sort magic...

115. Bonus question:
Should we bid on competitor keywords?

116. Absolutely*
*for this client

118. Which keywords have the best organic search
opportunity?
*my personal favourite

119. Export Google Search Console data.
(or just use the Search Analytics for Sheets add-on)

121. Filter keywords with
average position
between 10 & 20.

122. Filter keywords with
impressions > 500
(or whatever number
is appropriate).

124. Apply one of the weighted sort formulas to the CTR by
impressions dataset.

126. I can now focus my efforts on
keywords and pages that:
● Rank on page 2
● Have enough impressions
● Have a high CTR
● Have commercial intent

127. This sh*t is pure gold.

128. More questions to apply weighted sort to...

129. What are my most valuable pages?

130. Highest bouncing web browser?

131. The best converting PPC landing pages?

132. Top converting channel by revenue?

133. The opportunities for better insight by using weighted
sort are almost limitless.

134. So let’s wrap up

135. Go and buy a checkered shirt.
(unless you already have one)
Takeaway 1

136. Stop relying on standard sort.
It’s really not that accurate.
Takeaway 2

137. Download Dr Pete’s spreadsheet.
Takeaway 3

138. Add your own data and start asking
questions.
Takeaway 4

139. GETTINGYOUNOTICEDONLINE
Thank You
matthew.kay@aira.net
linkedin.com/in/matthewkay87
twitter.com/matthew_jkay