The document discusses site search analytics from both a bottom-up and top-down perspective. It describes analyzing search query data to understand common queries, failure rates, and metadata patterns. It also discusses defining search-related metrics and benchmarks to measure findability and performance from a top-down perspective. The key is putting both approaches together to understand what is being measured and why.
10. SSA from the Bottom Up
★ The basics: play and ask
questions
★ Five things you should be
doing
8
11. Generic questions help
you play with your data
★ What are the most frequent unique queries?
★ Are frequent queries retrieving quality results?
★ Click-through rates per frequent query?
★ Most frequently clicked result per query?
★ Which frequent queries retrieve zero results?
★ What are the referrer pages for frequent
queries?
★ Which queries retrieve popular documents?
★ What interesting patterns emerge in general?
9
12. Bottom Up SSA:
Five things you should do
1. Cluster your data to get a better picture
of metadata and content needs
2. Track for seasonality
3. Take failure further: beyond failed
searches
4. Leverage your best bets
5. Don’t be satisfied with generic reports
10
31. Analyzing data
from the bottom up:
play with the data,
look for patterns, trends,
and outliers
32. Analyzing data
from the bottom up:
play with the data,
look for patterns, trends,
and outliers
So what’s being measured?
33. SSA from the Top Down
★ The basics: why are we here?
★ The hard part: what can we
measure?
19
34. First: why are we here?
★ Commerce
★ Lead Generation
★ Content/Media
★ Support/Self-Service
20
35. First: why are we here?
★ Commerce
★ Lead Generation
★ Content/Media
★ Support/Self-Service
Data supports metrics... but
which metrics for search?
20
38. Vanguard and the
quantification of search
Target Oct 3 Oct 10 Oct 16
Mean distance from 1st 3 13 7 5
Median distance from 1st 2 7 3 1
Count: Below 1st 47% 84% 62% 58%
Count: Below 5th 12% 58% 38% 14%
Count: Below 10th 7% 38% 10% 7%
Precision – Strict 42% 15% 36% 39%
Precision – Loose 71% 38% 53% 65%
Precision – Permissive 96% 55% 72% 92%
Quantification, not monetization
22
39. Search-related metrics
★ Jeannine Bartlett’s SIX Metrics(tm)
Framework
★ Lee Romero’s search metrics
★ Both here: http://bit.ly/1a2mzk
Disconnect: analytics world
of KPI vs. experiential world
23
of search
52. The data that
drive our decisions
Web Analytics User Experience
behavioral attitudinal
quantitative qualitative
high fidelity artificial
high volume high quality
This data is about WHAT This data is about WHY
29
53. The data that
drive our decisions
Web Analytics User Experience
behavioral attitudinal
quantitative qualitative
high fidelity artificial
high volume high quality
This data is about WHAT This data is about WHY
29
54. The data that
drive our decisions
Web Analytics User Experience
behavioral attitudinal
quantitative qualitative
high fidelity artificial
high volume high quality
This data is about WHAT This data is about WHY
29
55. The data that
drive our decisions
Web Analytics User Experience
behavioral attitudinal
quantitative qualitative
high fidelity artificial
high volume high quality
This data is about WHAT This data is about WHY
29
56. The data that
drive our decisions
Web Analytics User Experience
behavioral attitudinal
quantitative qualitative
high fidelity artificial
high volume high quality
This data is about WHAT This data is about WHY
29
57. The data that
drive our decisions
Web Analytics User Experience
behavioral attitudinal
quantitative qualitative
high fidelity artificial
high volume high quality
This data is about WHAT This data is about WHY
29
59. Common queries
can drive task analysis
“Can you find a map of
the campus?”
“What study abroad
options are available to
students?”
“When is the last home
football game of the
season?”
30
70. What we covered
★ Quick intro
★ SSA from the Bottom Up
★ SSA from the Top Down
★ Putting them together
35
71. Some day my book
will come...
Search Analytics for Your Site:
Conversations with Your Customers
Louis Rosenfeld & Marko Hurst
Rosenfeld Media, 2009 (?)
rosenfeldmedia.com/books/searchanalytics
36
72. Until then...
Louis Rosenfeld
457 Third Street, #4R
Brooklyn, NY 11215 USA
lou@louisrosenfeld.com
www.louisrosenfeld.com
www.rosenfeldmedia.com
Twitter: louisrosenfeld, rosenfeldmedia
37
Editor's Notes
...so does Excel
“The center can not hold!”
You’ll notice this isn’t a canned report
This all means putting pressure on commercial analytics apps to change
“The center can not hold!”
You’ll notice this isn’t a canned report
This all means putting pressure on commercial analytics apps to change
“The center can not hold!”
You’ll notice this isn’t a canned report
This all means putting pressure on commercial analytics apps to change
“The center can not hold!”
You’ll notice this isn’t a canned report
This all means putting pressure on commercial analytics apps to change
BTW, Vanguard is now mining the Long Tail
Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
you can do this, regardless of how you feel about data
note that it’s in Excel
you can do this, regardless of how you feel about data
note that it’s in Excel
Rosenfeld Media is the publishing equivalent of the Slow Food movement