SlideShare a Scribd company logo
1 of 72
Download to read offline
Improving Findability
             through
Site Search Analytics


  Louis Rosenfeld • May 12, 2009
   ESS 2009 • New York, NY, USA


      1
What we’ll cover
★ Quick intro
★ SSA from the Bottom Up
★ SSA from the Top Down
★ Putting them together

              2
Quick Intro
★ Where search query data
comes from
★ Our friend Zipf
★ Long tail, meet short head

               3
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxy
  stylesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1quot;
  200 971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip
  =XXX.XXX.X.104 HTTP/1.1quot; 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxy
  stylesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1quot; 200 9718 62 0.17




Sample query data
(from Google Search Appliance)
                               4
The Zipf Distribution




    5
Zipf, textually:
the power of the short head




              6
Improving Findability through Site Search Analytics
Improving Findability through Site Search Analytics
But can you get at your data?
SSA from the Bottom Up
★ The basics: play and ask
questions
★ Five things you should be
doing

               8
Generic questions help
you play with your data
★ What are the most frequent unique queries?
★ Are frequent queries retrieving quality results?
★ Click-through rates per frequent query?
★ Most frequently clicked result per query?
★ Which frequent queries retrieve zero results?
★ What are the referrer pages for frequent
queries?
★ Which queries retrieve popular documents?
★ What interesting patterns emerge in general?
                       9
Bottom Up SSA:
Five things you should do
1. Cluster your data to get a better picture
of metadata and content needs
2. Track for seasonality
3. Take failure further: beyond failed
searches
4. Leverage your best bets
5. Don’t be satisfied with generic reports

                    10
Hunting for metadata patterns
CANDIDATE VALUES   CANDIDATE ATTRIBUTES




                   11
Hunting for metadata patterns
CANDIDATE VALUES   CANDIDATE ATTRIBUTES




                   11
Hunting for metadata patterns
CANDIDATE VALUES   CANDIDATE ATTRIBUTES




                   11
Hunting for metadata patterns
CANDIDATE VALUES   CANDIDATE ATTRIBUTES




                   11
Hunting for metadata patterns
CANDIDATE VALUES   CANDIDATE ATTRIBUTES




                   11
Hunting for content types




              12
Hunting for content types




              12
Hunting for content types




              12
Surfacing content types




            13
Surfacing content types




            13
The When of search
   14
Failure is underrated:
digging deeper




               15
Beyond best bets
             16
Netflix moves beyond
generic reports




            17
Netflix moves beyond
generic reports




            17
Netflix moves beyond
generic reports




            17
Netflix moves beyond
generic reports




            17
Netflix moves beyond
generic reports




            17
Analyzing data
from the bottom up:
play with the data,
look for patterns, trends,
and outliers
Analyzing data
from the bottom up:
play with the data,
look for patterns, trends,
and outliers

So what’s being measured?
SSA from the Top Down

★ The basics: why are we here?
★ The hard part: what can we
measure?


              19
First: why are we here?
★ Commerce
★ Lead Generation
★ Content/Media
★ Support/Self-Service


              20
First: why are we here?
★ Commerce
★ Lead Generation
★ Content/Media
★ Support/Self-Service
Data supports metrics... but
which metrics for search?
              20
Can we measure
findability?




           21
Can we measure
findability?


Does measure mean
monetize?

           21
Vanguard and the
quantification of search
                            Target         Oct 3   Oct 10   Oct 16
 Mean distance from 1st         3          13        7        5
 Median distance from 1st       2           7        3        1
 Count: Below 1st             47%         84%      62%      58%

 Count: Below 5th             12%         58%      38%      14%

 Count: Below 10th             7%         38%      10%      7%

 Precision – Strict           42%         15%      36%      39%
 Precision – Loose            71%         38%      53%      65%
 Precision – Permissive       96%         55%      72%      92%




Quantification, not monetization
                                     22
Search-related metrics
★ Jeannine Bartlett’s SIX Metrics(tm)
Framework
★ Lee Romero’s search metrics
★ Both here: http://bit.ly/1a2mzk

Disconnect: analytics world
of KPI vs. experiential world
                    23




of search
Analyzing data
the top down:
start with metrics,
benchmark and
measure performance
Analyzing data
the top down:
start with metrics,
benchmark and
measure performance
But you can’t measure
what you don’t know
Putting it all together

Top-down analysis
Bottom-up analysis



               25
Putting it all together what

Top-down analysis
Bottom-up analysis



             25
Putting it all together what

Top-down analysis
Bottom-up analysis

                       why
             25
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1quot; 200
  971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX
  X.XXX.X.104 HTTP/1.1quot; 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1quot; 200 9718 62 0.17




                               26
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1quot; 200
  971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX
  X.XXX.X.104 HTTP/1.1quot; 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1quot; 200 9718 62 0.17




  BU Q: “What are the most
  common queries?”
                               26
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1quot; 200
  971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX
  X.XXX.X.104 HTTP/1.1quot; 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1quot; 200 9718 62 0.17




                               27
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1quot; 200
  971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX
  X.XXX.X.104 HTTP/1.1quot; 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] quot;GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1quot; 200 9718 62 0.17




  TD Q: “Are we converting
  license plate renewals?”
                               27
!quot;#$%"'()*+),%(-).(%(quot;-&/)0(1/*$%)
                                                                   /
      Behavioral                                                         Eyetracking                            Data Mining/Analysis
                                                                                                                A/B (Live) Testing

                                                                   Usability Benchmarking (in lab)



                                                                                            /
           Data Source
                                  Usability Lab Studies                                          Online User Experience Assessments
                                                                                                 (“Vividence-like” studies)


                                  Ethnographic Field Studies
                         mix




                                                                    Diary/Camera Study
                                                                    Message Board Mining
                                  Participatory Design              Customer feedback via email
                                  Focus Groups                            Desirability studies                  Intercept Surveys
      Attitudinal                 Phone Interviews                        Cardsorting                           Email Surveys

                                                                               mix
                                                                         Approach
                               Qualitative (direct)                                                     Quantitative (indirect)
                                Key for Context of Product Use during data collection
                                  Natural use of product                             De-contextualized / not using product
  © 2008 Christian Rohrer         Scripted (often lab-based) use of product          Combination / hybrid
                                                                                                                                 20


A LOVELY USER RESEARCH STRAW MAN
                                                                    28
!quot;#$%"'()*+),%(-).(%(quot;-&/)0(1/*$%)
                                                                   /
      Behavioral                                                         Eyetracking                            Data Mining/Analysis
                                                                                                                A/B (Live) Testing

                                                                   Usability Benchmarking (in lab)



                                                                                            /
           Data Source
                                  Usability Lab Studies                                          Online User Experience Assessments
                                                                                                 (“Vividence-like” studies)


                                  Ethnographic Field Studies
                         mix




                                                                    Diary/Camera Study
                                                                    Message Board Mining
                                  Participatory Design              Customer feedback via email
                                  Focus Groups                            Desirability studies                  Intercept Surveys
      Attitudinal                 Phone Interviews                        Cardsorting                           Email Surveys

                                                                               mix
                                                                         Approach
                               Qualitative (direct)                                                     Quantitative (indirect)
                                Key for Context of Product Use during data collection
                                  Natural use of product                             De-contextualized / not using product
  © 2008 Christian Rohrer         Scripted (often lab-based) use of product          Combination / hybrid
                                                                                                                                 20


A LOVELY USER RESEARCH STRAW MAN
                                                                    28
The data that
drive our decisions




              29
The data that
drive our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           29
The data that
drive our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           29
The data that
drive our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           29
The data that
drive our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           29
The data that
drive our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           29
The data that
drive our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           29
Common queries
can drive task analysis




               30
Common queries
can drive task analysis
                      “Can you find a map of
                      the campus?”

                      “What study abroad
                      options are available to
                      students?”

                      “When is the last home
                      football game of the
                      season?”



               30
Query data
can augment
personas




              31
Query data
can augment
personas




              31
Query data
can augment
personas


 “What Steven
 Searches” added to
 existing persona
 (from Adaptive Path)
                        31
This is not statistics




               32
This is not statistics
This is not difficult




               32
This is not statistics
This is not difficult
This is very useful




               32
Systems can help us
objectify the
subjective


              33
Subjective
                      evaluations...




Systems can help us
objectify the
subjective


              33
Subjective
                      evaluations...


                                 ...lead to
Systems can help us             objective
                                decisions
objectify the
subjective


              33
Integrating through
shared goals




              34
What we covered
★ Quick intro
★ SSA from the Bottom Up
★ SSA from the Top Down
★ Putting them together

             35
Some day my book
will come...
Search Analytics for Your Site:
Conversations with Your Customers

Louis Rosenfeld & Marko Hurst
Rosenfeld Media, 2009 (?)

rosenfeldmedia.com/books/searchanalytics


                            36
Until then...

Louis Rosenfeld
457 Third Street, #4R
Brooklyn, NY 11215 USA

lou@louisrosenfeld.com
www.louisrosenfeld.com
www.rosenfeldmedia.com
Twitter: louisrosenfeld, rosenfeldmedia


                              37

More Related Content

Similar to Improving Findability through Site Search Analytics

The (poor) State of Google Analytics Data
The (poor) State of Google Analytics DataThe (poor) State of Google Analytics Data
The (poor) State of Google Analytics DataBrian Clifton
 
lec01-IntroductionToDataMining.pptx
lec01-IntroductionToDataMining.pptxlec01-IntroductionToDataMining.pptx
lec01-IntroductionToDataMining.pptxAmjadAlDgour
 
Increasing reporting value with statistics
Increasing reporting value with statisticsIncreasing reporting value with statistics
Increasing reporting value with statisticsvraopolisetti
 
Data preparation and processing chapter 2
Data preparation and processing chapter  2Data preparation and processing chapter  2
Data preparation and processing chapter 2Mahmoud Alfarra
 
Next Generation Fraud Solutions using Neo4j
Next Generation Fraud Solutions using Neo4jNext Generation Fraud Solutions using Neo4j
Next Generation Fraud Solutions using Neo4jNeo4j
 
Keyword Research in Autopilot by Google Spreadsheet Macros
Keyword Research in Autopilot by Google Spreadsheet MacrosKeyword Research in Autopilot by Google Spreadsheet Macros
Keyword Research in Autopilot by Google Spreadsheet MacrosYiğit Konur
 
Barga Data Science lecture 1
Barga Data Science lecture 1Barga Data Science lecture 1
Barga Data Science lecture 1Roger Barga
 
Site Search Analytics in a Nutshell
Site Search Analytics in a NutshellSite Search Analytics in a Nutshell
Site Search Analytics in a NutshellLouis Rosenfeld
 
Louis Rosenfeld: Nettstedssøk i et nøtteskall (Webdagene 2013)
Louis Rosenfeld: Nettstedssøk i et nøtteskall (Webdagene 2013)Louis Rosenfeld: Nettstedssøk i et nøtteskall (Webdagene 2013)
Louis Rosenfeld: Nettstedssøk i et nøtteskall (Webdagene 2013)webdagene
 
Regina Food Summit - Data for Good
Regina Food Summit - Data for GoodRegina Food Summit - Data for Good
Regina Food Summit - Data for GoodData For Good Regina
 
What are the odds of making that number risk analysis with crystal ball - O...
What are the odds of making that number   risk analysis with crystal ball - O...What are the odds of making that number   risk analysis with crystal ball - O...
What are the odds of making that number risk analysis with crystal ball - O...p6academy
 
SEO in the Shed - Chris Green - The SERPs Whisper: How to Learn All You Need ...
SEO in the Shed - Chris Green - The SERPs Whisper: How to Learn All You Need ...SEO in the Shed - Chris Green - The SERPs Whisper: How to Learn All You Need ...
SEO in the Shed - Chris Green - The SERPs Whisper: How to Learn All You Need ...Pi Datametrics
 
Conversion Rate Optimisation Presentation
Conversion Rate Optimisation PresentationConversion Rate Optimisation Presentation
Conversion Rate Optimisation PresentationMadhouse Associates
 
introDMintroDMintroDMintroDMintroDMintroDM.ppt
introDMintroDMintroDMintroDMintroDMintroDM.pptintroDMintroDMintroDMintroDMintroDMintroDM.ppt
introDMintroDMintroDMintroDMintroDMintroDM.pptDEEPAK948083
 
The state-of-dark-data-report
The state-of-dark-data-reportThe state-of-dark-data-report
The state-of-dark-data-reportVaibhav Agarwal
 

Similar to Improving Findability through Site Search Analytics (20)

The (poor) State of Google Analytics Data
The (poor) State of Google Analytics DataThe (poor) State of Google Analytics Data
The (poor) State of Google Analytics Data
 
lec01-IntroductionToDataMining.pptx
lec01-IntroductionToDataMining.pptxlec01-IntroductionToDataMining.pptx
lec01-IntroductionToDataMining.pptx
 
Datamining
DataminingDatamining
Datamining
 
Increasing reporting value with statistics
Increasing reporting value with statisticsIncreasing reporting value with statistics
Increasing reporting value with statistics
 
Data preparation and processing chapter 2
Data preparation and processing chapter  2Data preparation and processing chapter  2
Data preparation and processing chapter 2
 
Next Generation Fraud Solutions using Neo4j
Next Generation Fraud Solutions using Neo4jNext Generation Fraud Solutions using Neo4j
Next Generation Fraud Solutions using Neo4j
 
Keyword Research in Autopilot by Google Spreadsheet Macros
Keyword Research in Autopilot by Google Spreadsheet MacrosKeyword Research in Autopilot by Google Spreadsheet Macros
Keyword Research in Autopilot by Google Spreadsheet Macros
 
Barga Data Science lecture 1
Barga Data Science lecture 1Barga Data Science lecture 1
Barga Data Science lecture 1
 
Site Search Analytics in a Nutshell
Site Search Analytics in a NutshellSite Search Analytics in a Nutshell
Site Search Analytics in a Nutshell
 
Louis Rosenfeld: Nettstedssøk i et nøtteskall (Webdagene 2013)
Louis Rosenfeld: Nettstedssøk i et nøtteskall (Webdagene 2013)Louis Rosenfeld: Nettstedssøk i et nøtteskall (Webdagene 2013)
Louis Rosenfeld: Nettstedssøk i et nøtteskall (Webdagene 2013)
 
Regina Food Summit - Data for Good
Regina Food Summit - Data for GoodRegina Food Summit - Data for Good
Regina Food Summit - Data for Good
 
A6 big data_in_the_cloud
A6 big data_in_the_cloudA6 big data_in_the_cloud
A6 big data_in_the_cloud
 
Piano rubyslava final
Piano rubyslava finalPiano rubyslava final
Piano rubyslava final
 
What are the odds of making that number risk analysis with crystal ball - O...
What are the odds of making that number   risk analysis with crystal ball - O...What are the odds of making that number   risk analysis with crystal ball - O...
What are the odds of making that number risk analysis with crystal ball - O...
 
SEO in the Shed - Chris Green - The SERPs Whisper: How to Learn All You Need ...
SEO in the Shed - Chris Green - The SERPs Whisper: How to Learn All You Need ...SEO in the Shed - Chris Green - The SERPs Whisper: How to Learn All You Need ...
SEO in the Shed - Chris Green - The SERPs Whisper: How to Learn All You Need ...
 
Conversion Rate Optimisation Presentation
Conversion Rate Optimisation PresentationConversion Rate Optimisation Presentation
Conversion Rate Optimisation Presentation
 
introDMintroDMintroDMintroDMintroDMintroDM.ppt
introDMintroDMintroDMintroDMintroDMintroDM.pptintroDMintroDMintroDMintroDMintroDMintroDM.ppt
introDMintroDMintroDMintroDMintroDMintroDM.ppt
 
introDM.ppt
introDM.pptintroDM.ppt
introDM.ppt
 
The Art of Estimating - Andy Nolan
The Art of Estimating - Andy NolanThe Art of Estimating - Andy Nolan
The Art of Estimating - Andy Nolan
 
The state-of-dark-data-report
The state-of-dark-data-reportThe state-of-dark-data-report
The state-of-dark-data-report
 

More from Louis Rosenfeld

Information Architecture for Truth
Information Architecture for TruthInformation Architecture for Truth
Information Architecture for TruthLouis Rosenfeld
 
Falling in and out and in love with Information Architecture
Falling in and out and in love with Information ArchitectureFalling in and out and in love with Information Architecture
Falling in and out and in love with Information ArchitectureLouis Rosenfeld
 
Redesign Must Die (updated Feb 2014)
Redesign Must Die (updated Feb 2014)Redesign Must Die (updated Feb 2014)
Redesign Must Die (updated Feb 2014)Louis Rosenfeld
 
8 Information Architecture Better Practices
8 Information Architecture Better Practices8 Information Architecture Better Practices
8 Information Architecture Better PracticesLouis Rosenfeld
 
Search Analytics for Content Strategists
Search Analytics for Content StrategistsSearch Analytics for Content Strategists
Search Analytics for Content StrategistsLouis Rosenfeld
 
Closing the Findability Gap: 8 better practices from information architecture
Closing the Findability Gap: 8 better practices from information architectureClosing the Findability Gap: 8 better practices from information architecture
Closing the Findability Gap: 8 better practices from information architectureLouis Rosenfeld
 
Site search analytics workshop presentation
Site search analytics workshop presentationSite search analytics workshop presentation
Site search analytics workshop presentationLouis Rosenfeld
 
Design to Refine: Developing a tunable information architecture
Design to Refine: Developing a tunable information architectureDesign to Refine: Developing a tunable information architecture
Design to Refine: Developing a tunable information architectureLouis Rosenfeld
 
Adaptable Information Workshop slides
Adaptable Information Workshop slidesAdaptable Information Workshop slides
Adaptable Information Workshop slidesLouis Rosenfeld
 
Is there such a thing as a good business model for publishing these days?
Is there such a thing as a good business model  for publishing these days?Is there such a thing as a good business model  for publishing these days?
Is there such a thing as a good business model for publishing these days?Louis Rosenfeld
 
User Experience + Publishing
User Experience + PublishingUser Experience + Publishing
User Experience + PublishingLouis Rosenfeld
 
Marrying Web Analytics and User Experience
Marrying Web Analytics and User ExperienceMarrying Web Analytics and User Experience
Marrying Web Analytics and User ExperienceLouis Rosenfeld
 
Site Search Analytics: Conversations with your customers
Site Search Analytics:  Conversations with your customersSite Search Analytics:  Conversations with your customers
Site Search Analytics: Conversations with your customersLouis Rosenfeld
 
PhillyCHI Site Search Analytics presentation (April 2, 2008)
PhillyCHI Site Search Analytics presentation (April 2, 2008)PhillyCHI Site Search Analytics presentation (April 2, 2008)
PhillyCHI Site Search Analytics presentation (April 2, 2008)Louis Rosenfeld
 
Site Search Analytics Workshop Presentation
Site Search Analytics Workshop PresentationSite Search Analytics Workshop Presentation
Site Search Analytics Workshop PresentationLouis Rosenfeld
 
Eating Our Own Dog Food: Using UX Methods to Build a UX Business
Eating Our Own Dog Food:  Using UX Methods to Build a UX BusinessEating Our Own Dog Food:  Using UX Methods to Build a UX Business
Eating Our Own Dog Food: Using UX Methods to Build a UX BusinessLouis Rosenfeld
 

More from Louis Rosenfeld (20)

Information Architecture for Truth
Information Architecture for TruthInformation Architecture for Truth
Information Architecture for Truth
 
Falling in and out and in love with Information Architecture
Falling in and out and in love with Information ArchitectureFalling in and out and in love with Information Architecture
Falling in and out and in love with Information Architecture
 
Redesign Must Die (updated Feb 2014)
Redesign Must Die (updated Feb 2014)Redesign Must Die (updated Feb 2014)
Redesign Must Die (updated Feb 2014)
 
8 Information Architecture Better Practices
8 Information Architecture Better Practices8 Information Architecture Better Practices
8 Information Architecture Better Practices
 
Search Analytics for Content Strategists
Search Analytics for Content StrategistsSearch Analytics for Content Strategists
Search Analytics for Content Strategists
 
Closing the Findability Gap: 8 better practices from information architecture
Closing the Findability Gap: 8 better practices from information architectureClosing the Findability Gap: 8 better practices from information architecture
Closing the Findability Gap: 8 better practices from information architecture
 
Site search analytics workshop presentation
Site search analytics workshop presentationSite search analytics workshop presentation
Site search analytics workshop presentation
 
Design to Refine: Developing a tunable information architecture
Design to Refine: Developing a tunable information architectureDesign to Refine: Developing a tunable information architecture
Design to Refine: Developing a tunable information architecture
 
Redesign Must Die
Redesign Must DieRedesign Must Die
Redesign Must Die
 
Adaptable Information Workshop slides
Adaptable Information Workshop slidesAdaptable Information Workshop slides
Adaptable Information Workshop slides
 
Is there such a thing as a good business model for publishing these days?
Is there such a thing as a good business model  for publishing these days?Is there such a thing as a good business model  for publishing these days?
Is there such a thing as a good business model for publishing these days?
 
User Experience + Publishing
User Experience + PublishingUser Experience + Publishing
User Experience + Publishing
 
Marrying Web Analytics and User Experience
Marrying Web Analytics and User ExperienceMarrying Web Analytics and User Experience
Marrying Web Analytics and User Experience
 
Redesign Must Die
Redesign Must DieRedesign Must Die
Redesign Must Die
 
Site Search Analytics
Site Search AnalyticsSite Search Analytics
Site Search Analytics
 
Site Search Analytics: Conversations with your customers
Site Search Analytics:  Conversations with your customersSite Search Analytics:  Conversations with your customers
Site Search Analytics: Conversations with your customers
 
PhillyCHI Site Search Analytics presentation (April 2, 2008)
PhillyCHI Site Search Analytics presentation (April 2, 2008)PhillyCHI Site Search Analytics presentation (April 2, 2008)
PhillyCHI Site Search Analytics presentation (April 2, 2008)
 
Site Search Analytics Workshop Presentation
Site Search Analytics Workshop PresentationSite Search Analytics Workshop Presentation
Site Search Analytics Workshop Presentation
 
CanUX Keynote
CanUX KeynoteCanUX Keynote
CanUX Keynote
 
Eating Our Own Dog Food: Using UX Methods to Build a UX Business
Eating Our Own Dog Food:  Using UX Methods to Build a UX BusinessEating Our Own Dog Food:  Using UX Methods to Build a UX Business
Eating Our Own Dog Food: Using UX Methods to Build a UX Business
 

Recently uploaded

Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 

Recently uploaded (20)

Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 

Improving Findability through Site Search Analytics

  • 1. Improving Findability through Site Search Analytics Louis Rosenfeld • May 12, 2009 ESS 2009 • New York, NY, USA 1
  • 2. What we’ll cover ★ Quick intro ★ SSA from the Bottom Up ★ SSA from the Top Down ★ Putting them together 2
  • 3. Quick Intro ★ Where search query data comes from ★ Our friend Zipf ★ Long tail, meet short head 3
  • 4. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxy stylesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1quot; 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip =XXX.XXX.X.104 HTTP/1.1quot; 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxy stylesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1quot; 200 9718 62 0.17 Sample query data (from Google Search Appliance) 4
  • 6. Zipf, textually: the power of the short head 6
  • 9. But can you get at your data?
  • 10. SSA from the Bottom Up ★ The basics: play and ask questions ★ Five things you should be doing 8
  • 11. Generic questions help you play with your data ★ What are the most frequent unique queries? ★ Are frequent queries retrieving quality results? ★ Click-through rates per frequent query? ★ Most frequently clicked result per query? ★ Which frequent queries retrieve zero results? ★ What are the referrer pages for frequent queries? ★ Which queries retrieve popular documents? ★ What interesting patterns emerge in general? 9
  • 12. Bottom Up SSA: Five things you should do 1. Cluster your data to get a better picture of metadata and content needs 2. Track for seasonality 3. Take failure further: beyond failed searches 4. Leverage your best bets 5. Don’t be satisfied with generic reports 10
  • 13. Hunting for metadata patterns CANDIDATE VALUES CANDIDATE ATTRIBUTES 11
  • 14. Hunting for metadata patterns CANDIDATE VALUES CANDIDATE ATTRIBUTES 11
  • 15. Hunting for metadata patterns CANDIDATE VALUES CANDIDATE ATTRIBUTES 11
  • 16. Hunting for metadata patterns CANDIDATE VALUES CANDIDATE ATTRIBUTES 11
  • 17. Hunting for metadata patterns CANDIDATE VALUES CANDIDATE ATTRIBUTES 11
  • 23. The When of search 14
  • 31. Analyzing data from the bottom up: play with the data, look for patterns, trends, and outliers
  • 32. Analyzing data from the bottom up: play with the data, look for patterns, trends, and outliers So what’s being measured?
  • 33. SSA from the Top Down ★ The basics: why are we here? ★ The hard part: what can we measure? 19
  • 34. First: why are we here? ★ Commerce ★ Lead Generation ★ Content/Media ★ Support/Self-Service 20
  • 35. First: why are we here? ★ Commerce ★ Lead Generation ★ Content/Media ★ Support/Self-Service Data supports metrics... but which metrics for search? 20
  • 37. Can we measure findability? Does measure mean monetize? 21
  • 38. Vanguard and the quantification of search Target Oct 3 Oct 10 Oct 16 Mean distance from 1st 3 13 7 5 Median distance from 1st 2 7 3 1 Count: Below 1st 47% 84% 62% 58% Count: Below 5th 12% 58% 38% 14% Count: Below 10th 7% 38% 10% 7% Precision – Strict 42% 15% 36% 39% Precision – Loose 71% 38% 53% 65% Precision – Permissive 96% 55% 72% 92% Quantification, not monetization 22
  • 39. Search-related metrics ★ Jeannine Bartlett’s SIX Metrics(tm) Framework ★ Lee Romero’s search metrics ★ Both here: http://bit.ly/1a2mzk Disconnect: analytics world of KPI vs. experiential world 23 of search
  • 40. Analyzing data the top down: start with metrics, benchmark and measure performance
  • 41. Analyzing data the top down: start with metrics, benchmark and measure performance But you can’t measure what you don’t know
  • 42. Putting it all together Top-down analysis Bottom-up analysis 25
  • 43. Putting it all together what Top-down analysis Bottom-up analysis 25
  • 44. Putting it all together what Top-down analysis Bottom-up analysis why 25
  • 45. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1quot; 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1quot; 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1quot; 200 9718 62 0.17 26
  • 46. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1quot; 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1quot; 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1quot; 200 9718 62 0.17 BU Q: “What are the most common queries?” 26
  • 47. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1quot; 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1quot; 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1quot; 200 9718 62 0.17 27
  • 48. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1quot; 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1quot; 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] quot;GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1quot; 200 9718 62 0.17 TD Q: “Are we converting license plate renewals?” 27
  • 49. !quot;#$%"'()*+),%(-).(%(quot;-&/)0(1/*$%) / Behavioral Eyetracking Data Mining/Analysis A/B (Live) Testing Usability Benchmarking (in lab) / Data Source Usability Lab Studies Online User Experience Assessments (“Vividence-like” studies) Ethnographic Field Studies mix Diary/Camera Study Message Board Mining Participatory Design Customer feedback via email Focus Groups Desirability studies Intercept Surveys Attitudinal Phone Interviews Cardsorting Email Surveys mix Approach Qualitative (direct) Quantitative (indirect) Key for Context of Product Use during data collection Natural use of product De-contextualized / not using product © 2008 Christian Rohrer Scripted (often lab-based) use of product Combination / hybrid 20 A LOVELY USER RESEARCH STRAW MAN 28
  • 50. !quot;#$%"'()*+),%(-).(%(quot;-&/)0(1/*$%) / Behavioral Eyetracking Data Mining/Analysis A/B (Live) Testing Usability Benchmarking (in lab) / Data Source Usability Lab Studies Online User Experience Assessments (“Vividence-like” studies) Ethnographic Field Studies mix Diary/Camera Study Message Board Mining Participatory Design Customer feedback via email Focus Groups Desirability studies Intercept Surveys Attitudinal Phone Interviews Cardsorting Email Surveys mix Approach Qualitative (direct) Quantitative (indirect) Key for Context of Product Use during data collection Natural use of product De-contextualized / not using product © 2008 Christian Rohrer Scripted (often lab-based) use of product Combination / hybrid 20 A LOVELY USER RESEARCH STRAW MAN 28
  • 51. The data that drive our decisions 29
  • 52. The data that drive our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 29
  • 53. The data that drive our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 29
  • 54. The data that drive our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 29
  • 55. The data that drive our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 29
  • 56. The data that drive our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 29
  • 57. The data that drive our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 29
  • 58. Common queries can drive task analysis 30
  • 59. Common queries can drive task analysis “Can you find a map of the campus?” “What study abroad options are available to students?” “When is the last home football game of the season?” 30
  • 62. Query data can augment personas “What Steven Searches” added to existing persona (from Adaptive Path) 31
  • 63. This is not statistics 32
  • 64. This is not statistics This is not difficult 32
  • 65. This is not statistics This is not difficult This is very useful 32
  • 66. Systems can help us objectify the subjective 33
  • 67. Subjective evaluations... Systems can help us objectify the subjective 33
  • 68. Subjective evaluations... ...lead to Systems can help us objective decisions objectify the subjective 33
  • 70. What we covered ★ Quick intro ★ SSA from the Bottom Up ★ SSA from the Top Down ★ Putting them together 35
  • 71. Some day my book will come... Search Analytics for Your Site: Conversations with Your Customers Louis Rosenfeld & Marko Hurst Rosenfeld Media, 2009 (?) rosenfeldmedia.com/books/searchanalytics 36
  • 72. Until then... Louis Rosenfeld 457 Third Street, #4R Brooklyn, NY 11215 USA lou@louisrosenfeld.com www.louisrosenfeld.com www.rosenfeldmedia.com Twitter: louisrosenfeld, rosenfeldmedia 37

Editor's Notes

  1. ...so does Excel
  2. “The center can not hold!” You’ll notice this isn’t a canned report This all means putting pressure on commercial analytics apps to change
  3. “The center can not hold!” You’ll notice this isn’t a canned report This all means putting pressure on commercial analytics apps to change
  4. “The center can not hold!” You’ll notice this isn’t a canned report This all means putting pressure on commercial analytics apps to change
  5. “The center can not hold!” You’ll notice this isn’t a canned report This all means putting pressure on commercial analytics apps to change
  6. BTW, Vanguard is now mining the Long Tail
  7. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  8. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  9. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  10. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  11. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  12. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  13. you can do this, regardless of how you feel about data note that it’s in Excel
  14. you can do this, regardless of how you feel about data note that it’s in Excel
  15. Rosenfeld Media is the publishing equivalent of the Slow Food movement