Nice Docs Finish First - Designing Search Ranking for Fairness at Etsy: Presented by Fiona Condon, Etsy
•
0 likes•1,074 views
Lucene/Solr Revolution 2015
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (7)
Similar to Nice Docs Finish First - Designing Search Ranking for Fairness at Etsy: Presented by Fiona Condon, Etsy
Similar to Nice Docs Finish First - Designing Search Ranking for Fairness at Etsy: Presented by Fiona Condon, Etsy (20)
More from Lucidworks
More from Lucidworks (20)
Recently uploaded
Recently uploaded (20)
Nice Docs Finish First - Designing Search Ranking for Fairness at Etsy: Presented by Fiona Condon, Etsy
- 1. O C T O B E R 1 3 - 1 6 , 2 0 1 6 • A U S T I N , T X
- 2. Nice Docs Finish First: Designing Search Ranking for Fairness at Etsy Fiona Condon Senior Software Engineer, Etsy
- 3. 3 02
- 4. 4 02
- 5. 5 02
- 6. 6 02
- 7. 7 02 Etsy by the Numbers • 1.5 million active sellers • 21.7 million active buyers • 32 million listings for sale • Solr for listing search
- 8. 8 02 Etsy by the Numbers
- 13. 13 01 Our challenge To return the best listings
- 14. 14 01 Our challenge To return the most informative, honest, high-quality listings
- 15. 15 01 Our challenge To return a diverse, fresh mix of the most informative, honest, high-quality listings
- 16. 16 02
- 17. 17 01 Etsy Values We are a mindful, transparent, and humane business. We plan and build for the long term. We value craftsmanship in all we make. We believe fun should be part of everything we do. We keep it real, always.
- 18. 18 01 Don’t hate the player, change the game We keep it real, always.
- 19. 19 01 Don’t hate the player, change the game
- 20. 20 01 Don’t hate the player, change the game
- 21. 21 01 Don’t hate the player, change the game The count of the term in the document The inverse of the count of the term across all documents
- 22. 22 01 Don’t hate the player, change the game
- 23. 23 01 Don’t hate the player, change the game
- 24. 24 01 Don’t hate the player, change the game
- 25. 25 01 Don’t hate the player, change the game
- 26. 26 01 Don’t hate the player, change the game @Override public float tf(float freq) { return (freq > 0) ? 1.0f : 0.0f; }
- 27. 27 01 Don’t hate the player, change the game • Set TF to 1
- 28. 28 01 Don’t hate the player, change the game • Set TF to 1 • Choose the right fields to index
- 29. 29 01 Don’t hate the player, change the game
- 30. 30 01 Don’t hate the player, change the game
- 31. 31 01 Don’t hate the player, change the game • Set TF to 1 • Choose the right fields to index
- 32. 32 01 Crafting a quality signal We value craftsmanship in all we make
- 33. 33 01 Crafting a quality signal
- 34. 34 01 Crafting a quality signal
- 35. 35 01 Crafting a quality signal
- 36. 36 01 Crafting a quality signal
- 37. 37 01 Crafting a quality signal
- 38. 38 01 Crafting a quality signal • Avoid presentation bias
- 39. 39 01 Crafting a quality signal
- 40. 40 01 Crafting a quality signal
- 41. 41 01 Crafting a quality signal • Avoid presentation bias • Store outside the index
- 42. 42 01 Crafting a quality signal <fieldType name="listing_quality_file" keyField=“listing_id” defVal="0.5" stored="true" indexed="true" class="solr.ExternalFileField" valType=“float" />
- 43. 43 01 Crafting a quality signal • Avoid presentation bias • Store outside the index • Bootstrap
- 44. 44 01 Crafting a quality signal
- 45. 45 01 Freshness is fun We believe fun should be part of everything we do.
- 47. 47 01 Freshness is fun • Diversify by seller
- 48. 48 01 Freshness is fun public SearchResults diversify(SearchResults results) { SearchResults diversifiedResults; int nextWindow = diversityOptions.getWindow(); do { diversityOptions.window = nextWindow; diversifiedResults = shopDiversifier.diversify(results); DiversityStats shopDiversity = diversifiedResults.docs.stream() .collect(DiversityStatsCalculator.collector(ListingDoc::getShopId)).getStats(); // if the results are sufficiently diverse, we're done if (shopDiversity.getDiversityIndex() <= diversityOptions.progressive.getTargetDiversityIndex()) { break; } // otherwise, broaden the window and re-‐try nextWindow = Math.min( diversifiedResults.totalCount, Math.min(diversityOptions.getMaxWindow(), diversityOptions.window * 2) ); } while (diversityOptions.window < nextWindow); return diversifiedResults; }
- 49. 49 01 Freshness is fun . . . // if the list is sufficiently diverse, we're done if (shopDiversity.getDiversityIndex() <= diversityOptions.progressive.getTargetDiversityIndex()) { break; } // otherwise, broaden the window and re-‐try nextWindow = Math.min( diversifiedResults.totalCount, Math.min(diversityOptions.getMaxWindow(), diversityOptions.window * 2) ); . . .
- 52. 52 01 Freshness is fun • Diversify by seller • Recency boost
- 53. 53 01 Freshness is fun The inverse of the time elapsed between now and listing creation
- 55. 55 01 Evaluating for stability We plan and build for the long term.
- 56. 56 01 Evaluating for stability • Replayer • CL tool • Uses sampled request logs to “replay” real traffic • Accepts target hosts, duration, query rate • Programmatically filters or alters requests • Provides realistic stats on average/worst-case impact
- 58. 58 01 Evaluating for stability • RankDelta • Web UI • Allows user to specify query set, hosts and thrift params in PHP code • Provides high-level statistics about the results • Plus full result set deep-dive
- 60. 60 01 Evaluating for stability • In an ideal world…
- 61. 61 01 Evaluating for stability • In an ideal world… • Making trade-offs
- 62. 62 01 Communicating clearly We are a mindful, transparent, and humane business.
- 64. 64 01 Communicating clearly • Focus on the constants
- 65. 65 01 Communicating clearly • Focus on the constants • Provide a feedback loop
- 67. 67 01 Takeaways • Minor changes to the default scoring can be powerful • Handle quality contextually • Conscious diversity serves both searcher & searchee • Invest in a feedback loop on ranking changes • Be honest but keep it consistent
- 68. 68 02