SlideShare a Scribd company logo
1 of 19
Amazon CloudSearch
& Future Search Architectures
Iain Fletcher
ifletcher@Searchtechnologies.com
The expert in the search space
Agenda
A brief introduction to Search Technologies
Search Application Architectures
Common challenges, emerging architectures

Our thoughts about Amazon CloudSearch

The expert in the search space
Search Technologies Overview
Ascot, UK
Karlsruhe, DE

Cincinnati, OH
Herndon, VA
San Diego, CA
San Jose, CR

• The leading IT Services company dedicated to
Enterprise Search & Search-based Applications
• Implementation, Consulting, Managed Services
• 120 employees and growing
• Independent, working with all of the leading
software vendors and open source alternatives
500+ Customers
Future Search Architectures

The expert in the search space
A Traditional Integrated Architecture
Integrated Search Engine

Content
Sources
Employee
Directory
CMS

File Share

Etc.

Connectors

Index Pipeline

Search
Index

UI

The year is 2013, but we still have:
Dissatisfaction with search results
Scalability issues (various)
High initial investment
Inflexibility in the model
Data integration issues…
The expert in the search space
Reasons for Change
Today’s leading “indexing engines” are very capable
The challenges to success almost always lie in
peripheral tasks. On the indexing side of the house,
this means:
Gathering and feeding data to the search engine
Preparing data for indexing (normalization, metadata
enrichment. Etc.)

The expert in the search space
Reasons for Change
The need to cope with continuous and rapid data
growth
Things usually break first on the data gathering side of
the house
E.g. The Government department and the 7TB File Server

The expert in the search space
A Traditional Integrated Architecture
Integrated Search Engine

Content
Sources
Employee
Directory
CMS

Connectors

Index Pipeline

Search
Index

UI

RE-INDEX

File Share

Etc.

The expert in the search space
A Better Search Architecture
Iterative
Development

Content
Sources
Employee
Directory
CMS

Etc.

Content
Processing

Connectors

Secure
xml
Cache

Search Engine
Index
Pipeline

Search
Index

RE-INDEX

Better operational agility

The expert in the search space
A Better Search Architecture
Iterative
Development

Content
Sources
Employee
Directory
CMS

Etc.

Content
Processing

Connectors

Secure
xml
Cache

Amazon CloudSearch
Index
Pipeline

Search
Index

RE-INDEX

Better operational agility
Much improved cost of
ownership
The expert in the search space
Let’s take a look at Amazon CloudSearch
If you’ve ever switched on an EC2 server, then
you can switch on a CloudSearch server
And of course, switch it off just as easily

Xml or JSON in, for indexing
RESTful / xml out, for searching
Has the bases covered in terms of core search
functionality

The expert in the search space
Amazon CloudSearch: Important Search Functions

Full-on Search Navigators
Results sorting by property
Plenty of relevancy tuning controls
Indexing speed is impressive
We’ve tested up to 500 docs/sec

Very long results lists can be served

The expert in the search space
Cost Effective & Flexible
The usual benefits of AWS
Only use what you need
Elastic scalability

Maintenance inclusive
Compare that to the cost of a Service Level Agreement for
an on-premise alternative
Then add the labour costs….

From less than £50 per month

The expert in the search space
What do you need to get started
Repository Connectors
Capture content
Convert to xml
XSLT. Etc.

Content Processing
Java code, Groovy. Etc.
A framework product, E.g. Aspire
Commercial software to suit the
purpose

Amazon S3
Atlassian Confluence
Documentum
eRoom
Secure File Systems
IBM Connections
JIVE
Lotus Notes
RDBMS
SharePoint 2010, 2013
SocialCast
TeamForge
Web Crawling

A User Interface
The expert in the search space
Search-specific UI/UX builders are available
The expert in the search space
What Applications Suit Amazon CloudSearch?

In principle any
CloudSearch is an option that everyone should
consider for search-based applications
Especially where elastic scalability is an advantage
Or where the data already lives in the cloud,
including S3 buckets

The expert in the search space
To conclude
If you like Amazon Web Services, then you’ll like
CloudSearch for the same reasons
It is a full-function indexing engine, ideally suited to
agile, flexible search architectures
If you are looking for a search-based application, it
should be on your shortlist
For many applications, CloudSearch will prove to be
highly cost effective
The expert in the search space
My email:

ifletcher@searchtechnologies.com

Useful URLs:
Connectors for Cloudsearch
www.searchtechnologies.com/cloudsearch-data-connectors.html

Aspire Content Processing Framework
www.searchtechnologies.com/aspire.html

A Java API for Amazon Cloudsearch
www.searchtechnologies.com/java-api-amazon-cloudsearch.html

Enterprise Search Surveys
www.searchtechnologies.com/enterprise-search-surveys.html

Blog: Searching Wikipedia with Amazon CloudSearch
www.searchtechnologies.com/cloudsearch-data-connectors.html

White Paper: A Big Data Architecture for Search
www.searchtechnologies.com/big-data-enterprise-search-architecture.html
The expert in the search space

More Related Content

More from Search Technologies

The Evolution of Search and Big Data
The Evolution of Search and Big DataThe Evolution of Search and Big Data
The Evolution of Search and Big DataSearch Technologies
 
Enterprise Search Summit Keynote: A Big Data Architecture for Search
Enterprise Search Summit Keynote: A Big Data Architecture for SearchEnterprise Search Summit Keynote: A Big Data Architecture for Search
Enterprise Search Summit Keynote: A Big Data Architecture for SearchSearch Technologies
 
Advanced Query Parsing Techniques
Advanced Query Parsing TechniquesAdvanced Query Parsing Techniques
Advanced Query Parsing TechniquesSearch Technologies
 
The things you need to know about SharePoint 2013 Search
The things you need to know about SharePoint 2013 SearchThe things you need to know about SharePoint 2013 Search
The things you need to know about SharePoint 2013 SearchSearch Technologies
 
Enterprise Search Best Practices Webinar 4.2013
Enterprise Search Best Practices Webinar 4.2013Enterprise Search Best Practices Webinar 4.2013
Enterprise Search Best Practices Webinar 4.2013Search Technologies
 

More from Search Technologies (7)

The Evolution of Search and Big Data
The Evolution of Search and Big DataThe Evolution of Search and Big Data
The Evolution of Search and Big Data
 
Enterprise Search Summit Keynote: A Big Data Architecture for Search
Enterprise Search Summit Keynote: A Big Data Architecture for SearchEnterprise Search Summit Keynote: A Big Data Architecture for Search
Enterprise Search Summit Keynote: A Big Data Architecture for Search
 
Advanced Query Parsing Techniques
Advanced Query Parsing TechniquesAdvanced Query Parsing Techniques
Advanced Query Parsing Techniques
 
Wikipedia Cloud Search Webinar
Wikipedia Cloud Search WebinarWikipedia Cloud Search Webinar
Wikipedia Cloud Search Webinar
 
The things you need to know about SharePoint 2013 Search
The things you need to know about SharePoint 2013 SearchThe things you need to know about SharePoint 2013 Search
The things you need to know about SharePoint 2013 Search
 
Enterprise Search Best Practices Webinar 4.2013
Enterprise Search Best Practices Webinar 4.2013Enterprise Search Best Practices Webinar 4.2013
Enterprise Search Best Practices Webinar 4.2013
 
Advanced Relevancy Ranking
Advanced Relevancy RankingAdvanced Relevancy Ranking
Advanced Relevancy Ranking
 

Recently uploaded

Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Google AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGGoogle AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGSujit Pal
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Paola De la Torre
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 

Recently uploaded (20)

Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Google AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGGoogle AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAG
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 

Amazon CloudSearch & Future Search Architectures

  • 1. Amazon CloudSearch & Future Search Architectures Iain Fletcher ifletcher@Searchtechnologies.com The expert in the search space
  • 2. Agenda A brief introduction to Search Technologies Search Application Architectures Common challenges, emerging architectures Our thoughts about Amazon CloudSearch The expert in the search space
  • 3. Search Technologies Overview Ascot, UK Karlsruhe, DE Cincinnati, OH Herndon, VA San Diego, CA San Jose, CR • The leading IT Services company dedicated to Enterprise Search & Search-based Applications • Implementation, Consulting, Managed Services • 120 employees and growing • Independent, working with all of the leading software vendors and open source alternatives
  • 5. Future Search Architectures The expert in the search space
  • 6. A Traditional Integrated Architecture Integrated Search Engine Content Sources Employee Directory CMS File Share Etc. Connectors Index Pipeline Search Index UI The year is 2013, but we still have: Dissatisfaction with search results Scalability issues (various) High initial investment Inflexibility in the model Data integration issues… The expert in the search space
  • 7. Reasons for Change Today’s leading “indexing engines” are very capable The challenges to success almost always lie in peripheral tasks. On the indexing side of the house, this means: Gathering and feeding data to the search engine Preparing data for indexing (normalization, metadata enrichment. Etc.) The expert in the search space
  • 8. Reasons for Change The need to cope with continuous and rapid data growth Things usually break first on the data gathering side of the house E.g. The Government department and the 7TB File Server The expert in the search space
  • 9. A Traditional Integrated Architecture Integrated Search Engine Content Sources Employee Directory CMS Connectors Index Pipeline Search Index UI RE-INDEX File Share Etc. The expert in the search space
  • 10. A Better Search Architecture Iterative Development Content Sources Employee Directory CMS Etc. Content Processing Connectors Secure xml Cache Search Engine Index Pipeline Search Index RE-INDEX Better operational agility The expert in the search space
  • 11. A Better Search Architecture Iterative Development Content Sources Employee Directory CMS Etc. Content Processing Connectors Secure xml Cache Amazon CloudSearch Index Pipeline Search Index RE-INDEX Better operational agility Much improved cost of ownership The expert in the search space
  • 12. Let’s take a look at Amazon CloudSearch If you’ve ever switched on an EC2 server, then you can switch on a CloudSearch server And of course, switch it off just as easily Xml or JSON in, for indexing RESTful / xml out, for searching Has the bases covered in terms of core search functionality The expert in the search space
  • 13. Amazon CloudSearch: Important Search Functions Full-on Search Navigators Results sorting by property Plenty of relevancy tuning controls Indexing speed is impressive We’ve tested up to 500 docs/sec Very long results lists can be served The expert in the search space
  • 14. Cost Effective & Flexible The usual benefits of AWS Only use what you need Elastic scalability Maintenance inclusive Compare that to the cost of a Service Level Agreement for an on-premise alternative Then add the labour costs…. From less than £50 per month The expert in the search space
  • 15. What do you need to get started Repository Connectors Capture content Convert to xml XSLT. Etc. Content Processing Java code, Groovy. Etc. A framework product, E.g. Aspire Commercial software to suit the purpose Amazon S3 Atlassian Confluence Documentum eRoom Secure File Systems IBM Connections JIVE Lotus Notes RDBMS SharePoint 2010, 2013 SocialCast TeamForge Web Crawling A User Interface The expert in the search space
  • 16. Search-specific UI/UX builders are available The expert in the search space
  • 17. What Applications Suit Amazon CloudSearch? In principle any CloudSearch is an option that everyone should consider for search-based applications Especially where elastic scalability is an advantage Or where the data already lives in the cloud, including S3 buckets The expert in the search space
  • 18. To conclude If you like Amazon Web Services, then you’ll like CloudSearch for the same reasons It is a full-function indexing engine, ideally suited to agile, flexible search architectures If you are looking for a search-based application, it should be on your shortlist For many applications, CloudSearch will prove to be highly cost effective The expert in the search space
  • 19. My email: ifletcher@searchtechnologies.com Useful URLs: Connectors for Cloudsearch www.searchtechnologies.com/cloudsearch-data-connectors.html Aspire Content Processing Framework www.searchtechnologies.com/aspire.html A Java API for Amazon Cloudsearch www.searchtechnologies.com/java-api-amazon-cloudsearch.html Enterprise Search Surveys www.searchtechnologies.com/enterprise-search-surveys.html Blog: Searching Wikipedia with Amazon CloudSearch www.searchtechnologies.com/cloudsearch-data-connectors.html White Paper: A Big Data Architecture for Search www.searchtechnologies.com/big-data-enterprise-search-architecture.html The expert in the search space

Editor's Notes

  1. Here’s the agenda for my part of the presentation.After a brief introduction to Search Technologies, I have some thoughts to share with you about how search architectures are evolving, and why.That will set the scene for taking a look at Amazon CloudSearch.