AWS at Mendeley (London, September 27th 2011)
•
1 like•769 views
Mendeley uses Amazon Web Services (AWS) to power its research collaboration platform. It stores over 16 terabytes of research papers and citations uploaded by over 1 million researchers on AWS Simple Storage Service (S3). It uses AWS services like Elastic Beanstalk, CloudFront, and Elastic MapReduce to process and render previews of papers, serve search queries across multiple AWS resources, and perform machine learning recommendations at scale. The AWS architecture allows Mendeley to focus on its core product instead of managing infrastructure, while scaling elastically to support over 17 million papers and growing usage from researchers worldwide.
Recommended
Recommended
More Related Content
What's hot
What's hot (9)
Viewers also liked
Viewers also liked (17)
Similar to AWS at Mendeley (London, September 27th 2011)
Similar to AWS at Mendeley (London, September 27th 2011) (20)
More from Dan Harvey
Recently uploaded
Recently uploaded (20)
AWS at Mendeley (London, September 27th 2011)
- 1. Revolutionising the world of research with Amazon Web Services Dan Harvey Systems Architect
- 2. Mendeley helps researchers work smarter
- 3. Mendeley makes science more collaborative and transparent Sync between computers and peers around the world
- 4. 1 million researchers... Uploaded130 million citations 40 million unique 17 million papers 16 TB of data
- 5. Document Storage • Backed on to S3 • Previews for 16TB of PDFs?
- 6. PDF Previews Process Queue Load PDFs Elastic Beanstalk S3 Render to PDF Store PNG Serve via Cloud Front
- 7. System Overview S3 RD EM S R EC 2 EB S VPC
- 8. Article Search • Based on Solr (open source search) • 40GB index • Variable usage H-p Requests Day of the month
- 9. Solr Layout Solr Master EB S Inside VPC Outside VPC Solr Solr Solr Slave Slave Slave Elastic Search Load Balancer Queries
- 10. • Machine learning on Hadoop • Personalised article recommendations • Collaborative filtering based • Running on Elastic Map Reduce
- 11. Summary • Not all or nothing • Focus on your problem not “Undifferentiated heavy lifting” - Werner Vogels • Learn the building blocks AWS provide
- 12. Enjoy what you’ve seen? We’re hiring! Senior Java Engineers chat to me after or e-mail/tweet
- 13. Questions? @DanHarvey dan.harvey@mendeley.com
Editor's Notes
- Introduce myself\nAsk who knows about Mendeley\nMendeley are a start-up aiming to improve the world of research\n
- One of our two core products, Mendeley Desktop\nResearch can put research papers they read, annotate their thoughts and ideas\nUse them as citations inside Word, Openoffice, Latex\n
- Sync to Mendeley Web, where you can share notes and collaborate with other\nAlso allows us to produce statistics and trends on research speeding up the usual delay researchers get with feed back to their publications.\n
- A few numbers... we’ve got over 1 million research using the platform. They’ve uploaded over 130 million documents, which we’ve found around 40 million are unique. Of these 40 million we’ve got 17 million pdf articles users have uploaded, giving around 16TH of documents in total.\n\nSo quite a scaling and growth challenge!\n
- \n
- \n
- \n
- \n
- \n
- \n
- Hello how’s this looking...\n
- \n
- \n