From the beginning, Flipboard has run its infrastructure on Amazon Web Services (AWS). Greg Scallan, Chief Architect at Flipboard says, “Our service currently runs 100% on AWS in multiple availability zones.” He adds, “We chose AWS because they were able to provide a majority of the solution we needed as we built our data center. Also, we appreciated the flexibility as we tried out various solutions to our business vision.” Flipboard has incorporated AWS services as follows:Amazon Elastic Compute Cloud (Amazon EC2) for all server side processing from handling end user requests to data crunching, including a good sized Hadoop infrastructureAmazon SimpleDB for operational dataAmazon Relational Database Service (Amazon RDS) for mission critical dataAmazon Simple Queue Service (Amazon SQS)Amazon CloudWatchAmazon Elastic Load Balancing (Amazon ELB)Amazon Simple Storage Service (Amazon S3) and Amazon CloudFront for static and dynamic content distributionAmazon Mechanical Turk To build their solution, the team used Python/Ruby AWS SDKs in their operational tools and administrative servers. Scallan explains, “We leverage the Java AWS SDKs for core engineering services. We utilize many different languages, choosing the language that best meets the requirements at hand.”Flipboard has many hundreds of Amazon EC2 instances running and that number is continually growing with their increasing user base and feature sets. Scallan says, “We were able to go from concept to delivered product in approximately six months with just a handful of engineers, in large part because of the ability to deploy within a working cloud infrastructure very quickly.” Scallan would give the following advice to other developers: “If you plan to deploy in the cloud, embrace and leverage it for what it has to offer, and do not try to shoehorn traditional data center deployment strategies. One of the major perks of being in the cloud is flexibility and leveraging that can do wonders for operational costs.”The Flipboard team is always investigating additional AWS solutions and is open to using additional products as their needs evolve.The team has most appreciated the flexibility AWS has offered for prototyping before their first release. Scallan says, “This truly helped us release a working product on a highly aggressive schedule in a market with many emerging players.”
Mr.Horiuchi, CTO of gumiInc, describes what the company was using for their infrastructure prior to signing on with Amazon Web Services (AWS): “We used to rent racks at a datacenter, and ran our service using one Web server and one database server.” He notes that the company looked for a new offering mainly because they faced difficulty meeting demand: “So much traffic was coming from the social apps we were running on a mixi [a Japanese social networking site] platform, but we did not have the physical IT resources to manage it.”One of the deciding factors in their choice to use AWS was the capability to easily add servers as traffic increases, or remove them if it decreases. Horiuchi says, “Using AWS, we do not have to be limited by the number of physical servers. For example, we can easily carry out distribution of traffic using memcached or tokyotyrant when the database load gets higher, and if traffic decreases, we can reduce costs by canceling those instances. Even if there is no one who understands our in-house infrastructure very well, we are able to carry out this process automatically using AWS.” The number of gumi Inc. users is still increasing, and they are able to handle the increase in traffic.“Now,” says Horiuchi, “we are operating all services on AWS, except the delivery of static images.” The company uses Amazon Elastic Compute Cloud (Amazon EC2), Amazon Relational Database Service (Amazon RDS) with the Multi-AZ deployment option, and Amazon Simple Storage Service (Amazon S3) for running social applications in a very efficient way that requires only a small number of people to manage the system. “In particular,” notes Horiuchi, “Amazon RDS is a really good solution to reduce the time and heavy-lifting involved in database management.” The company is using the following server spec—the most high-spec instances they can choose—for their social application:Load Balancer:Amazon Elastic Load BalancingApp Server:Amazon EC2CentOS 5.4 + Apache2.2.3 + mod_wsgi + Python2.5.4Instances: c1.xlarge High-CPU Extra Large Instance x 8Memcached Server:Amazon EC2CentOS 5.4 + memcachedInstances: m1.xlarge Standard Extra Large Instance x 1Tokyotyrant (key-value store):Amazon EC2CentOS 5.4 + tokyotyrantInstances: m1.xlarge Standard Extra Large Instance x 2DB Server:Amazon RDSMySQL 5.1.38Instances: Quadruple Extra Large DB Instance: 68 GB of memory, 26 ECUs (8 virtual cores with 3.25 ECUs each), 64-bit platform x 2 (Multi-AZ HA Option)Horiuchi says gumiInc will use AWS for future social applications. The company is particularly pleased that AWS allows them to quickly perform testing and evaluation, reducing time to market and development costs. Horiuchi offers an example: “We quickly inspected and tested several candidates when we examined the introduction of the key-value store [KVS] for an anti-load measure of our database.” They look forward to quickly building their services with additional solutions from AWS, especially KVS-type services.
80-90% cost savings compared to traditional hosting environment.Minimum of 99.9% service availability using Multi-Availability zones without increasing costs.Easy expansion into other geographical regions as neededFocus on improving products and applications rather than managing infrastructure.Customer Quote:“If we were to use the traditional on-premise datacenter, we would have spent $34 million dollars in hardware and maintenance expenses during the first two years. With AWS cloud, we met our reliability and performance objectives at a fraction of the cost.”Mr. Chun KangPrincipal Engineer, Visual Display Division
Relational Databases Redefined with AWS
Relational Databases Redefinedon AWS Joseph Ziegler Technical Evangelistzieglerj@amazon.com @jiyosub
Migration Backup and recoverySchema design PatchingQuery construction ConfigurationQuery optimization Software upgrades Storage upgrades Focus on the “buck” Frequent server upgrades Hardware crash Off load the “muck”
Source: Forrester Security planning License training Backup, recovery load and unload Script automationPerformance Installation, upgrade,and tuning patching, migration
ProductivityMost admin tasks are managed by RDS Provisioning Database backups Patching Performance management
Productivity Multiple databases per instance Standard user accounts Connect and query using common MySQL tools & drivers Tune engine parameters Import and export data using standard MySQL tools (mysqldump) Diagnostics Native MySQL replication SSL for encryption over the wire Monitor metrics Shell, super user or direct file system access (Think security!)
ProductivityNear zero administration Painless patching, automatic upgrades Cloudwatch monitoring, Metric alarms One click. High Availability.
ProductivityOne click. High availability with Multi-AZ Automated deployment across multiple AZs Synchronous replication from master to replica Automatic fail-over; replica promoted to master Test fail-over
PerformancePush-button scale, high performance Scale storage from 5Gb to 1Tb of storage Scale instance from small to 4XL (better I/O) Add Read Replicas with asynchronous replication Add ElastiCache for performance
PerformanceAmazon ElastiCache In-memory cache service Ideal front end to RDS for read-heavy applications Low latency
PerformanceRead Replicas Scale beyond capacity constraints of a single instance Asynchronous replication to replicas Perfect for read-heavy loads and reporting
PerformanceReplicas vs Caches Caches are good for low latency access over a limited dataset Replicas provide relatively less performance over a substantially larger dataset
PerformanceProvisioned IOPS Fast, predictable, and consistent I/O 1,000 IOPS to 10,000 IOPS 100GB to 1 TB
Case study6 months to deploy a real time, social magazine5 million users, 2 billion page flips per monthRun on EC2, RDS and ElastiCache. Multi-AZ.Better response time (milliseconds) and uptimeComplex queries on use and relevance data
Case studyOutgrew existing IT environmentElasticity to respond to peaks; reduce costsRun on EC2, RDS10 million visitors a dayReduced time to market via
Data ProtectionAutomatic snapshots Daily database back-ups (save up to 35 days) Transaction log backups every 5 minutes Everything up to final 5 minutes can be recovered
Data ProtectionAutomatic backups Automated point-in-time backups 1 day retention (extendable to 35 days) On by default. No additional charge.
Data ProtectionDB Security groups DB instances are isolated by default Control access by source IP address or EC2 security group Specify IP range: 0.0.0.0/0 discouraged!
Data ProtectionRDS in the Virtual Private Cloud Virtual network, bring your own IP addresses Complete virtual network control: public, private subnets Includes Multi-AZ (for MySQL and Oracle)
Data ProtectionEncryption via SSL RDS generates SSL certificates for each DB instance Encryption of data between client and DB instance
Samsung saved $34M Problem: • Needed to reduce IT costs and were looking to create a more flexible IT environment Solution: • AWS and RDS services. With every request, the application authenticates devices, delivers apps and content, and pushes notifications. Business Benefits: • Saved $34M in hardware and maintenance expenses, 85% less than running on-premises
BonusReserved instances One time, up front payment Reserve capacity for your DB instance Discounted hourly rate Low, medium and high utilization rates
BonusInstant Development Data Use RDS for Production and Development Leverage Backups as Test Data
BonusPromoting RDS Read Replicasto Primary Fast Sharding Instant Replication of Fully Provisioned Database
BonusFree Tier Run RDS free for 1 Year! MySQL, Oracle (BYOL), SQL Server Express