1. The Evolution of
Web and Big Data
Edward J. Yoon

2. Who Am I
• Edward J. Yoon
– @eddieyoon
• Founder of Apache Hama
• PMC member of Apache BigTop
• Oracle Employee

3. Early era of Web
Google
• 2003: GFS
• 2004: MapReduce OSS
• 2005: SawZall • 2005: Hadoop
• 2006: BigTable HDFS
MapReduce
• 2006: Pig
• 2007: Hive
HBase

4. Google?
• World best “Full-text search engine”
• In 2003,
– 10,000+ Servers
– 4+ billion Documents
– 300+ Million Images

5. Hadoop 1.0
• HDFS + MapReduce
– And Pig. Hive, Hbase, Mahout

7. The New era of Web
Google OSS
• 2010: Pregel • 2010: Hama
Dremel Twitter
• 2012: Spanner Storm
• 2011: YARN
Giraph
• 2012: Drill

8. MR vs. Alternatives

9. YARN?
• Job scheduling and cluster resource
management

10. Future of CDH4 and Hadoop
• CDH4 will be based on 0.23.x or later
• 0.23.0 doesn’t include Map/Reduce 1.0
– Storm, Giraph, Hama, Spark, MPI, GraphLab