Hadoop basics

Hadoop at Yahoo! Owen O’Malley Yahoo!, Grid Team [email_address]

Who Am I? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Problem ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Solution ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Hadoop Timeline ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Commodity Hardware Cluster ,[object Object],[object Object],[object Object],[object Object],[object Object]

Distributed File System ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Block Placement ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Correctness ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Map/Reduce ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Map/Reduce features ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Word Count Example ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Example: Word Count Mapper ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Example: Word Count Reducer ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Word Count in Python ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Why Yahoo! is investing in Hadoop ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Running the Production WebMap ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Inverting the Webmap ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Finding Duplicate Web Pages ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],T ? ? ? ? T ? ? ? T ? ? T ? T

Finding Duplicates ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Research Clusters ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Hadoop clusters ,[object Object],[object Object],[object Object],[object Object]

NY Times ,[object Object],[object Object],[object Object],[object Object],[object Object],Published 1892, copyright New York Times

Terabyte Sort Benchmark ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Hadoop Community ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Who Uses Hadoop? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

What’s Next? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Q&A ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Hadoop basics

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Hadoop basics

Similar to Hadoop basics (20)

Recently uploaded

Recently uploaded (20)

Hadoop basics