Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big data Overview

13,617 views

Published on

An introduction to big data.
What's big data, why we'd want it , how is it applicable to CSPs, short intro to Hadoop

(some of the info is in the slide notes)

Published in: Software
  • Be the first to comment

Big data Overview

  1. 1. BIG DATA Arnon Rotem-Gal-Oz Director of Technology Research, Amdocs The blind men and the elephant. Poem by John Godfrey Saxe (Cartoon originally copyrighted by the authors; G. Renee Guzlas, artists http://www.nature.com/ki/journal/v62/n5/fig_tab/4493262f1.html
  2. 2. 1880 US Census
  3. 3. Hollerith Tabulating Machine Hollerith photos by Martin Wichary : http://www.flickr.com/photos/mwichary/4358926764/in/photostream/
  4. 4. ource: Silicon Angle http://siliconangle.com/blog/2013/11/13/how-big-is-big-data-really/ Big data happens when the data you have to process is bigger than what you can process in the given time with current technologies
  5. 5. Myth: Big data = keep all data Source: Big Data Public Private Forum : http://www.big- project.eu/sites/default/files/D2.2.1_First%20draft%20of%20Technical%20white%20papers_FINAL_v1.01_ 0.pdf
  6. 6. Source: Big Data Public Private Forum : http://www.big- project.eu/sites/default/files/D2.2.1_First%20draft%20of%20Technical%20white%20papers_FINAL_v1.01_ 0.pdf
  7. 7. Some Telco Numbers Source: Wikipedia http://upload.wikimedia.org/wikipedia/commons/5/50/Telephone_operators,_1952.jpg
  8. 8. So, what do we do with all this data? Wikipedia http://upload.wikimedia.org/wikipedia/commons/0/06/UPS_Truck.jpg
  9. 9. It’s the insights, stupid* * With apologies to Bill Clinton
  10. 10. ource: Silicon Angle http://siliconangle.com/blog/2013/11/13/how-big-is-big-data-really/ Big data analytics is when sample = N • Big data happens when the data you have to process is bigger than what you can process in the given time with current technologies
  11. 11. “My daughter got this in the mail!, She’s still in high school, and you’re sending her coupons for baby clothes and cribs? Are you trying to encourage her to get pregnant?” Source: Forbes http://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her- father-did/
  12. 12. We need to watch out that Analytics won’t get too creepy
  13. 13. When people hear big data they think fast data Source: Steve Jones Cap Gemini http://www.no.capgemini.com/node/778541
  14. 14. Subscribers Collect & Filter Correlate (simplified) Network proactive care flow Account Event Store Identify & Predict Network Failures Reimburse VIPs Prioritize technicians Identify impact on high valued Accounts
  15. 15. ource: Silicon Angle http://siliconangle.com/blog/2013/11/13/how-big-is-big-data-really/ Big data is when we can handle data fast enough to make a difference • Big data happens when the data you have to process is bigger than what you can process in the given time with current technologies • Big data analytics is when sample = N
  16. 16. Technology space
  17. 17. The Elephant in the room
  18. 18. Hadoop Stack Map/R educe HDFS HBase Pig Hive Zoo Keeper Oozie Mahout Giraph
  19. 19. Schema on read
  20. 20. Move data to computation
  21. 21. Maybe we should rethink moving data to computation… Source : http://my-inner-voice.blogspot.co.il/2012/06/haddop-101-paper-by-miha-ahronovitz-and.html
  22. 22. Map/reduce Source: http://www.bodhtree.com/blog/2012/10/18/ever-wondered-what-happens-between-map-and-reduce/
  23. 23. Customer Segmentation First name Last name ARPU Age Device Country … Mr. Smith 100 22 iPhone 5s,White USA John Doe 87 42 Samsung Galaxy S5,Gold France Lady In Red 105 21 Samsung Note 3, White UK … Uluru, Australia by Stuart Edwards (cc) http://en.wikipedia.org/wiki/Uluru#mediaviewer/File:Uluru_Panorama.jpg
  24. 24. K-Means ARPU Age Source : http://pypr.sourceforge.net/kmeans.html
  25. 25. K=3ARPU Age ARPU Age Source : http://pypr.sourceforge.net/kmeans.html
  26. 26. New paradigms Map/R educe HDFS HBase Pig Hive Zoo Keeper Oozie Mahout Giraph
  27. 27. New Paradigms Map/R educe HDFS HBase Pig Hive Zoo Keeper Oozie Mahout YARN Giraph
  28. 28. New Paradigms Map/R educe HDFS HBase Pig Hive Zoo Keeper Oozie Mahout YARN Giraph Spark Storm Slider Flink Impala Tez Presto
  29. 29. Amdocs Analytics & Data Management Heritage 2013 • Proactive Care • TerraScale • Network optimization • Real time analytics platform • Single product catalog • BSS–OSS Integration • CRM-Billing Integration OSS Analytics Platform, 16 Analytics Patents • aLDM logical data model • Policy control Network Analytics CRM 2000 2008 AcquisitionsPortfolio
  30. 30. 34 Information Security Level 2 – Sensitive © 2014 – Proprietary and Confidential Information of Amdocs Touchpoints & Applications CRM Self Service E-MailPCRF SMS OtherWi-Fi OffloadCampaign Mng. • • • • • • • Operational Envelope & Platform Administration • Security Management • Configuration Management • Services Inventory • Performance Management • Fault Management • LoggerCollect & Ingest Transform & Enrich Aggregate & Correlate Drive Insight Close the Loop Machine Learn & Score Application-Ready Data and Analytics/ML Insights Entities and Profiles Detailed Data OSS Probes SocialRAN Inventory Usage & Charging CRM Real-Time & Batch Connectors Insight Platform Marketing Analytical Application Framework: Dashboards & Visualisation Decisioning Engine Dynamic Micro Segmentation Network Care Operations
  31. 31. ource: Silicon Angle http://siliconangle.com/blog/2013/11/13/how-big-is-big-data-really/ • Big data happens when the data you have to process is bigger than what you can process in the given time with current technologies • Big data analytics is when sample = N • Big data is when we can handle data fast enough to make a difference
  32. 32. Additional takeaways • CSPs have always been in the big data business – they just didn’t know it • Big data is not a panacea • Hadoop is shaping up as the big data OS – Though there are alternatives arriving from the cloud arena (mesos, kubernetes)
  33. 33. What we covered here is not even the tip of the iceberg Source: wikimedia http://commons.wikimedia.org/wiki/File:Iceberg.jpg
  34. 34. Arnon Rotem-Gal-Oz Director of Technology Research, Amdocs arnonrot@amdocs.com / arnon@rgoarchitects.com

×