Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Usama Fayyad talk in South Africa: From BigData to Data Science


Published on

Public talk by Barclays CDO Usama Fayyad in South Africa: both at University of Pretoria (GIBS) - Johannesburg and at Workshop17 in Capetown July 14-15, 2015

Published in: Data & Analytics
  • Login to see the comments

Usama Fayyad talk in South Africa: From BigData to Data Science

  1. 1. From BigData to Data Science: Predictive Analytics in a Changing Data Landscape Usama Fayyad, Ph.D. Group Chief Data Officer and CIO Risk, Finance, & Treasury Technology Barclays Twitter: @usamaf Overviews for South Africa July 15-16, 2015
  2. 2. Outline • Big Data all around us: the problems that Data Science is NOT addressing • The CDO role and Data Axioms • Some of the issues in BigData • Case studies • Context and sentiment analysis • A case-study on IOT • Yahoo! predictions at scale • Summary and conclusions
  3. 3. Why should banks worry about data? 100 years ago • Smaller • Much more local • Deep understanding of customer needs • Deep knowledge of customer Now • Digital, decline of the branch • Global • No face of the customer • New generation of millennials A big part of banking was knowing the customer intimately – personal relationships making KYC, risk modelling simple
  4. 4. Why data is important: Every customer interaction in an opportunity to capture data, learn & act An instant car loan product offering is displayed in the app Connie logs into BMB to check her balance. From cookies and browsing behaviour we know she is looking for a car loan CRM data is used to pre-calculate Connie’s borrowing limit for a car loan Based on multiple internal and external data sources and predictive models, we identify cross-sell opportunities Connie is offered a competitively priced ‘bespoke’ offer for car servicing / MOT Connie’s journey is enhanced based on previous multivariate testing results User experience is continuously, iteratively improved by capturing user interaction in real- time during every session Connie has a personalised journey based on pre-calculated limit for the car loan amount Data Capture/ Opportunity to learn Actions Customer Interaction Event (branch, telephony, digital, mobile, sales) Millions of customer interactions per day Customer Interaction CRM Predictive Analytics Multivariate Testing Targeted offers during browsing Product Discovery Cross-sell Measure feedback per session Every interaction is an opportunity to capture data, to learn and to act Big Data platform Imagine we could move back to this model 100 years ago – on demand, consumable, understandable information to build intimate relationships & an understanding of the customer
  5. 5. What matters in the age of analytics? 1. Being able to exploit all the data that is available • Not just what you have available • What you can acquire and use to enhance your actions 2. Proliferating analytics throughout the organization • Make every part of your business smarter • Actions and not just insights 3. Driving significant business value • Embedding analytics into every area of your business can significantly drive top line revenues and/or bottom line cost efficiencies
  6. 6. Data Fusion: Can we bring all data together for analysis & action? Customer and Client Interactions Central Data Fusion Engine Ingesting, persisting, processing and servicing in Real-time. Analysis TransactionsSocial Data Trade Application Logs Network Traffic RiskMarketingFinancial CrimeFraudCyber Security DaaS
  7. 7. • Big Data: is a mix of structured, semi-structured, and unstructured data • Typically breaks barriers for traditional RDB storage • Typically breaks limits of indexing by “rows” • Typically requires intensive pre-processing before each query to extract “some structure” – usually using Map-Reduce type operations • Above leads to “messy” situations with no standard recipes or architecture: hence the need for “data scientists” • Conduct “Data Expeditions” • Discovery and learning on the spot Why Big Data? A new term, with associated ‘Data Scientist’ positions A Data Scientist is someone who Knows a lot more software engineering than Statisticians & Knows a lot more Statistics than software engineers
  8. 8. The 4-V’s of ‘Big Data’ Big Data is Characterized by the 3-V’s: Volume: larger than “normal” – challenging to load/process • Expensive to do ETL • Expensive to figure out how to index and retrieve • Multiple dimensions that are “key” Velocity: Rate of arrival poses real-time constraints on what are typically “batch ETL” operations • If you fall behind catching up is extremely expensive (replicate very expensive systems) • Must keep up with rate and service queries on-the-fly Variety: Mix of data types and varying degrees of structure • Non-standard schema • Lots of BLOB’s and CLOB’s • DB queries don’t know what to do with semi-structured and unstructured data
  9. 9. 9 Invited talk – #ODSC, Boston– Copyright Usama Fayyad © 2015 Male, age 32 Lives in SF Lawyer Searched on from London last week Searched on: “Italian restaurant Palo Alto” Checks Yahoo! Mail daily via PC & Phone Has 25 IM Buddies, Moderates 3 Y! Groups, and hosts a 360 page viewed by 10k people Searched on: “Hillary Clinton” Clicked on Sony Plasma TV SS ad Registration Campaign Behavior Unknown Spends 10 hour/week On the internet Purchased Da Vinci Code from Amazon “Classic” Data: e.g. Yahoo! User DNA
  10. 10. 10 Invited talk – #ODSC, Boston– Copyright Usama Fayyad © 2015 Male, age 32 Lives in SF Lawyer Searched on from London last week Searched on: “Italian restaurant Palo Alto” Checks Yahoo! Mail daily via PC & Phone Has 25 IM Buddies, Moderates 3 Y! Groups, and hosts a 360 page viewed by 10k people Searched on: “Hillary Clinton” Clicked on Sony Plasma TV SS ad Spends 10 hour/week On the internet Purchased Da Vinci Code from Amazon How Data Explodes: really big Social Graph (FB) Likes & friends likes Professional netwk - reputation Web searches on this person, hobbies, work, locationMetaData on everything Blogs, publications, news, local papers, job info, accidents
  11. 11. The Distinction between “Classic Data” and “Big Data” is fast disappearing • Most real data sets nowadays come with a serious mix of semi- structured and unstructured components: o Images o Video o Text descriptions and news, blogs, etc… o User and customer commentary o Reactions on social media: e.g. Twitter is a mix of data anyway • Using standard transforms, entity extraction, and new generation tools to transform unstructured raw data into semi-structured analyzable data
  12. 12. Text Data: The Big Driver • We speak of “big data” and the “Variety” in 3-V’s • Reality: biggest driver of growth of Big Data has been text data o Most work on analysis of “images” and “video” data has really been reduced to analysis of surrounding text Nowhere more so than on the internet • Map-Reduce popularized by Google to address the problem of processing large amounts of text data: o Many operations with each being a simple operation but done at large scale o Indexing a full copy of the web o Frequent re-indexing
  13. 13. A few words on: The Chief Data Officer Why are companies creating this position? • There is a fundamental realisation that Data needs to become a primary value driver at organizations o We have lots of Data o We spend much on it: in technology and people o We are not realising the value we expect from it • A strong business need to create the CDO role: o Traditional companies are not following, but adopting the model that actually works in other data-intensive industries • CDO has a seat at executive table: the voice of Data • Data done right is an essential element to unify large enterprises to unlock value from business synergies
  14. 14. Fundamental Data Principles to Support Analytics Usama’s Obvious Data Axioms 1. Data gains value exponentially when integrated and coalesced. • When fragmented: dramatic value loss takes place; • Increased costs; • Reduced utility/integrity; • Increased security risks 2. Fusing Data together from disparate/independent sources is difficult to achieve and impossible to maintain Hence only viable approach is: • Intercepting and documenting at the source • Fusing at the source • Controlling lifecycle and flow
  15. 15. Fundamental Data Principles to Support Analytics Usama’s Obvious Data Axioms 3. Standardisation is essential • For sustained ability to integrate data sources and hence growing value; • For simplifying down-stream systems and apps • For enforcing discipline as a firm increases its data sources 4. Data governance and policy must be centralised • Needs to be enforced strongly else we slip into chaos and a Babylon of terms/languages • An Enterprise Data Architecture spanning structured and unstructured data 5. Recency Matters -> data streaming in modelling and scoring • Often, accuracy of prediction drops quickly with time (e.g. consumer shopping) • Value of alerts drop exponentially with time… • Ability to trigger responses based on real-time scoring critical • Streaming, real-time model updates, real-time scoring
  16. 16. Fundamental Data Principles to Support Analytics Usama’s Obvious Data Axioms 6. Abstraction layer of data from Apps/platforms is a must • Rapid renewal & modernization: the pace of change and development of technology are very rapid • Design for migration and infrastructure replacement via abstraction layers that remove tech dependencies • Encryption and Masking: Persisting unencrypted confidential and secret data (even within secure firewalls) is an invitation for problems and risks 7. Data is a primary competency and not a side-activity supporting other processes • Hence specialized skills and know-how are a must • Generalists will create a hopeless mess • Data is difficult: modelling, architecture, and design to support analytics
  17. 17. Driving the need for integrated processes, data & architecture Regulatory Demand  Data quality, completeness, lineage and aggregation  Company financial reporting presented at most granular level through various lenses: business units, legal entities, regional, sector level, …  Faster turnaround for increasingly complex ad hoc data requests  Additional sophisticated stress tests delivered in a joined-up manner between Risk and Finance  Enhanced hybrid capital computation methodologies Business Drivers  Better and smarter controls  Capital precision  Better customer experience  Better colleagues experience  More cost effective
  18. 18. Data Governance is BORING for most… Policy, governance & control Business ownership & data stewardship Definitions & metadata Permit to build (change governance) Reporting & analytics Data architecture Data sourcing Reconciliation Data quality management Documentation, tracking & audit …but it shouldn’t be!
  19. 19. Reality check: So what should users of analytics in Big Data world do? Load the Data into a “Data Lake” Your life will be so much easier as you can now do Data Acrobatics & other amazing data feats
  20. 20. The Data Lake – according to Waterline We loaded the Data! Congratulations Now What?
  21. 21. From a Data Lake to Amazon Browser Amazon Simple Search Amazon gives you facets Product details
  22. 22. Reality check: So where do analysts & Data Scientists spend all their time? Let’s Mine the Data
  23. 23. Reality check: So what do Technology people worry about these days? To Hadoop or not to Hadoop? When to use techniques requiring Map-Reduce and grid computing? Typically organizations try to use Map-Reduce for everything to do with Big Data • This is actually very inefficient and often irrational • Certain operations require specialized storage o Updating segment memberships over large numbers of users o Defining new segments on user or usage data
  24. 24. Drivers of Hadoop in large enterprises Cost of storage Fastest growing demand is more storage Data in Data Warehouses have traditionally required expensive storage technology: • $20K to $50k per terabyte per year – cost of premium storage • $2K per terabyte – much lower per year – cost of Hadoop on commodity storage
  25. 25. Analysis & programming software PIG HIPI
  26. 26. Hadoop Stack
  27. 27. Big Data Landscape
  28. 28. Data Platforms Landscape Map
  29. 29. Reality check: If storage is biggest driver of Hadoop adoption, what is next biggest? ETL • Replaces expensive licenses • Much higher performance with lower infrastructure costs (processors, memory) • Flexibility in changing schema and representation • Flexibility on taking on unstructured and semi-structured data • Plus suite of really cool tools…
  30. 30. Turning the 3-Vs of Big Data into Value Understand context and content • What are appropriate actions? • Is it Ok to associate my brand with this content? • Is content sad?, happy?, serious?, informative? Understand community sentiment • What is the emotion? • Is it negative or positive? • What is the health of my brand online? Understand customer intent? • What is each individual trying to achieve? • Can we predict what to do next? • Critical in cross-sell, personalization, monetization, advertising, etc…
  31. 31. Many business uses of predictive analytics Analytic technique Uses in business Marketing and sales Identify potential customers; establish the effectiveness of a Understanding customer behavior model churn, affinities, propensities, … Web analytics & metrics model user preferences from data, collaborative filtering, targeting, Fraud detection Identify fraudulent transactions Credit scoring credit worthiness of a customer requesting a loan, secured and loans Manufacturing process analysis Identify the causes of manufacturing problems Portfolio trading optimize a portfolio of financial instruments by maximizing returns & minimizing risks Healthcare Application fraud detection, cost optimization, detection of events like epidemics, etc... Insurance fraudulent claim detection, risk assessment Security and Surveillance intrusion detection, sensor data analysis, remote sensing, detection, link analysis, etc... Application Log File Analytics Understanding application, network, event logs in IT
  32. 32. Case Studies: 1. Context Analysis (unstructured data) 2. IOT Case Study 3. Yahoo! Predictive Modeling
  33. 33. Reality Check So who is the company we think is best at handling Big Data?
  34. 34. Case Study: Biggest Big Data in advertising? Understanding context for ads What Ad would you place here?
  35. 35. Case Study: Biggest Big Data in advertising? Understanding context for ads Damaging to Brand?
  36. 36. 38 Invited talk – #ODSC, Boston– Copyright Usama Fayyad © 2015 The Display Ads Challenge Today What Ad would you place here?
  37. 37. NetSeer: Solving accuracy issues Ambiguity, waste, brand, safety Why did Google Serve this Ad? 39 this is how NetSeer actually sees this content
  38. 38. NetSeer: How it works 40 high MPG ford low emission fuel efficiency ECONOMY CARS economy vehicles microscope lenses reading glasses autofocus bifocal refraction VISION TOOLS eye chart focus groups A/B testing consumer study surveying blind study analytics MARKET RESEARCH ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ WEBSITE.COM ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ electric vehicles service record safety rating focus <CONCEPT> DISCERNS AND MONETIZES HUMAN INTENT + Identifies Concepts expressed on a page + Disambiguates language + Builds increasingly rich profile over time 52M 2.3B CONCEPTS RELATIONSHIPS BETWEEN CONCEPTS
  39. 39. NetSeer: Intent for Display Positive vs negative content re a particular topic
  40. 40. Problem: Hard to understand user intent Contextual Ad served by Google What NetSeer Sees:
  41. 41. Case Studies: 1. Context Analysis (unstructured data) 2. IOT Case Study 3. Yahoo! Predictive Modeling
  42. 42. The Connected Cow Joseph Sirosh VP, Machine Learning Microsoft
  43. 43. Case Studies: 1. Context Analysis (unstructured data) 2. IOT Case Study 3. Yahoo! Predictive Modeling
  44. 44. Yahoo! – One of Largest Destinations on the Web 80% of the U.S. Internet population uses Yahoo! – Over 600 million users per month globally! Global network of content, commerce, media, search and access products 100+ properties including mail, TV, news, shopping, finance, autos, travel, games, movies, health, etc. 25+ terabytes of data collected each day • Representing 1000’s of cataloged consumer behaviors More people visited Yahoo! in the past month than: • Use coupons • Vote • Recycle • Exercise regularly • Have children living at home • Wear sunscreen regularly Sources: Mediamark Research, Spring 2004 and comScore Media Metrix, February 2005. Data is used to develop content, consumer, category and campaign insights for our key content partners and large advertisers
  45. 45. Yahoo! Big Data – A league of its own… Terrabytes of Warehoused Data 25 49 94 100 500 1,000 5,000 Amazon Korea Telecom AT&T Y!LiveStor Y!Panama Warehouse Walmart Y!Main warehouse GRAND CHALLENGE PROBLEMS OF DATA PROCESSING TRAVEL, CREDIT CARD PROCESSING, STOCK EXCHANGE, RETAIL, INTERNET Y! Data Challenge Exceeds others by 2 orders of magnitude Millions of Events Processed Per Day 50 120 225 2,000 14,000 SABRE VISA NYSE YSM Y! Global
  46. 46. Behavioral Targeting (BT) Search Ad Clicks Content Search Clicks BT Targeting ads to consumers whose recent behaviors online indicate which product category is relevant to them
  47. 47. Male, age 32 Lives in SF Lawyer Searched on from London last week Searched on: “Italian restaurant Palo Alto” Checks Yahoo! Mail daily via PC & Phone Has 25 IM Buddies, Moderates 3 Y! Groups, and hosts a 360 page viewed by 10k people Searched on: “Hillary Clinton” Clicked on Sony Plasma TV SS ad Registration Campaign Behavior Unknown Spends 10 hour/week On the internet Purchased Da Vinci Code from Amazon Yahoo! User DNA • On a per consumer basis: maintain a behavioral/interests profile and profitability (user value and LTV) metrics
  48. 48. How it works | Network + Interests + Modelling Analyze predictive patterns for purchase cycles in over 100 product categories In each category, build models to describe behaviour most likely to lead to an ad response (i.e. click). Score each user for fit with every category…daily. Target ads to users who get highest ‘relevance’ scores in the targeting categories Varying Product Purchase CyclesMatch Users to the ModelsRewarding Good BehaviourIdentify Most Relevant Users
  49. 49. Recency Matters, So Does Intensity Active now… …and with feeling
  50. 50. Differentiation | Category specific modelling time intensityscore time intensityscore IntenseClickZone Example 1: Category Automotive Example 2: Category Travel/Last Minute Different models allow us to weight and determine intensity and recency Alt Behaviour 1: 5 pages, 2 search keywords, 1 search click, 1 ad click Alt Behaviour 1: 5 pages, 2 search keywords, 1 search click, 1 ad click IntenseClickZone
  51. 51. Differentiation | Category specific modelling time intensityscore Intense Click Zone Example 1: Category Automotive Different models allow us to weight and determine intensity and recency with no further activity, decay takes effect Alt Behaviour 1: 5 pages, 2 search keywords, 1 search click, 1 ad click user is in the Intense Click Zone
  52. 52. Automobile Purchase Intender Example A test ad-campaign with a major Euro automobile manufacturer  Designed a test that served the same ad creative to test and control groups on Yahoo  Success metric: performing specific actions on Jaguar website Test results: 900% conversion lift vs. control group  Purchase Intenders were 9 times more likely to configure a vehicle, request a price quote or locate a dealer than consumers in the control group  ~3x higher click through rates vs. control group
  53. 53. Mortgage Intender Example We found: 1,900,000 people looking for mortgage loans. +122% CTR Lift Mortgages Home Loans Refinancing Ditech Financing section in Real Estate Mortgage Loans area in Finance Real Estate section in Yellow Pages +626% Conv Lift Example search terms qualified for this target: Example Yahoo! Pages visited: Source: Campaign Click thru Rate lift is determined by Yahoo! Internal research. Conversion is the number of qualified leads from clicks over number of impressions served. Audience size represents theaudiencewithinthis behavioralinterest categorythat hasthe highestpropensitytoengagewitha brandorproductandtoclickon anoffer. Date: March 2006 Results from a client campaign on Yahoo! Network Example: Mortgages
  54. 54. Experience summary at Yahoo! • Dealing with one of the largest data sources (25 Terabyte per day) • Behavioral Targeting business was grown from $20M to > $400M in 3 years of investment! • Yahoo! Specific? -- BigData critical to operations – Ad targeting creates huge value – Right teams to build technology (3 years of recruiting) – Search is a BigData problem (but this has moved to mainstream)
  55. 55. Lessons Learned A lot more data than qualified talent  Finding talent in BigData is very difficult  Retaining talent in BigData is even harder At Yahoo! we created central group that drove huge value to company Data people need to feel like they have critical mass  Makes it easier to attract the right people  Makes it easier to retain Drive data efforts by business need, not by technology priorities  Chief Data Officer role at Yahoo! – now popular
  56. 56. Where does this leave us? 1. What matters in the age of analytics?. • Being able to exploit all the data that is available • Proliferating analytics throughout the organization • Driving significant business value 2. Where are we today? • New Data Landscape via Hadoop • Confusion by marketers, analysts, and technical community • Struggling with basics of managing data because of the new flexibility 3. What are the real issues? • Data management and governance • Talent for Data and Data Science is rare but critical • Data and Analytics are specialisms that need management and know-how: not for generalists…
  57. 57. The early days of mass auto production
  58. 58. Today’s Auto: It just works! No need to understand what happens when you turn on ignition Very complex inside, but all simplicity on the outside
  59. 59. Data Science, AI, Algorithms Data Science Artificial Intelligence / Machine Learning Algorithms Data as a Service (DaaS) to provide our business units with better, faster, cheaper data Algorithms Create leading edge Algorithms for Automation via AI, Machine Learning, and leading edge data science Automation Tech via AI CoE for Intelligent Automation via AI, ML, case-based reasoning & Planning Secure Cloud Data Lab Secure Cloud data lab with open source stack to be able to leverage external datasets and work with the start-ups to innovate at scale Experimentation evaluation Data-fuelled evaluation of experimentation at scale, e.g. digital experiences empowered by data Data Solutions for Customers/ Clients Create a Lab for innovative solutions for customers and Clients KEY ADVANCED CAPABILITIES FRAUD / CYBERSECURITY • Reduced false alarms • Less write-offs • Better intrusion detection, cyber attacks, internal financial crime BUSINESS IMPACT - EXAMPLES RAPID AUTOMATION • Learning algorithms automate and roboticise manual tasks automatically by ‘crowd-sourcing’ • Automated reporting and reduced manual reconciliation SMARTER TRADING ALGORITHMS • Smarter trading algorithms and quantitative analytics enabled by new and more powerful technology TARGETED MARKETING • Leverage automated learning and classification to customer acquisition, retention, and activation REVENUE DRIVER • Leverage social media / unstructured data for Cross-sell, up-sell, next-best-action, CRM, etc.
  60. 60. Barclays Local Insights: Providing insights about people
  61. 61. Application Dates: Rising Eagles Graduate Programme: April-June | Explorer Programme: 1 week in July We recruit on a rolling bases & early applications are advised – find out more and apply at
  62. 62. Technology opportunities at Barclays We seek world class technologists, data scientists, problem solvers, Data systems engineers – the team that will reinvent Financial Services Create game changing new products and services for our Africa businesses across retail, business banking, Card, Corporate/IB, and Wealth/Investment management that do not only generate new experiences and growth but new business models. We are looking for hungry data talent that is looking to move the needle in all our businesses. Yassi Hadjibashi Chief Data Officer, Barclays Africa - DSI Africa is a huge growth opportunity for Barclays. If you know your Data Science, We want you! If you love Hadoop & BigData, We love you! And if you want to be part of an industry transformation around Data in Financial Services, come help us make it happen! Dr. Usama Fayyad Group Chief Data Officer, Barclays Bank Security, Data, and Software as code. – Basically everything automated for sophisticated rapid innovation, deployment cycles, and new age software engineering. Come join our journey and drive this with backing of most senior leadership and attention!!! Peter Rix Chief Technology Officer, Barclays Africa
  63. 63. Thank you & questions Usama Fayyad @Usamaf Africa