Data plays a big role in building the kinds of experiences demanded by the market today. In this session, we’ll unpack what goes into building a data-driven app, case studies of how organizations have successfully overcome siloed data and analytics to bring new predictive features into their applications, and what your next steps for data should be on your digital transformation journey. Speaker: Les Klein, EMEA CTO Data, Pivotal

2. The Role of Data in Your
Digital Transformation
Les Klein
Field CTO Data
Pivotal
@LesKlein #PivotalForum #DigitalTransformation

3. Great software companies leverage Big Data
to fundamentally change the consumer
experience and pioneer entirely new business
models

4. 4© 2016 Pivotal Software, Inc. All rights reserved.
$4BN
Financial Services
$26BN
Hospitality
$50BN
Transportation
$54BN
Entertainment
$30BN
Automotive
$3.2BN
Industrial Products
CLOUD NATIVE SOFTWARE IS CHANGING INDUSTRIES
Data is Fueling Software

5. 5© Copyright 2015 Pivotal. All rights reserved.
Hundreds of
thousands of “trip”
events each day
400+ billion of
viewing-related
events per day
Five billion
training data
points for Price
Tip feature
Disruptors Use a LOT of Data

6. 6© Copyright 2015 Pivotal. All rights reserved.
“We’ve found that when a
host selects a price that’s
within 5% of their tip,
they’re nearly 4 times
more likely to get booked”
“The importance of accuracy and
efficiency […], will continue to
rise as we expand and improve
products like uberPOOL and
beyond.”
“Over 75% of what
people watch come
from our
recommendations”
Data manifests as features in an app

7. 7© Copyright 2015 Pivotal. All rights reserved.
(Data)
Microservices
Loosely coupled
services architecture,
bounded by context
Cloud-Native
Platforms
Enabling continuous
delivery & automated
operations
Open Source
Database
Innovation
Extreme scale &
performance advantages,
built for the cloud
Machine
Learning
Use of predictive
analytics to build
smart apps
How are they accomplishing this?

8. 8© Copyright 2015 Pivotal. All rights reserved.
These companies…
Release new features in minutes, multiple times a day
Support a micro-services architecture
Consume a wide range of data sources and protocols
Store and Analyze all their data
Update algorithms and predictive models daily
Continuously ask lots of questions of their data
Modify data pipelines and add processing steps daily
They are Data Driven, are you?

9. 9© Copyright 2015 Pivotal. All rights reserved.
“Companies need to learn how to
catch
people or things in the act of doing
something and affect the outcome“
PAUL MARITZ
Executive Chairman, Pivotal
Real-time and
Personalised Information
in Context is what Wins!

10. 10© Copyright 2015 Pivotal. All rights reserved.
A lot of reports do
not make you data
driven
A lot of
dashboards do not
make you data
driven A lot of alerts do
not make you data
driven

11. 11© Copyright 2015 Pivotal. All rights reserved.
Converging
Trends
Innovation
New Data New Processes New Insights
The Journey to the Data-Driven Enterprise
Data Science
and Machine
Learning
Big Data
Internet
of Things &
Social Media

12. 12© Copyright 2015 Pivotal. All rights reserved.
HDFS
Data Lake
Ingest Store Analytics
Hard to change
Labor intensive
Inefficient
Coding based
No real-time information
Based on expensive ETL
Migrating from a Reactive, Static and
Constrained Model…

13. 13© Copyright 2015 Pivotal. All rights reserved.
HDFSData Lake
Expert System /
Machine Learning
In-Memory Real-
Time Data
Continuous Learning
Continuous Improvement
Continuous Adapting
Data Stream Pipeline
Multiple Data Sources
Real-Time Processing
Store Everything
To Pro-Active, Self-Improving, Machine
Learning Systems

14. 14© 2016 Pivotal Software, Inc. All rights reserved.
What does “Customer” mean to you?

15. 15© 2016 Pivotal Software, Inc. All rights reserved.
The new normal: “an audience of one”
DATA DEVICES
Media
Banks
Delivery
Services
Marketers
Government
Individual
s Employers
Analytic
Services
Advertising
Catalog
Co-ops
List
Brokers
Websites
Information
Brokers
Credit
BureausMedia
Archives
GOVERNMENTGOVERNMENT
PHONE/
TV
PHONE/
TV
INTERNETINTERNET
AD
AGENCY
AD
AGENCY
RETAILRETAIL

16. 16© Copyright 2015 Pivotal. All rights reserved.

17. 17© Copyright 2015 Pivotal. All rights reserved.

18. 18© 2016 Pivotal Software, Inc. All rights reserved.
Your customers are a new breed

19. 19© 2016 Pivotal Software, Inc. All rights reserved.
Evolving from digital natives to data
natives

20. 20© 2016 Pivotal Software, Inc. All rights reserved.
“A data native is someone who expects their world to not just be digital,
but to be smart and to adjust immediately to their taste and habits.”
Beyond tech savvy …..

21. 21© 2016 Pivotal Software, Inc. All rights reserved.
Digital natives program their
thermostat

22. 22© 2016 Pivotal Software, Inc. All rights reserved.
Data natives expect the thermostat to
program itself

23. 23© 2016 Pivotal Software, Inc. All rights reserved.
Do you know the full value of your data
and how to leverage it?

24. 24© 2016 Pivotal Software, Inc. All rights reserved.
µs ms s hour day month yr+
“Fast Data” “Big Data”
year
Traditional
Systems
Pivotal Data
Science Labs
Value of
Data ($)
Time
Value of data over time…

25. 25© 2016 Pivotal Software, Inc. All rights reserved.
Analytics - the journey so far (BI)
Complexity
Value of
Analytics ($)
Descriptive
Analytics
Diagnostic
Analytics
What
happened?
Why did it
happen?
Hindsight
Insight

26. 26© 2016 Pivotal Software, Inc. All rights reserved.
Analytics – where you need to be today
Value of
Analytics ($)
Descriptive
Analytics
Diagnostic
Analytics
Predictive
Analytics
Prescriptive
Analytics
What
happened?
Why did it
happen?
What will
happen?
How can we
make it happen?
Hindsight
Insight
Foresight
Complexity

27. 27© 2016 Pivotal Software, Inc. All rights reserved.
Interesting questions are found here
Value of
Analytics ($)
Descriptive
Analytics
Diagnostic
Analytics
Predictive
Analytics
Prescriptive
Analytics
What
happened?
Why did it
happen?
What will
happen?
How can we
make it happen?
Hindsight
Insight
Foresight
Complexity
Data science

28. 28© 2016 Pivotal Software, Inc. All rights reserved.
Data Science at Pivotal
Theory-driven approach
‘start with the problem
and work towards the data’
‘start with the data
and work towards the problem’
Data-driven approach

29. 29© 2016 Pivotal Software, Inc. All rights reserved.
Data Science at Pivotal
‘start with the data
and work towards the problem’
Data-driven approach

30. 30© 2016 Pivotal Software, Inc. All rights reserved.
User Feedback
Scoping Data Review
Model Building
Feature Review
Feature Engineering
Model Evaluation
Operationalization
Our tried and tested approach helps extract
value from data

31. “
THE MAGIC HAPPENS WHEN YOU MARRY THE
TRADITIONAL ENGINEERING APPROACH WITH THE
DATA SCIENCE ENABLED BY THE DATA LAKE. IT
OPENS UP A WHOLE NEW WORLD OF POSSIBLE
‘WHAT IF’ QUESTIONS.
”DAVE BARTLETT, GE AVIATION

32. 32© 2016 Pivotal Software, Inc. All rights reserved.
IT’S ALL ABOUT
HOW YOU
OPERATIONALISE
YOUR INSIGHTS

33. 33© 2016 Pivotal Software, Inc. All rights reserved.
Scale-out
analytic
database
Model API Cloud Native
Application
Platform
Data
Sources
0 5
Smart Apps: Models Manifesting as Microservices

34. 34© 2016 Pivotal Software, Inc. All rights reserved.
OSS
Smart/Data-driven App Technology Needs
Fast Ingest/Pipelining
• Pipelines to consume
streaming and batch data
from various endpoints
Speed/Serving Layer
• In-memory data grids for
real-time and aggregated
data sets
• Low latency
Big Data Analytics Platform
• Scale-out Storage
• Ability to process large structured and unstructured data
• Advanced SQL query capabilities
• Ability to run machine learning at scale
Platform for Scalable
Web/Mobile Apps
• Infrastructure Automation
• Develop, run and manage
Web applications without
the complexity of building
and maintaining the
infrastructure

35. 35© 2016 Pivotal Software, Inc. All rights reserved.
Fast Ingest/Pipelining Speed/Serving Layer
Big Data Analytics Platform
Platform for Scalable
Web/Mobile Apps
Pivotal’s Answer for Technology Needs
Spring Cloud Data Flow GemFire
Greenplum
Database
Pivotal
HDB
Pivotal
Cloud Foundry
Spring
OSS

36. 36© 2016 Pivotal Software, Inc. All rights reserved.
● Silo’d and aging database systems
● Spaghetti data pipelines
● Expensive, proprietary data
management systems
● Lack of structured platforms for
continuous software delivery
● Monolithic application architectures
● Batch-oriented data integration
● Limited operationalization of analytics
● Proprietary systems
Today’s Enterprise Challenge

37. 37© 2016 Pivotal Software, Inc. All rights reserved.
Data Programming Model
Cloud-Native Platform
Microservices FrameworkPlatform Runtime
Stream Data Platform
Hadoop Spark DW
Apps & Microservices
DBMS
IMDG
K/V Store
Relational DB
Big Data & Machine Learning
Modern Cloud-Native Data Architecture

38. 38© 2016 Pivotal Software, Inc. All rights reserved.
1. Modernize your data platforms:
• Improve performance and scalability for
mission-critical data workloads
• Lower overall TCO by adopting industry-leading open
source database technologies on modern cloud platforms
2. Prove out a specific business use-case:
• Build a smart app end-to-end
• Focus on a specific analytical model / data-microservice
• Automate ingestion and data pipelines
Pivotal can help you to:

39. 39© 2016 Pivotal Software, Inc. All rights reserved.
Pivotal Offerings
Big Data Suite:
• Best-in-class open source data management software
• GemFire: In-Memory Data Grid
• Greenplum: Data Warehouse
• Pivotal HDB (Apache HAWQ): Hadoop-Native SQL
Pivotal Cloud Foundry
• Industry’s Leading Cloud-Native Platform
Spring Boot, Cloud and Data Flow
• Modern-Java micro-services framework
Pivotal Labs & Data Science
• Agile software development and machine learning

40. Let’s build something
MEANINGFUL
Let’s build something
MEANINGFUL

41. 41© 2016 Pivotal Software, Inc. All rights reserved.
APPA Real-World Example

42. 42© 2016 Pivotal Software, Inc. All rights reserved.
App
Development
Data
analytics
Cloud-native
App platform
Data
Science
& Model
building
Data
Micro-
service
APPA Real-World Example

43. 43© 2016 Pivotal Software, Inc. All rights reserved.
App
Development
Data
analytics
Cloud-native
App platform
Data
Science
& Model
building
Data
Micro-
service
APP
Must support scale-out
query processing
Must support scale-out
query processing
Must deliver as an APIMust deliver as an API
Must embrace agile development,
focus on outcomes
Must embrace agile development,
focus on outcomes
Must support
microservices, agile dev, and
connect to big data analytics
Must support
microservices, agile dev, and
connect to big data analytics
A Real-World Example

44. 44© 2016 Pivotal Software, Inc. All rights reserved.
Major Credit Card Issuer Builds Infrastructure Determining
Fraud Markers Across 30 Disparate Sources
Challenge:
• Desire to identify potential fraudulent activity within Fortune
100 credit card issuer with 100K employees
• Identifying markers across 30 internal data sources caused
lags in reporting
• Establish Infrastructure to support Palantir, Platfora and RSA
reporting frameworks
Solution:
• Provided ability to store 3 months of data at 60-70TB, interest
in moving to 3-year retention period
• Build large-scale security analytics platform to scale with data
needs, analytics and reporting requirements
• Support organizational desire to evolve from pure BI to
advanced, predictive analytics
Pivotal Solution includes: GPDB, Hadoop, HAWQ

45. 45© 2016 Pivotal Software, Inc. All rights reserved.
Leading Swiss Bank Re-shapes Business to Better Monetize and
Regulate Fraudulent Activity with Big Data
Challenge:
• Needed to monetize data around privilege user access and
assure security around fraudulent activity
• Reduce the cost of monitoring privileged user access for
regulatory requirements. (currently taking 45 people to do this)
• Extend across the enterprise to solve business pains around
faster data as opposed to just longer looks in “Hadoop”
Solution:
• Built on top of Big Data Suite to operationalize, automate and
extend across the enterprise
• What took 6 hours should now take less than 1 hour with real
time decision-making
• Capability of monitoring relationship manager’s activity with
analytics to ensure regulatory compliance
Pivotal Solution includes: Big Data Suite and
Data Labs

46. 46© 2016 Pivotal Software, Inc. All rights reserved.
Pivotal Data Innovation Across Industries
RETAIL / TRANSPORT
MEDIA / SERVICES
TELCO / ISP
FINANCIAL SERVICES
HIGH TECH / ELECTRONICS
EDUCATION
GOVERNMENT
OTHER
INSURANCE
AUTO / INDUSTRIAL
HEALTHCARE
MANUFACTURING