More Related Content Similar to Fast Data:The Rebirth of Streaming Analytics (20) Fast Data:The Rebirth of Streaming Analytics1. www.ovum.com
© Copyright Ovum 2014. All rights reserved.
Fast Data:
The Rebirth of Streaming Analytics
Tony Baer
Ovum
Teradata Partners, October 21, 2015
2. © Copyright Ovum 2014. All rights reserved.
What is Streaming Analytics?
The Rebirth
Technology Landscape
Takeaways
Agenda
3. © Copyright Ovum 2014. All rights reserved.
What is Streaming Analytics?
Analyzing & Acting on data in motion
Incoming
data
In Motion
Filtered extract
Streaming Analytics
Conventional Analytics
Sense,
Transform/Filter,
Analyze Data Respond
Analyze Respond
Event
processor
Ingest,
Persist Data
Data store
Data with perishable
value
Data with historical value
Incoming
data
4. © Copyright Ovum 2014. All rights reserved.
Streaming Analytics is not
simply responding to
alarms or outliers
5. © Copyright Ovum 2014. All rights reserved.
Streaming Analytics examples
Telcos – process CDRs for mediation,
revenue assurance, fraud detection,
churn prevention
FS – process trades for fraud
detection & anomalous activity, refine
trading strategies
Utilities – process smart meter data
for demand-side management
programs
Healthcare – patient monitoring for
alerts (e.g., sepsis outbreaks) &
offline clinical research
7. © Copyright Ovum 2014. All rights reserved.
Streaming Analytics roots
Complex Event Processing (CEP)
Define Event &
relationships to
other events
Define
Event/state
Transition
Define Pattern
matching rules
Define
Response rules
Event Stream Processing (ESP)
Sliding time
windows for
correlation &
aggregation of
events
8. © Copyright Ovum 2014. All rights reserved.
The showstopper?
Complexity
Costly hardware
Limited bandwidth
Proprietary software
Narrow market appealLimited skills base
No standards
CEP
10. © Copyright Ovum 2014. All rights reserved.
What’s changed?
Use Cases – Driven by explosion of
Mobile & IoT data
Commodity Infrastructure – Scale-
out clusters, multi-core CPUs,
gigabit networks, affordable DRAM
& Flash storage
Open Source – lowering barriers to
entry for developers, data scientists,
enterprises, and vendors
Machine Learning provides more
flexible, adaptive alternative to rules
12. © Copyright Ovum 2014. All rights reserved.
IoT growth
Source: Cisco
2014 2019
67%
40%
By 2019, most IP
traffic will come from
non-PC devices
By 2019 Global IP traffic will
grow 3x to 2 zettabytes/yr.
By 2016, most IP traffic to come from wireless devices
13. © Copyright Ovum 2014. All rights reserved.
Emerging use cases
Retail – real-time customer
engagement via smartphone
interaction
Manufacturing – prescriptive
maintenance
Telco – Real-time message routing
optimization & bottleneck prevention
Local govt. – Real-time Smart City
applications
Cybersecurity – Real-time detection
& thwarting of intrusions/attacks
14. © Copyright Ovum 2014. All rights reserved.
Streaming Analytics Technology Landscape –
Then
Tibco Streambase
Software AG Apama
SAP Complex Event Processing
Oracle Event Processing
Informatica Rulepoint
IBM InfoSphere Streams
15. © Copyright Ovum 2014. All rights reserved.
Streaming Analytics Technology Landscape –
Now
Veterans
Community
Open Source
New Players
Tibco
Software AG
SAP
Oracle
Informatica
IBM
Spark Streaming
Flink
Kafka
Storm
Samza
DataTorrent
Msft. Azure Stream Analytics
Amazon Kinesis
Teradata Listener
Tigon
Heron
SAS
16. © Copyright Ovum 2014. All rights reserved.
Streaming Analytics Technology Landscape –
Contrasts
Veterans
Community
Open Source
New Players
• CEP/ESP rebranded &
leveraging modern
commodity
infrastructure
• Mature enterprise
software
• Mix of proprietary &
vendor-lead open
source
• Cloud prominence
• Expanding the
practitioner base
• Leveraging ML instead
or in addition to rules
• Manual coding
Tibco
Software AG
SAP
Oracle
Informatica
IBM
Spark Streaming
Flink
Kafka
Storm
Samza
DataTorrent
Msft. Azure Stream Analytics
Amazon Kinesis
Teradata Listener
Tigon
Heron
SAS
17. © Copyright Ovum 2014. All rights reserved.
Takeaways
Streaming Analytics… is back!
It’s not only for Wall St. anymore
Mobile & IoT driving compelling real-time use cases outside traditional
FS/capital markets niche
Machine Learning provides more adaptive, flexible alternative (or addition) to
rules
Commodity infrastructure & open source makes Streaming Analytics
affordable, scalable & performant
Open source erodes barriers to entry – but the software is still raw
Don’t rule out mature commercial products – but they must exploit modern
commodity, scale-out distributed architectures!