2. Content
• Watson Foundation for CSPs (WFC)
• Discovery and Predictive Modeling using SPSS
• Detection and Real-time Analytics using InfoSphere Streams
• Component Integration
• Conclusions
3. Real-Time Analysis and
Event Processing
(RTAP)
BSS Netezza / ISAS / …
Real Time Analytical Processing Data Warehouse
Network
OSS
(Landing Area)
Reference Data
(from EDW)
Input Data
Mediation
Our
Telco
Evolu,on
–
Ini,ally
Just
Media,on
and
Complex
Event
Detec,on
4. Mediation
Real-Time Analysis and
Event Processing
(RTAP)
DWH, Analytics
Foundation
(ELT & In-Database
Analytics)
Real Time Analytical Processing Data Warehouse
Network
OSS
(Landing Area)
Reference Data
(from EDW)
BSS
Input Data
Real-Time Event
Triggers
PMML Model
Deployent
Marketing Campaign
System
Mobile Advertising
Predictive Analytics
(SPSS, etc.)
Analytical Monetization Platforms Source Systems
Our
Telco
Evolu,on
–
Adding
on
Marke,ng
and
Predic,ve
Analy,cs
5. Deep Packet Inspection
Real Time Analytical Processing Network Analytics
Network Analytics
Insights
(Dashboards)
EDW
Real-Time Event
Triggers
Marketing Campaign
System
Mobile Advertising
Analytical Monetization Platforms Source Systems
Network
OSS
(Landing Area)
Reference Data
(from EDW)
BSS
Our
Telco
Evolu,on
–
Adding
DPI
and
Network
Analy,cs
BI & Visualization
Customer Experience
Management
(Data Model)
Other BI Dashboards &
Reports
Analytics Foundation
(ELT & In-Database
Analytics)
CRM
Predictive Analytics
TNF
Real-Time Analysis and
Event Processing
(RTAP)
Input Data
Mediation
SPSS-Streams
Toolkit
TNF
6. Deep Packet Inspection
Geo-Spatial Analytics
Cyber-security Analytics
Web Analytics
Real Time Analytical Processing Network Analytics
Network Analytics
Insights
(Dashboards)
EDW
Real-Time Event
Triggers
Analytical Monetization Platforms Source Systems
Looking
Ahead-‐-‐
More
Analy,cs
,
Cybersecurity,
BigInsights
Marketing Campaign
System
Mobile Advertising
BI & Visualization
Customer Experience
Management
(Data Model)
Other BI Dashboards &
Reports
Analytics Foundation
(ELT & In-Database
Analytics)
CRM
Predictive Analytics
TNF
Real-Time Analysis and
Event Processing
(RTAP)
Network
OSS
(Landing Area)
Reference Data
(from EDW)
BSS
Unstructured
Data
Input Data
Mediation
Hadoop Distributed File
System
(HDFS )
Text Analytics
Machine Learning
Large Scale Analytics
Big Insights
Web Log Analytics
Sentiment Analytics
Topic-based Influencers
Social Media Applications
TNF
SPSS-Streams
Toolkit
7. Drive
WFC uses D4 for tight integration across four
analytics components
Apply
the
results
of
inves1ga1on
to
take
ac1on
by
interac1ng
with
the
subscribers
in
real-‐
1me.
Collect
feedback
from
ac1on
for
future
analysis.
Discover
Collect
historical
behavioral
data,
past
acts,
and
success
rates.
Analyze
historical
data
to
formulate
pa?erns
and
changes
required
to
detect,
and
inves1gate
steps
Decide
Gather
data
on
targeted
customers
from
a
variety
of
sources
over
1me
to
establish
behavioral
pa?erns
and
iden1fy
how
to
respond
to
an
emerging
pa?ern.
Detect
Detect
in
real
1me
if
a
transac1on,
request,
applica1on,
document,
etc.
is
required
for
targe1ng.
Flag
the
selected
dataset
and
ignore
the
rest.
Detect
observa,ons
about
a
target
Take
ac,on
in
real
,me
–
when
it
ma8ers
Find
new
targets
by
analyzing
historical
data
Iden,fy
pa8erns
over
,me
and
ac,ons
required
Drive
Detect
Discover
Decide
Target
Subscriber
7
8. Drive
Interact
with
the
customer
to
seek
permission
to
use
loca1on
informa1on
and
send
campaign,
record
interac1on
and
results.
Smarter Campaigns using D4
Discover
Collect
historical
behavioral
data,
past
acts,
and
success
rates.
Analyze
historical
data
to
formulate
pa?erns
and
changes
required
to
detect,
and
inves1gate
steps
Decide
Use
background
informa1on,
past
campaigns,
privacy
preferences,
customer
reac1on
to
past
campaigns,
purchase
intent,
preferences
expressed
in
social
media
to
design
campaign.
Detect
Detect
in
real
1me
if
a
transac1on
relates
to
targeted
subscribers.
Iden1fy,
align,
score,
and
send
for
further
processing
(e.g.,
a
targeted
customer
driving
towards
mall)
Detect
observa,ons
about
a
target
Take
ac,on
in
real
,me
–
when
it
ma8ers
Find
new
targets
by
analyzing
historical
data
Iden,fy
pa8erns
over
,me
and
ac,ons
required
Drive
Detect
Discover
Decide
Target
Subscriber
8
9. Drive
Take
appropriate
ac1on
to
minimize
losses
due
to
fraud.
Record
ac1ons
and
results
for
future
analysis.
Fraud Analytics and Management using D4
Discover
Collect
historical
fraudulent
pa?erns.
Analyze
historical
data
to
formulate
pa?erns
and
changes
required
to
detect,
and
inves1gate
steps
Decide
Use
background
informa1on,
past
usage,
loca1ons,
subscrip1on,
bill
payments
to
find
if
the
fraudulent
transac1ons
are
associated
with
a
subscrip1on.
Seek
more
data
as
needed.
Raise
an
alarm.
Detect
Detect
in
real
1me
if
a
transac1on
relates
to
a
fraudulent
subscrip1on
(e.g.,
inconsistent
geography
or
usage
in
consecu1ve
transac1ons).
Send
alert
for
further
Inves1ga1on.
Detect
observa,ons
about
a
target
Take
ac,on
in
real
,me
–
when
it
ma8ers
Find
new
targets
by
analyzing
historical
data
Iden,fy
pa8erns
over
,me
and
ac,ons
required
Drive
Detect
Discover
Decide
Target
Subscriber
9
10. WFC Use Cases
Business Capability Use Case Name
1
0
Business
Intelligence
Centralized Business Intelligence
EDW/BI Transformation
Competitive Monitoring
Customer
Experience
Management
Effective Customer Care
Apologize for Poor NW exp
Enhance Consumer Billing Reports
CX for Roamers
Personalized Experience
Best Video Experience
Customer Care for VIP
Quality of Experience for Apps
Service Trouble Shooting at C-Center
Cross-channel
Optimization
Channel Optimization
Lead Management
Sales and Support Integration
Shopping Carts and Lists
Management
Insight Analytics
Data usage Patterns
Multi-Sim Behavior
Online Behavior – Trending
Behaviors
Online Behavior – website Analysis
Roamer Behavior
Voice/SMS Apps impact on
Traditional voice and sms services
Tethering Behaviors
Business
Capability Use Case Name
Data Monetization
Online Market Analysis
Enterprise M2M Proposition
Family Online Protection
Sports Analytics
Effective Advertising
Media Metrics
Monetize market data
3rd Party Advertising Networks
Device
Management
Device Analysis
Device SW version Upgrade
Notifications
Device Migrations Traffic Analysis
Voice and Data Performance
Fault detection/notification/NFF
Dynamic Pricing &
Service Models
Advanced Thetering on Shared
Billing Accounts
Proactive High Speed Access at
Partnered Locations
B-width onDemand at Strategic
Hotspots
Field Service
Management
Agenda Optimization & Resource
Activation at Customer Premises
Fraud
Management
Equipment Fraud
Subscription Fraud
Identity Theft
VoIP Hacking
Dealer Fraud
Business
Capability Use Case Name
Smarter
Campaigns
Improve Conversion rates by Timing offers
effectively
Contract Renewal/Retention/Up/X-sell
Intelligent Data
Services
Content Based Advertising
Content Based third-party Advertising based
on Location
Content provider Advertising
QoS Charging for Content Providers
Location-based
Services
Contextual marketing
Co-Presence
Dynamic AdHoc group
Geo-fence Mktg Propositions
Massive Events
Mobility patterns
Hang out Occupancy/Traversal/Anomaly
Tariff plan
innovation /
optimization
App/Location/Time- based Plans
Tariff Planning
Plans and Add-on Impact on App Usage
Network and
Service
Optimization
Service Usage & Usage Location
Customer Centric NW Monitoring
3G Locked Subscribers
IB roaming performance analysis
Intelligent network policies
Capacity Planning
Subscriber and
Service
Management
Interactive multimedia ticketing
Mobile Real Time Social Campagn
New Business Models- Loyalty
Self Care Multi-Device Family Accounts
11. Content
• Watson Foundation for CSPs (WFC)
• Discovery and Predictive Modeling using SPSS
• Detection and Real-time Analytics using InfoSphere Streams
• Component Integration
• Conclusions
12. Introduction to Subscriber Dimensions from Mobility Analytics
Usage Style
l Heavy Voice
l SMS Mostly
l No Data
Interests
l From DPI
l Webpage analytics
l e.g. Golf, Betting
Quality of Service
l From xDR
l Network
l The Now Factory
Demographics
l Based on usage
patterns
l Websites
l Buddies
Lifestyle
l Commuter
l Homebody
l Night Owl
Preferences
l OTT Messaging
l Travel, Games
l Handset prefs
Preferred
Locations
l Hangouts
l Home Work
l Mode of Travel
Best Buddies
l Who calls who
l Who hangs out with
Who?
13. How to turn streaming noisy Telco Location data into meaningful location, then
discover customer insights
Stream data
Call Detail
Records
SMS Voice
GPS Tracking
Reference Data
Cell Tower
Wifi AP Maps
GIS, POI
Special Service
Numbers
e.g bank, 1-800
Analyzable Location Event
Data
Who, when, where and what
subscriberId:
Timestamp:
Position: latitude +
longitude
Precision: 0~2 km
Direction: nullable
Speed: nullable
Activity : nullable
Meaningful Location
subscriberId:
home:
Work:
POIs & period …
Sequence of
meaningful
Locations…
Commute means: car/
subway/bus
Micro segmentaton
Business traveler
Regular commuter
Heavy driver
Social Butterfly
Mom
…..
Location Patterns on
Individual and Group
level
ü Every Sunday
noon, Bob goes to
xxx mall to shopping
and has lunch
ü Every Thursday
afternoon, Bob goes
to customer site at
XXX
ü …..
Mobile Location Data
Processing: Map mapping,
Business rules et.
Big Data
Integration
Spatio-Temporal Event
Association Analysis
Wifi off load
Location Pattern Analytics
14. Discovery using structured data
• A typical discovery uses statistical tools to identify pattern in data.
• Discovery may contribute new derived attributes for further analysis or reporting.
Night Owls at Night
Delivery People
During the Day
Quiet Weekday people
go for dinner on weekends
Almost no Homebodies any time
15. Mobility Lifestyles (developed by IBM)
* from the Television show, “Cheers”. Norm was an accountant who went to the same
pub every night
16. Mobility Lifestyles
• How do the lifestyles of subscribers vary by location and time
of day
• Why do lifestyles matter for Retailers?
§ Certain lifestyles tend follow habits much more than others: Daily
Grinders and Homebodies go to the same locations often and
are predictable. The other lifestyles tend to be less predictable
17. Mobility and Usage Lifestyles
Very distinct patterns, this level of differential is high
21. Content
• Watson Foundation for CSPs (WFC)
• Discovery and Predictive Modeling using SPSS
• Detection and Real-time Analytics using InfoSphere
Streams
• Component Integration
• Conclusions
22. Streams Data Processing in Telco Environment
Decoding, filtering, aggregation, correlation,
summation, transformation, formatting, ...
Streams Telco
Realtime Processing
Performance Data
CDRs
Logs
Event Data
Configuration Data
Telco Network Elements
Source data format: ASN.1, XML, ASCII, binary
Standardized or proprietary, via edge adapters
Output into dashboards, databases, files
Statistics, monitoring, archiving
Tap into Message Transfer
Telco Solutions
23. Visual Representation
A New Paradigm: In-Motion analytics for High throughput and Ultra-low latencies
Continuous Ingestion Continuous Queries / Analytics of data in
motion
Streams Application
Data Sink
Data Tuple Operator
Data Sources
InfoSphere Streams Overview
24. From Vast Data to Actionable Insights
InfoSphere Streams addresses the challenge for CSPs is to turn the vast
amounts of customer data they collect into usable and actionable insight
Millions
of
events
per
second
CDRs
Billing
CRM
Location
Account Mgt
Internet
Network
Dropped
Calls
Outgoing
Interna,onal
Calls
Call
Dura,on
Extra
Call
Invoice
Issued
Invoice
Paid
Contract
Expira,on
Acquired
new
products
Change
contracts
Entered
new
cell
Customer
is
roaming
Customer
is
at
home
New
Top-‐Up
5
minutes
leM
on
pre-‐paid
Changed
Home
Loca,on
Brand
Reputa,on
Customer
Sen,ment
from
Social
network
Broadband
Satura,on
Congested
Cells
Streams
of
Intelligence
3
dropped
calls
in
10
minutes
Customer
is
close
to
a
store
Customer
enters
a
shopping
area
Invoice
paid
+
‘liked’
compe,tor
Smart
phone
browsing
pa8ern
Customer
is
watching
a
video
Microsecond
Latency
Required
Ac,onable
Insight
Who is THIS
customer and what
does S/HE want?
MDM,
EDW
25. Possible Architecture (for live system)
Network/2 Internet Analytics 1 Data Collection
TAP
IBM
CONFIDENTIAL
Filter
(Brocade, Gigamon, etc)
Load Balance
(Brocade, Gigamon, etc)
ISP
Blade 1
Blade 2
Blade 3
Blade 4
Blade 5
Blade 6
Blade 7
Blade 8
Blade 9
Blade 10
Blade …
Each line
represents a link
to a physical
network interface
on a blade. It
carries data for
one or more pre-specified
protocols. All
packets
belonging to one
session are sent
on one link.
Netezza
26. Integration with Discovery
1. Data Collection and Pattern identification.
2. The offline modeling step- using the SPSS Modeler- creates analytic models based on labeled
training data. The data can be hosted on any platform example: data warehouse, Pure Data Systems
for Analytics, Hadoop.
3. The intermediate integration step - There are two alternatives to deploy SPSS models in streams.
One is to generate the PMML model. There is a limited set of models that generate the PMML
format. The PMML model is then deployed in the Streams mining toolkit. The other approach is to
publish the model to generate the .pim, .par and .xml files which are supported by the SPSS Analtyics
Toolkit for Streams. These files are then configured on the SPSS Modeler Solution Publisher.
4. The on-line phase– Using Sreams SPL (Streams Processing Language), streams developer further
uses appropriate operators and input output definitions of the models. This enables realtime
analytics.
5. The action is triggered for instance executing a mobile campaign when a defined threshold is
interpreted in real time.
27. InfoSphere
Streams
Publish the Model
SPSS Scoring
node
NETEZZA
Published
.pim,
.par,
.xml
files
generated
from
SPSS
IBM SPSS
Modeler Solution
Publisher In-‐database
mining
During the offline phase, SPSS Modeler accesses the training data residing in the Pure
Data System for Analytics and creates the Model Nugget. The Modeler ODBC node can
access the database table’s definitions as well as the data, and retrieves the relevant
training data according to the selection criteria. Once trained, the Modeler creates a
Model Nugget which can be published to Streams.
28. DB2
InfoSphere
Streams
UNICA
Campaign
polling
Worklight
Server Mobile
App
IBM SPSS
Modeler Solution
Publisher
Model execution with Streaming data
29. Content
• Watson Foundation for CSPs (WFC)
• Discovery and Predictive Modeling using SPSS
• Detection and Real-time Analytics using InfoSphere Streams
• Component Integration
• Conclusions
30. WFC Application Architecture A
B
C
D
G
AAP Capabilities
High Performance Historical analysis
Model Based Predictive Analytics
Real-time scoring, classification,
detection and action
Visualize, explore, investigate, search
and report
High Performance
Unstructured Data
analysis
Discovery Analytics
Take action on
analytics
F
Simulation
Dashboards
Information
Interaction
Streaming Data Categorize,
Count,
Focus
Score,
Decide
Streaming Engine
Analytics
Engine
Prediction / Policy Engine
Sense,
Identify,
Align
Reports
Geo/
Semantic
Mapping
Outcome
Optimization
Model
Creation
Semi
Structured
Data
Data Repositories
Network
Events
Network
Policies
Continuous Feed
Sources
XDR
Batch
Data
Data for
Historical
Analysis
Deploy Model
Historical
Data
Models
In Database Mining
Reports &
Dashboards
Ad-hoc
Queries
Campaign Mgmt.
Pro-active
Customer
Experience
Management
Pro-active
Network Mgmt
Real time Scoring
& Decision Mgmt.
...
Actions
Event
Execution
Policy
Mgmt
External
Data
Social
3rd party
High Velocity
High Volume
Open API
Customer
Activities
A
C
B
D G
Customer Care
Customer Care
Marketing
Marketing
Network
Planning
...
Network
Planning
...
NOC/SOC
NOC/SOC
Users
Deploy Model
Policy
Management
Standardize
Deduplicate
Identity
Resolution
Data Integration
ETL
Network
Topology
Data
Application
& Usage
Data
Customer
Data
Capture
Changes
Un-
Structured
Data
Hadoop
E
E
Structured
Data
Search, Pattern Matching, Quantitative, Qualitative
F Insigh t
EDW
Advanced Analytics Platform
Create & Deliver
Smarter Services Transform Operations Build Smarter
Networks
Personalize Customer
Engagements
Database Server
31. A
B
C
D
G
AAP Capabilities
High Performance Historical analysis
Model Based Predictive Analytics
Real-time scoring, classification,
detection and action
Visualize, explore, investigate, search
and report
High Performance
Unstructured Data
analysis
Discovery Analytics
Take action on
analytics
F
Database Server SPSS
Simulation
Dashboards
Information
Interaction
Streaming Data Categorize,
Count,
Focus
Score,
Decide
Streaming Engine
Analytics
Engine
Prediction / Policy Engine
Sense,
Identify,
Align
Reports
Geo/
Semantic
Mapping
Outcome
Optimization
Model
Creation
Semi
Structured
Data
Data Repositories
Network
Events
Network
Policies
Continuous Feed
Sources
XDR
Batch
Data
Data for
Historical
Analysis
Deploy Model
Historical
Data
Models
In Database Mining
Reports &
Dashboards
Ad-hoc
Queries
Campaign Mgmt.
Pro-active
Customer
Experience
Management
Pro-active
Network Mgmt
Real time Scoring
& Decision Mgmt.
...
Actions
Event
Execution
Policy
Mgmt
External
Data
Social
3rd party
High Velocity
High Volume
Open API
Customer
Activities
A
C
B
D G
Customer Care
Customer Care
Marketing
Marketing
Network
Planning
...
Network
Planning
...
NOC/SOC
NOC/SOC
Users
Deploy Model
Policy
Management
Standardize
Deduplicate
Identity
Resolution
Data Integration
ETL
Network
Topology
Data
Application
& Usage
Data
Customer
Data
Capture
Changes
Un-
Structured
Data
Hadoop
E
E
Structured
Data
Search, Pattern Matching, Quantitative, Qualitative
F Insigh t
Enterprise Data
Warehouse
Advanced Analytics Platform
Create & Deliver
Smarter Services Transform Operations Build Smarter
Networks
Personalize Customer
Engagements
InfoSphere Streams
SPSS
ODM, Optim,
Open Pages
PDA
Social Media
Analytics
Watson
Explorer
Cognos
InfoSphere
BigInsights
IBM
(Unica)
Campaign
ODM
PDOA
BPM
TNF SourceWorks TNF Smart Works
Watson
Analytics
WFC Application Architecture
using IBM products
InfoServer
EA
33. Content
• Watson Foundation for CSPs (WFC)
• Discovery and Predictive Modeling using SPSS
• Detection and Real-time Analytics using InfoSphere Streams
• Component Integration
• Conclusions
34. Reading Material
• IBM Developer Works
§ Explore the advanced analytics platform, Part 1: Support your business requirements using big
data and advanced analytics
§ Explore the advanced analytics platform, Part 2: Explore use cases that cross multiple
industries using the advanced analytics platform
§ Explore the advanced analytics platform, Part 3: Analyze unstructured text using patterns
§ Explore the advanced analytics platform, Part 4: Analyze location data to determine movement
patterns using a mobility profile pattern
§ Explore the advanced analytics platform, Part 5: Deep dive into discovery and visualization
§ Explore the advanced analytics platform, Part 6: Dive into orchestration with a combination of
SPSS, Operational Decision Management (ODM), and Streams using care and fraud
management case studies
• IBM Data Magazine
§ Mining Data in a High-Performance Sandbox - Fulfill data analysts’ dreams with data
warehouse appliances for in-database analytics and data mining
§ Target Behavior in Real Time for Effective Outcomes: Part 1 - How real-time, adaptive
architectures can drive management decisions for specific use cases
§ Target Behavior in Real Time for Effective Outcomes: Part 2 Drive marketing and business
management decisions using a real-time, adaptive architecture
• Books
§ Big Data Analytics: Disruptive Technologies for Changing the Game
§ Engaging Customers Using Big Data: How Marketing Analytics Are Transforming Business
35. We Value Your Feedback!
• Don’t forget to submit your Insight session and speaker feedback!
Your feedback is very important to us – we use it to continually
improve the conference.
• Access the Insight Conference Connect tool to quickly submit your
surveys from your smartphone, laptop or conference kiosk.
35