SlideShare a Scribd company logo
1 of 51
Tim Bass, CISSP Director, Principal Global Architect Emerging Technologies Group Complex Event Processing (CEP) for Next-Generation Security Event Management, Fraud and Intrusion Detection  April 17, 2007 (First Draft) London
Our Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object]
Who We Are and What We Do We help our customers… Improve  operational visibility, collaboration and ability to be proactive Increase  operational   efficiency and effectiveness Accelerate  projects, initiatives  and go-to-market cycles A leading provider of  business integration and process management software.
How TIBCO Delivers for Customers Accelerate projects, initiatives, and  go-to-market cycles Increase operational efficiency and effectiveness. Improve  operational  visibility, security, collaboration and responsiveness
TIBCO is Trusted by Thousands of Companies ,[object Object],* By annual revenues except for investment banking which is measured by assets Retail Banking  — 17 of top 20 Consumer Package Goods  — 5 of top 10 Energy  — 5 of top 10 Hi-Tech Manufacturing  — 15 of top 20 Investment Banking  — 9 of top 10 Manufacturing (non High-tech)  — 5 of top 10 Pharmaceutical  — 6 of top 10 Telecommunications  — 8 of top 10 Transportation  — 4 of top 10
TIBCO History and Acquisitions  IPO 1999 eXtensibility InConcert Staffware TIBCO Today Teknekron 2000 2002 2001 2003 2004 2005 2005 ,[object Object],[object Object],[object Object],[object Object],[object Object],Acquired by Reuters Est. 1980s Palo Alto Campus Est. 1997 2004
TIBCO Runs a Strong and Viable Business 14 consecutive quarters of yr/yr total revenue growth ,[object Object],[object Object],[object Object]
Revenue Numbers FY 2004 – 2006  (in thousands of dollars) 15.8% $61,060 $73,715 $387,220 FY2004 16.4% $73,127 $67,081 $445,910 FY2005 16.6% $85,923   $90,558  $517,279   FY 2006 R&D SPEND AS A % OF REVENUE R&D SPEND PRE-TAX PROFIT REVENUE
Our Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object]
PredictiveBusiness TM Source:  Ranadiv é , V.,  The Power to Predict , 2006.
Complex Event Processing  " Events in several forms, from simple events to complex events, will become very widely used in business applications during 2004 through 2008 "  --- Gartner July 2003
What is Complex Event Processing? Detecting Threats and Opportunities with PredictiveBusiness®
When Do You Need to Think About CEP? ,[object Object],[object Object],[object Object],[object Object],[object Object],The Power of Events , Addison Wesley, ISBN: 0-201-72789-7, 2002
Bloor Report on Event Processing Event Processing and Decision Making Automated Operational Decisions   Automated Predictive Decisions   Human Predictive Decisions   Human Operational Decisions   Decision Latency   Event Complexity   Process Complexity   Pattern Matching and Inferencing   Anti-Money Laundering   Credit-Card Fraud Exchange Compliance Database Monitoring Algorithmic Trading Trade Desk Monitoring Customer Interaction Order Routing RFID Tariff Look-Up Rail Networks Search & Rescue Baggage Handling Liquidity Management
Our Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object]
[object Object],[object Object],[object Object],[object Object],[object Object],Industry and Business Drivers A Sample of the Problems with Network Security and Fraud Detection
[object Object],Detection-Oriented Systems - Design Goals What are the overall design goals for detection systems?  (Illustrative Purposes Only)
Classification of Intrusion and Fraud Detection Systems Traditional View Before Data Fusion Approach to FDS and IDS Distributed Fraud and Intrusion Detection Systems, Logs Detection Approach Systems Protected Architecture Data Sources Analysis Timing Detection Actions HIDS NIDS Hybrid Audit Logs Net Traffic System Stats Real Time Data Mining Anomaly Detection Signature Detection Centralized Distributed Active Passive Agent Based Security “Stovepipes” Centralized
Intrusion Detection and Data Fusion (2000) Next-Generation Intrusion Detection Systems Source:  Bass, T., CACM, 2000
PredictiveBusiness TM
A Business Optimization Perspective What Classes of Rule-Based Problems Do Businesses Need to Solve? Rule-Based ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Detection Prediction Scheduling ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Example PredictiveBusiness® Applications
Emerging Event-Decision Architecture Customer Profiles Purpose-Built Analytics Secure, Distributed Messaging Backbone Internet/Extranet  Sensors Human  Sensors Edge/POC Sensors Operations Center  Other Reference  Data Rule-Based Event Processors
Complex Event Processing Reference Architecture Next-Generation Functional Architecture for Fraud and Intrusion Detection 24 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL .  .  .  LEVEL ONE EVENT TRACKING Visualization, BAM, User Interaction CEP Reference Architecture DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION DETECTION LEVEL THREE PREDICTIVE ANALYSIS LEVEL FOUR ADAPTIVE BPM
CEP – Situation Detection Hierarchy 22 Adapted from: Waltz, E. & Llinas, J., Multisensor Data Fusion, 1990 Impact Assessment Situational Assessment Relationship of Events Identify Events Location, Times and Rates of Events of Interest Existence of Possible Event of Interest Data/Event Cloud Analysis of Situation & Plans Contextual and Causal  Analysis, Rules Causal Analysis, Bayesian Belief Networks, Rules, NNs, Correlation, State Estimation, Classification Use of Distributed Sensors for Estimations Raw Sensor Data (Passive and Active) HIGH LOW MED
CEP High Level Architecture 22 Adapted from:  Engelmore, R. S., Morgan, A.J., & and Nii, H. P., Blackboard Systems, 1988 & Luckham, D., The Power of Events, 2002 EVENT CLOUD (DISTRIBUTED DATA SET) KS KS KS KS KS KS KS KS KS KS KS KS KS KS
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],HLA - Knowledge Sources KS KS KS
Complex Event Processing Reference Architecture Next-Generation Functional Architecture for Fraud and Intrusion Detection 24 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL .  .  .  LEVEL ONE EVENT TRACKING Visualization, BAM, User Interaction CEP Reference Architecture DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION DETECTION LEVEL THREE PREDICTIVE ANALYSIS LEVEL FOUR ADAPTIVE BPM
Structured Processing for Event-Decision ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Level of  Inference Low Med High
CEP Level 0 –  Event Preprocessing  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CEP Level 1 – Event Refinement  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CEP Level 2 – Situation Refinement ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CEP Level 3 – Impact Assessment ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CEP Level 4 – Process Refinement  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Database Management Examples ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
User Interface / Interaction ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Business Optimization Summary A Simplified View of the CEP Reference Architecture Flexible SOA and Event-Driven Architecture
Our Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object]
TIBCO’s Real-Time Agent-Based SEM Approach A Multisensor Data Fusion Approach to Security Event Management Distributed Fraud and Intrusion Detection Systems, Logs Detection Approach Systems Protected Architecture Data Sources Analysis Timing Detection Actions HIDS NIDS Hybrid Audit Logs Net Traffic System Stats Real Time Data Mining Anomaly Detection Signature Detection Centralized Distributed Active Passive Agent Based Enterprise Correlation  of Security Events
Security Event Management High Level Event-Driven Architecture (EDA) for SEM (CEP and BPM) JAVA  MESSAGING SERVICE  (JMS) DISTRIBUTED EVENTS (TIBCO EMS) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) SENSOR NETWORK RULES NETWORK FDS BW JMS LOGFILE JMS BW LOGFILE JMS BW LOGFILE JMS BW IDS JMS BW FDS JMS BW SQL DB BW JMS ADB SQL DB BW JMS ADB MESSAGING NETWORK SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM BPM Compliance  Workflow (TIBCO iProcess)
TIBCO BusinessEvents™ Solutions Overview BusinessEvents™ Solutions Space Data: Events & Databases -Real-Time & Historical Data Models: Statistical Financial Optimization Comms: Pub/Sub Messaging Queues Topics UIs Knowledge: Facts & Rules
TIBCO BusinessEvents™ Overview ,[object Object],[object Object],[object Object],[object Object],[object Object],Modeling Tools, Statefulness, Business Rules and Process Integration UML Conceptual UML State Business Rules Business Users Event Analyzer
TIBCO BusinessEvents™ Overview Collection, Normalization Metric of Managed Objects, Normalized Non-Contextual Events Metadata  Repository Event Management, Correlation,  Aggregation, Inference and  Analysis Correlated, Analyzed, Contextual Dialogue Events   Rules, Knowledge,  Patterns, Models Visualization, Reporting, Alert Management Application Interface Feeds Visualization: Detection Metrics Agents Synthetic  Warehouse Visualization: Process View Dialogue Manager Inference Engine FDS/IDS Logfiles Edge Devs Semantic Model Events Rules Design Environment State Model Sensors
TIBCO BusinessEvents™ Awards 2006 Best Complex Event Processing Software Winner: TIBCO 2006 Event Processing   General Purpose  Gold Award Winner
CEP and BusinessEvents™ Case Study: Real-Time On-Line Fraud Detection Requirements ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
On-Line Fraud Detection Use Case   Architecture and Capacity Planning Approx. 12,000 Hits Per Second During Peak Period Across the Three Sites – One Instance Of TIBCO BusinessEvents™ Capable of Handling Maximum Hits  Overall 100 Million Hits Handled Between 3PM – 4 PM   Peak Approx. 250 Million Hits Per Day Across the Three Sites TIBCO   EMS™ TIBCO   Business Events™ Session Info Three Server Farms  ~600-700 Application Servers
Characteristics of Solutions Architecture ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Potential Extensions to Solutions Architecture ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
TIBCO SOA and BPM Architecture
Key Takeaways ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Our Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object]
Thank You! Tim Bass, CISSP Director, Principal Global Architect Emerging Technologies Group [email_address] Event Processing at TIBCO

More Related Content

Viewers also liked

Developing Distributed Web Applications, Where does REST fit in?
Developing Distributed Web Applications, Where does REST fit in?Developing Distributed Web Applications, Where does REST fit in?
Developing Distributed Web Applications, Where does REST fit in?
Srinath Perera
 
Event Management System Document
Event Management System Document Event Management System Document
Event Management System Document
LJ PROJECTS
 
WSO2 Mashups and BPM
WSO2 Mashups and BPMWSO2 Mashups and BPM
WSO2 Mashups and BPM
WSO2
 
Extending Spark Streaming to Support Complex Event Processing
Extending Spark Streaming to Support Complex Event ProcessingExtending Spark Streaming to Support Complex Event Processing
Extending Spark Streaming to Support Complex Event Processing
Oh Chan Kwon
 

Viewers also liked (20)

Complex Event Processing - A brief overview
Complex Event Processing - A brief overviewComplex Event Processing - A brief overview
Complex Event Processing - A brief overview
 
Complex Event Processing in Practice at jDays 2012
Complex Event Processing in Practice at jDays 2012Complex Event Processing in Practice at jDays 2012
Complex Event Processing in Practice at jDays 2012
 
Semantic Complex Event Processing
Semantic Complex Event ProcessingSemantic Complex Event Processing
Semantic Complex Event Processing
 
Complex Event Processing: What?, Why?, How?
Complex Event Processing: What?, Why?, How?Complex Event Processing: What?, Why?, How?
Complex Event Processing: What?, Why?, How?
 
Developing Distributed Web Applications, Where does REST fit in?
Developing Distributed Web Applications, Where does REST fit in?Developing Distributed Web Applications, Where does REST fit in?
Developing Distributed Web Applications, Where does REST fit in?
 
Esper - CEP Engine
Esper - CEP EngineEsper - CEP Engine
Esper - CEP Engine
 
Complex Event Processing with Esper
Complex Event Processing with EsperComplex Event Processing with Esper
Complex Event Processing with Esper
 
Semantic Complex Event Processing at Sem Tech 2010
Semantic Complex Event Processing at Sem Tech 2010Semantic Complex Event Processing at Sem Tech 2010
Semantic Complex Event Processing at Sem Tech 2010
 
Complex Event Processing with Esper
Complex Event Processing with EsperComplex Event Processing with Esper
Complex Event Processing with Esper
 
Event Management System Document
Event Management System Document Event Management System Document
Event Management System Document
 
Social media for event management
Social media for event managementSocial media for event management
Social media for event management
 
SOA-based Business Integration with Eclipse BPEL and Apache ODE
SOA-based Business Integration with Eclipse BPEL and Apache ODESOA-based Business Integration with Eclipse BPEL and Apache ODE
SOA-based Business Integration with Eclipse BPEL and Apache ODE
 
WSO2 Mashups and BPM
WSO2 Mashups and BPMWSO2 Mashups and BPM
WSO2 Mashups and BPM
 
The Future of Real-Time in Spark
The Future of Real-Time in SparkThe Future of Real-Time in Spark
The Future of Real-Time in Spark
 
WSO2 Business Process Server 3.5.0 - Product Overview
WSO2 Business Process Server 3.5.0 - Product OverviewWSO2 Business Process Server 3.5.0 - Product Overview
WSO2 Business Process Server 3.5.0 - Product Overview
 
CEP: from Esper back to Akka
CEP: from Esper back to AkkaCEP: from Esper back to Akka
CEP: from Esper back to Akka
 
Spark Summit San Francisco 2016 - Ali Ghodsi Keynote
Spark Summit San Francisco 2016 - Ali Ghodsi KeynoteSpark Summit San Francisco 2016 - Ali Ghodsi Keynote
Spark Summit San Francisco 2016 - Ali Ghodsi Keynote
 
Graph Stream Processing : spinning fast, large scale, complex analytics
Graph Stream Processing : spinning fast, large scale, complex analyticsGraph Stream Processing : spinning fast, large scale, complex analytics
Graph Stream Processing : spinning fast, large scale, complex analytics
 
Camunda BPM 7.2 - English
Camunda BPM 7.2 - EnglishCamunda BPM 7.2 - English
Camunda BPM 7.2 - English
 
Extending Spark Streaming to Support Complex Event Processing
Extending Spark Streaming to Support Complex Event ProcessingExtending Spark Streaming to Support Complex Event Processing
Extending Spark Streaming to Support Complex Event Processing
 

Similar to Complex Event Processing (CEP) for Next-Generation Security Event Management, Fraud and Intrusion Detection

PowerPoint presentation
PowerPoint presentationPowerPoint presentation
PowerPoint presentation
webhostingguy
 
The “Predictive” Battlespace: Leveraging the Power of Event-Driven Architect...
The “Predictive” Battlespace:  Leveraging the Power of Event-Driven Architect...The “Predictive” Battlespace:  Leveraging the Power of Event-Driven Architect...
The “Predictive” Battlespace: Leveraging the Power of Event-Driven Architect...
Nathaniel Palmer
 
The IQ Business Group
The IQ Business GroupThe IQ Business Group
The IQ Business Group
mbeck94
 
The IQ Business Group
The IQ Business GroupThe IQ Business Group
The IQ Business Group
kejensen810
 
Business Intelligence For Aml
Business Intelligence For AmlBusiness Intelligence For Aml
Business Intelligence For Aml
Kartik Mehta
 

Similar to Complex Event Processing (CEP) for Next-Generation Security Event Management, Fraud and Intrusion Detection (20)

CEP and SOA: An Open Event-Driven Architecture for Risk Management
CEP and SOA: An Open Event-Driven Architecture for Risk ManagementCEP and SOA: An Open Event-Driven Architecture for Risk Management
CEP and SOA: An Open Event-Driven Architecture for Risk Management
 
Detecting Opportunities and Threats with Complex Event Processing: Case St...
Detecting Opportunities and Threats with Complex Event Processing: Case St...Detecting Opportunities and Threats with Complex Event Processing: Case St...
Detecting Opportunities and Threats with Complex Event Processing: Case St...
 
Processing Patterns for Predictive Business
Processing Patterns for Predictive BusinessProcessing Patterns for Predictive Business
Processing Patterns for Predictive Business
 
Combating Fraud and Intrusion Threats with Event Processing
Combating Fraud and Intrusion Threats with Event ProcessingCombating Fraud and Intrusion Threats with Event Processing
Combating Fraud and Intrusion Threats with Event Processing
 
Optimizing Your SOA with Event Processing
Optimizing Your SOA with Event ProcessingOptimizing Your SOA with Event Processing
Optimizing Your SOA with Event Processing
 
Using Event Processing to Enable Enterprise Security
Using Event Processing to Enable Enterprise SecurityUsing Event Processing to Enable Enterprise Security
Using Event Processing to Enable Enterprise Security
 
Event Driven Architecture (EDA), November 2, 2006
Event Driven Architecture (EDA), November 2, 2006Event Driven Architecture (EDA), November 2, 2006
Event Driven Architecture (EDA), November 2, 2006
 
Cognitivo - Tackling the enterprise data quality challenge
Cognitivo - Tackling the enterprise data quality challengeCognitivo - Tackling the enterprise data quality challenge
Cognitivo - Tackling the enterprise data quality challenge
 
PowerPoint presentation
PowerPoint presentationPowerPoint presentation
PowerPoint presentation
 
The “Predictive” Battlespace: Leveraging the Power of Event-Driven Architect...
The “Predictive” Battlespace:  Leveraging the Power of Event-Driven Architect...The “Predictive” Battlespace:  Leveraging the Power of Event-Driven Architect...
The “Predictive” Battlespace: Leveraging the Power of Event-Driven Architect...
 
KNIME Meetup 2016-04-16
KNIME Meetup 2016-04-16KNIME Meetup 2016-04-16
KNIME Meetup 2016-04-16
 
Recom Banking Solution
Recom Banking  SolutionRecom Banking  Solution
Recom Banking Solution
 
Business Intelligence For Anti-Money Laundering
Business Intelligence For Anti-Money LaunderingBusiness Intelligence For Anti-Money Laundering
Business Intelligence For Anti-Money Laundering
 
Arvind Mehrotra
Arvind MehrotraArvind Mehrotra
Arvind Mehrotra
 
Real-Time Analytics for Industries
Real-Time Analytics for IndustriesReal-Time Analytics for Industries
Real-Time Analytics for Industries
 
5733 a deep dive into IBM Watson Foundation for CSP (WFC)
5733   a deep dive into IBM Watson Foundation for CSP (WFC)5733   a deep dive into IBM Watson Foundation for CSP (WFC)
5733 a deep dive into IBM Watson Foundation for CSP (WFC)
 
SaaS Vs On Premise BI
SaaS Vs On Premise BISaaS Vs On Premise BI
SaaS Vs On Premise BI
 
The IQ Business Group
The IQ Business GroupThe IQ Business Group
The IQ Business Group
 
The IQ Business Group
The IQ Business GroupThe IQ Business Group
The IQ Business Group
 
Business Intelligence For Aml
Business Intelligence For AmlBusiness Intelligence For Aml
Business Intelligence For Aml
 

More from Tim Bass

More from Tim Bass (11)

A High Level Blackboard Architecture for Cyber SA
A High Level Blackboard Architecture for Cyber SAA High Level Blackboard Architecture for Cyber SA
A High Level Blackboard Architecture for Cyber SA
 
A Journey Into Cyberspace
A Journey Into CyberspaceA Journey Into Cyberspace
A Journey Into Cyberspace
 
Mythbusters: Event Stream Processing v. Complex Event Processing
Mythbusters: Event Stream Processing v. Complex Event ProcessingMythbusters: Event Stream Processing v. Complex Event Processing
Mythbusters: Event Stream Processing v. Complex Event Processing
 
Event Processing Technical Society Event Processing Reference Architecture W...
Event Processing Technical SocietyEvent Processing Reference Architecture W...Event Processing Technical SocietyEvent Processing Reference Architecture W...
Event Processing Technical Society Event Processing Reference Architecture W...
 
Leveraging Business Rules in TIBCO BusinessEvents
Leveraging Business Rules in TIBCO BusinessEventsLeveraging Business Rules in TIBCO BusinessEvents
Leveraging Business Rules in TIBCO BusinessEvents
 
Next-Generation IDS: A CEP Use Case in 10 Minutes
Next-Generation IDS: A CEP Use Case in 10 MinutesNext-Generation IDS: A CEP Use Case in 10 Minutes
Next-Generation IDS: A CEP Use Case in 10 Minutes
 
A Survey of Event Processing Languages (EPLs), October 7, 2006
A Survey of Event Processing Languages (EPLs), October 7, 2006A Survey of Event Processing Languages (EPLs), October 7, 2006
A Survey of Event Processing Languages (EPLs), October 7, 2006
 
Proposed Event Processing Definitions ,September 20, 2006
Proposed Event Processing Definitions,September 20, 2006Proposed Event Processing Definitions,September 20, 2006
Proposed Event Processing Definitions ,September 20, 2006
 
Event Processing Reference Architecture, March 2006
Event Processing Reference Architecture, March 2006Event Processing Reference Architecture, March 2006
Event Processing Reference Architecture, March 2006
 
Using Event Processing to Enable Enterprise Security
Using Event Processing to Enable Enterprise SecurityUsing Event Processing to Enable Enterprise Security
Using Event Processing to Enable Enterprise Security
 
Adding Rules to Improve Flexibility and Effectively Manage Complex Events
Adding Rules to Improve Flexibility and Effectively Manage Complex EventsAdding Rules to Improve Flexibility and Effectively Manage Complex Events
Adding Rules to Improve Flexibility and Effectively Manage Complex Events
 

Recently uploaded

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Recently uploaded (20)

Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu SubbuApidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 

Complex Event Processing (CEP) for Next-Generation Security Event Management, Fraud and Intrusion Detection

  • 1. Tim Bass, CISSP Director, Principal Global Architect Emerging Technologies Group Complex Event Processing (CEP) for Next-Generation Security Event Management, Fraud and Intrusion Detection April 17, 2007 (First Draft) London
  • 2.
  • 3. Who We Are and What We Do We help our customers… Improve operational visibility, collaboration and ability to be proactive Increase operational efficiency and effectiveness Accelerate projects, initiatives and go-to-market cycles A leading provider of business integration and process management software.
  • 4. How TIBCO Delivers for Customers Accelerate projects, initiatives, and go-to-market cycles Increase operational efficiency and effectiveness. Improve operational visibility, security, collaboration and responsiveness
  • 5.
  • 6.
  • 7.
  • 8. Revenue Numbers FY 2004 – 2006 (in thousands of dollars) 15.8% $61,060 $73,715 $387,220 FY2004 16.4% $73,127 $67,081 $445,910 FY2005 16.6% $85,923  $90,558 $517,279  FY 2006 R&D SPEND AS A % OF REVENUE R&D SPEND PRE-TAX PROFIT REVENUE
  • 9.
  • 10. PredictiveBusiness TM Source: Ranadiv é , V., The Power to Predict , 2006.
  • 11. Complex Event Processing " Events in several forms, from simple events to complex events, will become very widely used in business applications during 2004 through 2008 " --- Gartner July 2003
  • 12. What is Complex Event Processing? Detecting Threats and Opportunities with PredictiveBusiness®
  • 13.
  • 14. Bloor Report on Event Processing Event Processing and Decision Making Automated Operational Decisions Automated Predictive Decisions Human Predictive Decisions Human Operational Decisions Decision Latency Event Complexity Process Complexity Pattern Matching and Inferencing Anti-Money Laundering Credit-Card Fraud Exchange Compliance Database Monitoring Algorithmic Trading Trade Desk Monitoring Customer Interaction Order Routing RFID Tariff Look-Up Rail Networks Search & Rescue Baggage Handling Liquidity Management
  • 15.
  • 16.
  • 17.
  • 18. Classification of Intrusion and Fraud Detection Systems Traditional View Before Data Fusion Approach to FDS and IDS Distributed Fraud and Intrusion Detection Systems, Logs Detection Approach Systems Protected Architecture Data Sources Analysis Timing Detection Actions HIDS NIDS Hybrid Audit Logs Net Traffic System Stats Real Time Data Mining Anomaly Detection Signature Detection Centralized Distributed Active Passive Agent Based Security “Stovepipes” Centralized
  • 19. Intrusion Detection and Data Fusion (2000) Next-Generation Intrusion Detection Systems Source: Bass, T., CACM, 2000
  • 21.
  • 22. Emerging Event-Decision Architecture Customer Profiles Purpose-Built Analytics Secure, Distributed Messaging Backbone Internet/Extranet Sensors Human Sensors Edge/POC Sensors Operations Center Other Reference Data Rule-Based Event Processors
  • 23. Complex Event Processing Reference Architecture Next-Generation Functional Architecture for Fraud and Intrusion Detection 24 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL . . . LEVEL ONE EVENT TRACKING Visualization, BAM, User Interaction CEP Reference Architecture DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION DETECTION LEVEL THREE PREDICTIVE ANALYSIS LEVEL FOUR ADAPTIVE BPM
  • 24. CEP – Situation Detection Hierarchy 22 Adapted from: Waltz, E. & Llinas, J., Multisensor Data Fusion, 1990 Impact Assessment Situational Assessment Relationship of Events Identify Events Location, Times and Rates of Events of Interest Existence of Possible Event of Interest Data/Event Cloud Analysis of Situation & Plans Contextual and Causal Analysis, Rules Causal Analysis, Bayesian Belief Networks, Rules, NNs, Correlation, State Estimation, Classification Use of Distributed Sensors for Estimations Raw Sensor Data (Passive and Active) HIGH LOW MED
  • 25. CEP High Level Architecture 22 Adapted from: Engelmore, R. S., Morgan, A.J., & and Nii, H. P., Blackboard Systems, 1988 & Luckham, D., The Power of Events, 2002 EVENT CLOUD (DISTRIBUTED DATA SET) KS KS KS KS KS KS KS KS KS KS KS KS KS KS
  • 26.
  • 27. Complex Event Processing Reference Architecture Next-Generation Functional Architecture for Fraud and Intrusion Detection 24 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL . . . LEVEL ONE EVENT TRACKING Visualization, BAM, User Interaction CEP Reference Architecture DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION DETECTION LEVEL THREE PREDICTIVE ANALYSIS LEVEL FOUR ADAPTIVE BPM
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36. Business Optimization Summary A Simplified View of the CEP Reference Architecture Flexible SOA and Event-Driven Architecture
  • 37.
  • 38. TIBCO’s Real-Time Agent-Based SEM Approach A Multisensor Data Fusion Approach to Security Event Management Distributed Fraud and Intrusion Detection Systems, Logs Detection Approach Systems Protected Architecture Data Sources Analysis Timing Detection Actions HIDS NIDS Hybrid Audit Logs Net Traffic System Stats Real Time Data Mining Anomaly Detection Signature Detection Centralized Distributed Active Passive Agent Based Enterprise Correlation of Security Events
  • 39. Security Event Management High Level Event-Driven Architecture (EDA) for SEM (CEP and BPM) JAVA MESSAGING SERVICE (JMS) DISTRIBUTED EVENTS (TIBCO EMS) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) SENSOR NETWORK RULES NETWORK FDS BW JMS LOGFILE JMS BW LOGFILE JMS BW LOGFILE JMS BW IDS JMS BW FDS JMS BW SQL DB BW JMS ADB SQL DB BW JMS ADB MESSAGING NETWORK SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM BPM Compliance Workflow (TIBCO iProcess)
  • 40. TIBCO BusinessEvents™ Solutions Overview BusinessEvents™ Solutions Space Data: Events & Databases -Real-Time & Historical Data Models: Statistical Financial Optimization Comms: Pub/Sub Messaging Queues Topics UIs Knowledge: Facts & Rules
  • 41.
  • 42. TIBCO BusinessEvents™ Overview Collection, Normalization Metric of Managed Objects, Normalized Non-Contextual Events Metadata Repository Event Management, Correlation, Aggregation, Inference and Analysis Correlated, Analyzed, Contextual Dialogue Events Rules, Knowledge, Patterns, Models Visualization, Reporting, Alert Management Application Interface Feeds Visualization: Detection Metrics Agents Synthetic Warehouse Visualization: Process View Dialogue Manager Inference Engine FDS/IDS Logfiles Edge Devs Semantic Model Events Rules Design Environment State Model Sensors
  • 43. TIBCO BusinessEvents™ Awards 2006 Best Complex Event Processing Software Winner: TIBCO 2006 Event Processing General Purpose Gold Award Winner
  • 44.
  • 45. On-Line Fraud Detection Use Case Architecture and Capacity Planning Approx. 12,000 Hits Per Second During Peak Period Across the Three Sites – One Instance Of TIBCO BusinessEvents™ Capable of Handling Maximum Hits Overall 100 Million Hits Handled Between 3PM – 4 PM Peak Approx. 250 Million Hits Per Day Across the Three Sites TIBCO EMS™ TIBCO Business Events™ Session Info Three Server Farms ~600-700 Application Servers
  • 46.
  • 47.
  • 48. TIBCO SOA and BPM Architecture
  • 49.
  • 50.
  • 51. Thank You! Tim Bass, CISSP Director, Principal Global Architect Emerging Technologies Group [email_address] Event Processing at TIBCO

Editor's Notes

  1. What do we mean by Real-Time Business?