SlideShare a Scribd company logo
1 of 36
Download to read offline
Raffael Marty, CEO
Creating Your Own Threat Intel
Through Hunting & Visualization
Tenerife, Spain
February, 2016
Creating Your Own Threat Intel
Through Hunting & Visualization
Raffael Marty, CEO
Security. Analytics. Insight.3
Contents
HUNTING
AKA INTERNAL THREAT INTELLIGENCE
THREAT INTELLIGENCE
A PROCESS AND INFRASTRUCTURE VIEW
1
2 VISUALIZATION
A THREAT INTELLIGENCE GOLD MINE
3
Threat Intelligence
Security. Analytics. Insight.5
• Products / Tools
• Firewall - Blocks traffic based on pre-defined rules
• Web Application Firewall - Monitors for signs of known malicious activity in Web traffic
• Intrusion Prevention System - Looks for ‘signs’ of known attacks in traffic and protocol violations
• Anti Virus - Looks for ‘signs’ of known attacks on the end system
• Malware Sandbox - Runs new binaries and monitors their behavior for malicious signs
• Security Information Management - Uses pre-defined rules to correlate signs from different data
streams to augment intelligence
• Vulnerability Scanning - Searches for known vulnerabilities and vulnerable software
• Rely on pattern matching and signatures based knowledge from the past
• Reactive -> always behind
• Unknown and new threats -> won’t be detected
• ‘Imperfect’ patterns and rules -> cause a lot of false positives
We Are Monitoring - What is Going Wrong?
Defense Has Been Relying
On Past Knowledge
Security. Analytics. Insight.6
Event Funnel - How We Used To Do It
data
rule-based

correlation
prioritization
simple

statistics
attack
candidates
• What rules do you write?
• Do the vendor provided rules work for you?
• How do you define a priority 10 event?
• High false positive rate!
• Unless alerts are VERY focussed
• High false negative rate!
• Do you know what you don’t know?
Security. Analytics. Insight.7
Then Came Threat Intelligence
• How many hits do you really get?
• You are missing most attacks
IOCs
• How do you match
these efficiently
against a real-time
stream?
• How do you de-
duplicate and
normalize these
feeds?
attack
candidates
70–90%
OF MALWARE SAMPLES ARE UNIQUE TO AN ORGANIZATION.
Security. Analytics. Insight.8
Removing the Event Funnel - Hello Data Lake
any 

data
Big Data Lake
Rules
• Storing more, and more diverse data
• Kafka and “dynamic parsing”
• Enabling large-scale processing
• Spark, SparkStreaming, Storm, Parquet
• Using “standard” data access (SQL, REST)
• Plug in any other tool!
context
IOCs
This per-se is not new …
Security. Analytics. Insight.9
Adding Interactive - Analyst Driven Exploration
any 

data
Big Data Lake
Rules
context
IOCs
… but first we get the human in the loop …
Hunting
• interactive visualization
• analyst driven
• machine assisted
Security. Analytics. Insight.10
Hunting Creates Internal Threat Intelligence
any 

data
Big Data Lake
Rules
context
IOCs
… then, let’s rethink our rules … Novel, Advanced Attacks
internal TI
Security. Analytics. Insight.11
Hunting Creates Internal Threat Intelligence
any 

data
Big Data Lake
Rules
context
IOCs
… then, let’s rethink our rules … patterns anyone?
internal TI
Novel, Advanced Attacks
Low False Positive Alerts
Patterns
Security. Analytics. Insight.12
Buzzword Bingo
any 

data
Big Data Lake
Rules
context
IOCs
… and finally, we are buzzword compliant …
behavioral monitoring

scoring
anomaly detection
machine learning
artificial intelligence
“models”
data science
internal TI
Patterns
Security. Analytics. Insight.13
How Does All That Architecture Stuff Matter?
In the following we’ll explore how this all matters …
… but first, let’s see how visualization plays a key role here.
14Visualization
Security. Analytics. Insight.15
“How Can We See, 

Not To Confirm -
But To Learn”
- Edward Tufte
Security. Analytics. Insight.16
Why Visualization?dport
time
Security. Analytics. Insight.17
SELECT count(distinct protocol) FROM flows;
SELECT count(distinct port) FROM flows;
SELECT count(distinct src_network) FROM flows;
SELECT count(distinct dest_network) FROM flows;
SELECT port, count(*) FROM flows GROUP BY port;
SELECT protocol,
count(CASE WHEN flows < 200 THEN 1 END) AS [<200],
count(CASE WHEN flows>= 201 AND flows < 300 THEN 1 END) 

AS [201 - 300],
count(CASE WHEN flows>= 301 AND flows < 350 THEN 1 END) 

AS [301 - 350],
count(CASE WHEN flows>= 351 THEN 1 END) AS [>351]
FROM flows GROUP BY protocol;
SELECT port, count(distinct src_network) FROM flows GROUP BY port;
SELECT src_network, count(distinct dest_network) FROM flows GROUP
BY port;
SELECT src_network, count(distinct dest_network) AS dn, sum(flows)
FROM flows GROUP BY port, dn;
SELECT port, protocol, count(*) FROM flows GROUP BY port, protocol;
SELECT sum(flows), dest_network FROM flows GROUP BY dest_network;
…
One Graph Summarizes Dozens of Queries
port dest_network
protocol src_network flows
Security. Analytics. Insight.18
Visualization To …
Present / Communicate Discover / Explore
Security. Analytics. Insight.19
We will have a look at a couple components from earlier:
• Context
• Data Science
• Clustering
• Seriation - Data Science Gone Wrong
• Time-series Analysis
Analytics Components
Security. Analytics. Insight.20
Did You Know?
Users accessing Sharepoint
servers
User
Sharepoint Server
data processing visualization
This graph of users accessing
sharepoint servers, does not
immediately reveal any interesting
patterns.
Security. Analytics. Insight.21
Did You Know - How Context Tells a Story
Using HR data as context
Remote User
San Francisco Office User
Sharepoint Server
data processing visualization
HR data
Using color to add context to the
graph helps immediately identify
outliers and potential problems.
Security. Analytics. Insight.22
• Simple stuff works!
• dc(dest), dc(d_port)
• What is normal?
• Use data science / data mining to prepare
data. Then visualize the output for human
analyst.
Data Science in Security - Words of Caution
Security. Analytics. Insight.23
Challenges With Clustering Network Traffic
The graph shows an abstract
space with colors being
machine identified clusters.
Hard Questions:
• What are these clusters?
• Do Web servers cluster?
• What are good clusters?
• What’s anomalous?
Security. Analytics. Insight.24
Data Science That Works
threshold
outliers have different magnitudes
Security. Analytics. Insight.25
Approximate Curve
fitting a curve distance to curve
Security. Analytics. Insight.26
Data Mining Applied
better 

threshold
27Hunting
Security. Analytics. Insight.28
Hunting - Ready, Fire, Aim
• Analysts are your best and most expensive resource
• They need the right tools and data
• Speed (see earlier architecture)
• Interaction (visual!)
• Machine-assisted insight
Examples
• Exploring DNS traffic
• High business impact machine analysis
• Lateral movement
Security. Analytics. Insight.29
HBI Metric Analysis
Visually learn, Test, Automate
Security. Analytics. Insight.30
HBI Metric Analysis - If you like Black Backgrounds
Security. Analytics. Insight.31
We have tried many thing:
• Social Network Analysis
• Seasonality detection
• Entropy over time
• Frequent pattern mining
• Clustering
All kinds of challenges.
Simple works!
Let’s Get Mathematical
U−matrix
4.28e−05
0.0461
0.0921
Security. Analytics. Insight.32
Simple - Data Abstraction
Security. Analytics. Insight.33
Lateral Movement - Cross Network Communications
Challenges
• Scale
• You will find one of everything
• Defining white-lists and
keeping them up to date (i.e.,
network and asset hygiene)
VPN
DMZ
Office
GIA
Unknown
Internet
AWS
Security. Analytics. Insight.34
http://secviz.org
List: secviz.org/mailinglist
Twitter: @secviz
Share, discuss, challenge, and learn about security visualization.
Security Visualization Community
Security. Analytics. Insight.35
BlackHat Workshop
Visual Analytics
Delivering Actionable Security
Intelligence
July 30,31 & August 1,2 - Las Vegas, USA
big data | analytics | visualization
http://secviz.org
Security. Analytics. Insight.36
After some exploration …
raffael.marty@pixlcloud.com
http://slideshare.net/zrlram
http://secviz.org and @secviz
Further resources:

More Related Content

What's hot

Anti forensic
Anti forensicAnti forensic
Anti forensic
Milap Oza
 
Starting Over with Sub-Techniques
Starting Over with Sub-TechniquesStarting Over with Sub-Techniques
Starting Over with Sub-Techniques
MITRE - ATT&CKcon
 

What's hot (20)

Digital forensic
Digital forensicDigital forensic
Digital forensic
 
Threat Intelligence
Threat IntelligenceThreat Intelligence
Threat Intelligence
 
Anti forensic
Anti forensicAnti forensic
Anti forensic
 
Cyber Threat Hunting Workshop
Cyber Threat Hunting WorkshopCyber Threat Hunting Workshop
Cyber Threat Hunting Workshop
 
Windows Threat Hunting
Windows Threat HuntingWindows Threat Hunting
Windows Threat Hunting
 
Threat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement MatriceThreat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement Matrice
 
Cyber Threat Intelligence
Cyber Threat IntelligenceCyber Threat Intelligence
Cyber Threat Intelligence
 
Starting Over with Sub-Techniques
Starting Over with Sub-TechniquesStarting Over with Sub-Techniques
Starting Over with Sub-Techniques
 
OpenSourceIntelligence-OSINT.pptx
OpenSourceIntelligence-OSINT.pptxOpenSourceIntelligence-OSINT.pptx
OpenSourceIntelligence-OSINT.pptx
 
Web Hacking
Web HackingWeb Hacking
Web Hacking
 
Security Operation Center Fundamental
Security Operation Center FundamentalSecurity Operation Center Fundamental
Security Operation Center Fundamental
 
User Behavior Analytics Using Machine Learning
User Behavior Analytics Using Machine LearningUser Behavior Analytics Using Machine Learning
User Behavior Analytics Using Machine Learning
 
Transforming Adversary Emulation Into a Data Analysis Question
Transforming Adversary Emulation Into a Data Analysis QuestionTransforming Adversary Emulation Into a Data Analysis Question
Transforming Adversary Emulation Into a Data Analysis Question
 
Measure What Matters: How to Use MITRE ATTACK to do the Right Things in the R...
Measure What Matters: How to Use MITRE ATTACK to do the Right Things in the R...Measure What Matters: How to Use MITRE ATTACK to do the Right Things in the R...
Measure What Matters: How to Use MITRE ATTACK to do the Right Things in the R...
 
Windows forensic artifacts
Windows forensic artifactsWindows forensic artifacts
Windows forensic artifacts
 
ATT&CK Updates- ATT&CK's Open Source
ATT&CK Updates- ATT&CK's Open SourceATT&CK Updates- ATT&CK's Open Source
ATT&CK Updates- ATT&CK's Open Source
 
OSINT
OSINTOSINT
OSINT
 
Threat Hunting
Threat HuntingThreat Hunting
Threat Hunting
 
Ce hv6 module 57 computer forensics and incident handling
Ce hv6 module 57 computer forensics and incident handlingCe hv6 module 57 computer forensics and incident handling
Ce hv6 module 57 computer forensics and incident handling
 
ATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat IntelligenceATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat Intelligence
 

Similar to Creating Your Own Threat Intel Through Hunting & Visualization

Novetta Cyber Analytics
Novetta Cyber AnalyticsNovetta Cyber Analytics
Novetta Cyber Analytics
Novetta
 
CyberCrime in the Cloud and How to defend Yourself
CyberCrime in the Cloud and How to defend Yourself CyberCrime in the Cloud and How to defend Yourself
CyberCrime in the Cloud and How to defend Yourself
Alert Logic
 

Similar to Creating Your Own Threat Intel Through Hunting & Visualization (20)

Protecting Financial Networks from Cyber Crime
Protecting Financial Networks from Cyber CrimeProtecting Financial Networks from Cyber Crime
Protecting Financial Networks from Cyber Crime
 
RIoT (Raiding Internet of Things) by Jacob Holcomb
RIoT  (Raiding Internet of Things)  by Jacob HolcombRIoT  (Raiding Internet of Things)  by Jacob Holcomb
RIoT (Raiding Internet of Things) by Jacob Holcomb
 
Splunk Enterpise for Information Security Hands-On
Splunk Enterpise for Information Security Hands-OnSplunk Enterpise for Information Security Hands-On
Splunk Enterpise for Information Security Hands-On
 
Cybersecurity - Jim Butterworth
Cybersecurity - Jim ButterworthCybersecurity - Jim Butterworth
Cybersecurity - Jim Butterworth
 
Splunk Enterprise for InfoSec Hands-On Breakout Session
Splunk Enterprise for InfoSec Hands-On Breakout SessionSplunk Enterprise for InfoSec Hands-On Breakout Session
Splunk Enterprise for InfoSec Hands-On Breakout Session
 
Understanding Intrusion Detection & Prevention Systems (1).pptx
Understanding Intrusion Detection & Prevention Systems (1).pptxUnderstanding Intrusion Detection & Prevention Systems (1).pptx
Understanding Intrusion Detection & Prevention Systems (1).pptx
 
Using Splunk for Information Security
Using Splunk for Information SecurityUsing Splunk for Information Security
Using Splunk for Information Security
 
Using Splunk for Information Security
Using Splunk for Information SecurityUsing Splunk for Information Security
Using Splunk for Information Security
 
Visualization in the Age of Big Data
Visualization in the Age of Big DataVisualization in the Age of Big Data
Visualization in the Age of Big Data
 
Discover advanced threats with threat intelligence - Jeremy Li
Discover advanced threats with threat intelligence - Jeremy LiDiscover advanced threats with threat intelligence - Jeremy Li
Discover advanced threats with threat intelligence - Jeremy Li
 
AI on Spark for Malware Analysis and Anomalous Threat Detection
AI on Spark for Malware Analysis and Anomalous Threat DetectionAI on Spark for Malware Analysis and Anomalous Threat Detection
AI on Spark for Malware Analysis and Anomalous Threat Detection
 
Novetta Cyber Analytics
Novetta Cyber AnalyticsNovetta Cyber Analytics
Novetta Cyber Analytics
 
Sumo Logic Cert Jam - Security Analytics
Sumo Logic Cert Jam - Security AnalyticsSumo Logic Cert Jam - Security Analytics
Sumo Logic Cert Jam - Security Analytics
 
CyberCrime in the Cloud and How to defend Yourself
CyberCrime in the Cloud and How to defend Yourself CyberCrime in the Cloud and How to defend Yourself
CyberCrime in the Cloud and How to defend Yourself
 
Security Certification: Security Analytics using Sumo Logic - Oct 2018
Security Certification: Security Analytics using Sumo Logic - Oct 2018Security Certification: Security Analytics using Sumo Logic - Oct 2018
Security Certification: Security Analytics using Sumo Logic - Oct 2018
 
SplunkSummit 2015 - Splunk User Behavioral Analytics
SplunkSummit 2015 - Splunk User Behavioral AnalyticsSplunkSummit 2015 - Splunk User Behavioral Analytics
SplunkSummit 2015 - Splunk User Behavioral Analytics
 
The Golden Rules - Detecting more with RSA Security Analytics
The Golden Rules  - Detecting more with RSA Security AnalyticsThe Golden Rules  - Detecting more with RSA Security Analytics
The Golden Rules - Detecting more with RSA Security Analytics
 
The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?
 
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network Security
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network SecurityMMIX Peering Forum and MMNOG 2020: Packet Analysis for Network Security
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network Security
 
Security From The Big Data and Analytics Perspective
Security From The Big Data and Analytics PerspectiveSecurity From The Big Data and Analytics Perspective
Security From The Big Data and Analytics Perspective
 

More from Raffael Marty

AI & ML in Cyber Security - Why Algorithms are Dangerous
AI & ML in Cyber Security - Why Algorithms are DangerousAI & ML in Cyber Security - Why Algorithms are Dangerous
AI & ML in Cyber Security - Why Algorithms are Dangerous
Raffael Marty
 

More from Raffael Marty (20)

Exploring the Defender's Advantage
Exploring the Defender's AdvantageExploring the Defender's Advantage
Exploring the Defender's Advantage
 
Extended Detection and Response (XDR) An Overhyped Product Category With Ulti...
Extended Detection and Response (XDR)An Overhyped Product Category With Ulti...Extended Detection and Response (XDR)An Overhyped Product Category With Ulti...
Extended Detection and Response (XDR) An Overhyped Product Category With Ulti...
 
How To Drive Value with Security Data
How To Drive Value with Security DataHow To Drive Value with Security Data
How To Drive Value with Security Data
 
Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?
Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?
Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?
 
Artificial Intelligence – Time Bomb or The Promised Land?
Artificial Intelligence – Time Bomb or The Promised Land?Artificial Intelligence – Time Bomb or The Promised Land?
Artificial Intelligence – Time Bomb or The Promised Land?
 
Understanding the "Intelligence" in AI
Understanding the "Intelligence" in AIUnderstanding the "Intelligence" in AI
Understanding the "Intelligence" in AI
 
Security Chat 5.0
Security Chat 5.0Security Chat 5.0
Security Chat 5.0
 
AI & ML in Cyber Security - Why Algorithms are Dangerous
AI & ML in Cyber Security - Why Algorithms are DangerousAI & ML in Cyber Security - Why Algorithms are Dangerous
AI & ML in Cyber Security - Why Algorithms are Dangerous
 
AI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are DangerousAI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are Dangerous
 
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't Changed
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't ChangedAI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't Changed
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't Changed
 
Security Insights at Scale
Security Insights at ScaleSecurity Insights at Scale
Security Insights at Scale
 
Creating Your Own Threat Intel Through Hunting & Visualization
Creating Your Own Threat Intel Through Hunting & VisualizationCreating Your Own Threat Intel Through Hunting & Visualization
Creating Your Own Threat Intel Through Hunting & Visualization
 
Big Data Visualization
Big Data VisualizationBig Data Visualization
Big Data Visualization
 
Workshop: Big Data Visualization for Security
Workshop: Big Data Visualization for SecurityWorkshop: Big Data Visualization for Security
Workshop: Big Data Visualization for Security
 
Visualization for Security
Visualization for SecurityVisualization for Security
Visualization for Security
 
The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?
 
DAVIX - Data Analysis and Visualization Linux
DAVIX - Data Analysis and Visualization LinuxDAVIX - Data Analysis and Visualization Linux
DAVIX - Data Analysis and Visualization Linux
 
Cloud - Security - Big Data
Cloud - Security - Big DataCloud - Security - Big Data
Cloud - Security - Big Data
 
Cyber Security – How Visual Analytics Unlock Insight
Cyber Security – How Visual Analytics Unlock InsightCyber Security – How Visual Analytics Unlock Insight
Cyber Security – How Visual Analytics Unlock Insight
 
AfterGlow
AfterGlowAfterGlow
AfterGlow
 

Recently uploaded

Russian Call girls in Abu Dhabi 0508644382 Abu Dhabi Call girls
Russian Call girls in Abu Dhabi 0508644382 Abu Dhabi Call girlsRussian Call girls in Abu Dhabi 0508644382 Abu Dhabi Call girls
Russian Call girls in Abu Dhabi 0508644382 Abu Dhabi Call girls
Monica Sydney
 
哪里办理美国迈阿密大学毕业证(本硕)umiami在读证明存档可查
哪里办理美国迈阿密大学毕业证(本硕)umiami在读证明存档可查哪里办理美国迈阿密大学毕业证(本硕)umiami在读证明存档可查
哪里办理美国迈阿密大学毕业证(本硕)umiami在读证明存档可查
ydyuyu
 
Top profile Call Girls In Dindigul [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Dindigul [ 7014168258 ] Call Me For Genuine Models ...Top profile Call Girls In Dindigul [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Dindigul [ 7014168258 ] Call Me For Genuine Models ...
gajnagarg
 
一比一原版奥兹学院毕业证如何办理
一比一原版奥兹学院毕业证如何办理一比一原版奥兹学院毕业证如何办理
一比一原版奥兹学院毕业证如何办理
F
 
pdfcoffee.com_business-ethics-q3m7-pdf-free.pdf
pdfcoffee.com_business-ethics-q3m7-pdf-free.pdfpdfcoffee.com_business-ethics-q3m7-pdf-free.pdf
pdfcoffee.com_business-ethics-q3m7-pdf-free.pdf
JOHNBEBONYAP1
 
Russian Escort Abu Dhabi 0503464457 Abu DHabi Escorts
Russian Escort Abu Dhabi 0503464457 Abu DHabi EscortsRussian Escort Abu Dhabi 0503464457 Abu DHabi Escorts
Russian Escort Abu Dhabi 0503464457 Abu DHabi Escorts
Monica Sydney
 
在线制作约克大学毕业证(yu毕业证)在读证明认证可查
在线制作约克大学毕业证(yu毕业证)在读证明认证可查在线制作约克大学毕业证(yu毕业证)在读证明认证可查
在线制作约克大学毕业证(yu毕业证)在读证明认证可查
ydyuyu
 

Recently uploaded (20)

Russian Call girls in Abu Dhabi 0508644382 Abu Dhabi Call girls
Russian Call girls in Abu Dhabi 0508644382 Abu Dhabi Call girlsRussian Call girls in Abu Dhabi 0508644382 Abu Dhabi Call girls
Russian Call girls in Abu Dhabi 0508644382 Abu Dhabi Call girls
 
20240509 QFM015 Engineering Leadership Reading List April 2024.pdf
20240509 QFM015 Engineering Leadership Reading List April 2024.pdf20240509 QFM015 Engineering Leadership Reading List April 2024.pdf
20240509 QFM015 Engineering Leadership Reading List April 2024.pdf
 
Trump Diapers Over Dems t shirts Sweatshirt
Trump Diapers Over Dems t shirts SweatshirtTrump Diapers Over Dems t shirts Sweatshirt
Trump Diapers Over Dems t shirts Sweatshirt
 
Best SEO Services Company in Dallas | Best SEO Agency Dallas
Best SEO Services Company in Dallas | Best SEO Agency DallasBest SEO Services Company in Dallas | Best SEO Agency Dallas
Best SEO Services Company in Dallas | Best SEO Agency Dallas
 
APNIC Policy Roundup, presented by Sunny Chendi at the 5th ICANN APAC-TWNIC E...
APNIC Policy Roundup, presented by Sunny Chendi at the 5th ICANN APAC-TWNIC E...APNIC Policy Roundup, presented by Sunny Chendi at the 5th ICANN APAC-TWNIC E...
APNIC Policy Roundup, presented by Sunny Chendi at the 5th ICANN APAC-TWNIC E...
 
Call girls Service in Ajman 0505086370 Ajman call girls
Call girls Service in Ajman 0505086370 Ajman call girlsCall girls Service in Ajman 0505086370 Ajman call girls
Call girls Service in Ajman 0505086370 Ajman call girls
 
哪里办理美国迈阿密大学毕业证(本硕)umiami在读证明存档可查
哪里办理美国迈阿密大学毕业证(本硕)umiami在读证明存档可查哪里办理美国迈阿密大学毕业证(本硕)umiami在读证明存档可查
哪里办理美国迈阿密大学毕业证(本硕)umiami在读证明存档可查
 
Meaning of On page SEO & its process in detail.
Meaning of On page SEO & its process in detail.Meaning of On page SEO & its process in detail.
Meaning of On page SEO & its process in detail.
 
Top profile Call Girls In Dindigul [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Dindigul [ 7014168258 ] Call Me For Genuine Models ...Top profile Call Girls In Dindigul [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Dindigul [ 7014168258 ] Call Me For Genuine Models ...
 
一比一原版奥兹学院毕业证如何办理
一比一原版奥兹学院毕业证如何办理一比一原版奥兹学院毕业证如何办理
一比一原版奥兹学院毕业证如何办理
 
Vip Firozabad Phone 8250092165 Escorts Service At 6k To 30k Along With Ac Room
Vip Firozabad Phone 8250092165 Escorts Service At 6k To 30k Along With Ac RoomVip Firozabad Phone 8250092165 Escorts Service At 6k To 30k Along With Ac Room
Vip Firozabad Phone 8250092165 Escorts Service At 6k To 30k Along With Ac Room
 
Nagercoil Escorts Service Girl ^ 9332606886, WhatsApp Anytime Nagercoil
Nagercoil Escorts Service Girl ^ 9332606886, WhatsApp Anytime NagercoilNagercoil Escorts Service Girl ^ 9332606886, WhatsApp Anytime Nagercoil
Nagercoil Escorts Service Girl ^ 9332606886, WhatsApp Anytime Nagercoil
 
2nd Solid Symposium: Solid Pods vs Personal Knowledge Graphs
2nd Solid Symposium: Solid Pods vs Personal Knowledge Graphs2nd Solid Symposium: Solid Pods vs Personal Knowledge Graphs
2nd Solid Symposium: Solid Pods vs Personal Knowledge Graphs
 
best call girls in Hyderabad Finest Escorts Service 📞 9352988975 📞 Available ...
best call girls in Hyderabad Finest Escorts Service 📞 9352988975 📞 Available ...best call girls in Hyderabad Finest Escorts Service 📞 9352988975 📞 Available ...
best call girls in Hyderabad Finest Escorts Service 📞 9352988975 📞 Available ...
 
pdfcoffee.com_business-ethics-q3m7-pdf-free.pdf
pdfcoffee.com_business-ethics-q3m7-pdf-free.pdfpdfcoffee.com_business-ethics-q3m7-pdf-free.pdf
pdfcoffee.com_business-ethics-q3m7-pdf-free.pdf
 
Russian Escort Abu Dhabi 0503464457 Abu DHabi Escorts
Russian Escort Abu Dhabi 0503464457 Abu DHabi EscortsRussian Escort Abu Dhabi 0503464457 Abu DHabi Escorts
Russian Escort Abu Dhabi 0503464457 Abu DHabi Escorts
 
在线制作约克大学毕业证(yu毕业证)在读证明认证可查
在线制作约克大学毕业证(yu毕业证)在读证明认证可查在线制作约克大学毕业证(yu毕业证)在读证明认证可查
在线制作约克大学毕业证(yu毕业证)在读证明认证可查
 
Ballia Escorts Service Girl ^ 9332606886, WhatsApp Anytime Ballia
Ballia Escorts Service Girl ^ 9332606886, WhatsApp Anytime BalliaBallia Escorts Service Girl ^ 9332606886, WhatsApp Anytime Ballia
Ballia Escorts Service Girl ^ 9332606886, WhatsApp Anytime Ballia
 
Tadepalligudem Escorts Service Girl ^ 9332606886, WhatsApp Anytime Tadepallig...
Tadepalligudem Escorts Service Girl ^ 9332606886, WhatsApp Anytime Tadepallig...Tadepalligudem Escorts Service Girl ^ 9332606886, WhatsApp Anytime Tadepallig...
Tadepalligudem Escorts Service Girl ^ 9332606886, WhatsApp Anytime Tadepallig...
 
Story Board.pptxrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Story Board.pptxrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrStory Board.pptxrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Story Board.pptxrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
 

Creating Your Own Threat Intel Through Hunting & Visualization

  • 1. Raffael Marty, CEO Creating Your Own Threat Intel Through Hunting & Visualization Tenerife, Spain February, 2016
  • 2. Creating Your Own Threat Intel Through Hunting & Visualization Raffael Marty, CEO
  • 3. Security. Analytics. Insight.3 Contents HUNTING AKA INTERNAL THREAT INTELLIGENCE THREAT INTELLIGENCE A PROCESS AND INFRASTRUCTURE VIEW 1 2 VISUALIZATION A THREAT INTELLIGENCE GOLD MINE 3
  • 5. Security. Analytics. Insight.5 • Products / Tools • Firewall - Blocks traffic based on pre-defined rules • Web Application Firewall - Monitors for signs of known malicious activity in Web traffic • Intrusion Prevention System - Looks for ‘signs’ of known attacks in traffic and protocol violations • Anti Virus - Looks for ‘signs’ of known attacks on the end system • Malware Sandbox - Runs new binaries and monitors their behavior for malicious signs • Security Information Management - Uses pre-defined rules to correlate signs from different data streams to augment intelligence • Vulnerability Scanning - Searches for known vulnerabilities and vulnerable software • Rely on pattern matching and signatures based knowledge from the past • Reactive -> always behind • Unknown and new threats -> won’t be detected • ‘Imperfect’ patterns and rules -> cause a lot of false positives We Are Monitoring - What is Going Wrong? Defense Has Been Relying On Past Knowledge
  • 6. Security. Analytics. Insight.6 Event Funnel - How We Used To Do It data rule-based
 correlation prioritization simple
 statistics attack candidates • What rules do you write? • Do the vendor provided rules work for you? • How do you define a priority 10 event? • High false positive rate! • Unless alerts are VERY focussed • High false negative rate! • Do you know what you don’t know?
  • 7. Security. Analytics. Insight.7 Then Came Threat Intelligence • How many hits do you really get? • You are missing most attacks IOCs • How do you match these efficiently against a real-time stream? • How do you de- duplicate and normalize these feeds? attack candidates 70–90% OF MALWARE SAMPLES ARE UNIQUE TO AN ORGANIZATION.
  • 8. Security. Analytics. Insight.8 Removing the Event Funnel - Hello Data Lake any 
 data Big Data Lake Rules • Storing more, and more diverse data • Kafka and “dynamic parsing” • Enabling large-scale processing • Spark, SparkStreaming, Storm, Parquet • Using “standard” data access (SQL, REST) • Plug in any other tool! context IOCs This per-se is not new …
  • 9. Security. Analytics. Insight.9 Adding Interactive - Analyst Driven Exploration any 
 data Big Data Lake Rules context IOCs … but first we get the human in the loop … Hunting • interactive visualization • analyst driven • machine assisted
  • 10. Security. Analytics. Insight.10 Hunting Creates Internal Threat Intelligence any 
 data Big Data Lake Rules context IOCs … then, let’s rethink our rules … Novel, Advanced Attacks internal TI
  • 11. Security. Analytics. Insight.11 Hunting Creates Internal Threat Intelligence any 
 data Big Data Lake Rules context IOCs … then, let’s rethink our rules … patterns anyone? internal TI Novel, Advanced Attacks Low False Positive Alerts Patterns
  • 12. Security. Analytics. Insight.12 Buzzword Bingo any 
 data Big Data Lake Rules context IOCs … and finally, we are buzzword compliant … behavioral monitoring
 scoring anomaly detection machine learning artificial intelligence “models” data science internal TI Patterns
  • 13. Security. Analytics. Insight.13 How Does All That Architecture Stuff Matter? In the following we’ll explore how this all matters … … but first, let’s see how visualization plays a key role here.
  • 15. Security. Analytics. Insight.15 “How Can We See, 
 Not To Confirm - But To Learn” - Edward Tufte
  • 16. Security. Analytics. Insight.16 Why Visualization?dport time
  • 17. Security. Analytics. Insight.17 SELECT count(distinct protocol) FROM flows; SELECT count(distinct port) FROM flows; SELECT count(distinct src_network) FROM flows; SELECT count(distinct dest_network) FROM flows; SELECT port, count(*) FROM flows GROUP BY port; SELECT protocol, count(CASE WHEN flows < 200 THEN 1 END) AS [<200], count(CASE WHEN flows>= 201 AND flows < 300 THEN 1 END) 
 AS [201 - 300], count(CASE WHEN flows>= 301 AND flows < 350 THEN 1 END) 
 AS [301 - 350], count(CASE WHEN flows>= 351 THEN 1 END) AS [>351] FROM flows GROUP BY protocol; SELECT port, count(distinct src_network) FROM flows GROUP BY port; SELECT src_network, count(distinct dest_network) FROM flows GROUP BY port; SELECT src_network, count(distinct dest_network) AS dn, sum(flows) FROM flows GROUP BY port, dn; SELECT port, protocol, count(*) FROM flows GROUP BY port, protocol; SELECT sum(flows), dest_network FROM flows GROUP BY dest_network; … One Graph Summarizes Dozens of Queries port dest_network protocol src_network flows
  • 18. Security. Analytics. Insight.18 Visualization To … Present / Communicate Discover / Explore
  • 19. Security. Analytics. Insight.19 We will have a look at a couple components from earlier: • Context • Data Science • Clustering • Seriation - Data Science Gone Wrong • Time-series Analysis Analytics Components
  • 20. Security. Analytics. Insight.20 Did You Know? Users accessing Sharepoint servers User Sharepoint Server data processing visualization This graph of users accessing sharepoint servers, does not immediately reveal any interesting patterns.
  • 21. Security. Analytics. Insight.21 Did You Know - How Context Tells a Story Using HR data as context Remote User San Francisco Office User Sharepoint Server data processing visualization HR data Using color to add context to the graph helps immediately identify outliers and potential problems.
  • 22. Security. Analytics. Insight.22 • Simple stuff works! • dc(dest), dc(d_port) • What is normal? • Use data science / data mining to prepare data. Then visualize the output for human analyst. Data Science in Security - Words of Caution
  • 23. Security. Analytics. Insight.23 Challenges With Clustering Network Traffic The graph shows an abstract space with colors being machine identified clusters. Hard Questions: • What are these clusters? • Do Web servers cluster? • What are good clusters? • What’s anomalous?
  • 24. Security. Analytics. Insight.24 Data Science That Works threshold outliers have different magnitudes
  • 25. Security. Analytics. Insight.25 Approximate Curve fitting a curve distance to curve
  • 26. Security. Analytics. Insight.26 Data Mining Applied better 
 threshold
  • 28. Security. Analytics. Insight.28 Hunting - Ready, Fire, Aim • Analysts are your best and most expensive resource • They need the right tools and data • Speed (see earlier architecture) • Interaction (visual!) • Machine-assisted insight Examples • Exploring DNS traffic • High business impact machine analysis • Lateral movement
  • 29. Security. Analytics. Insight.29 HBI Metric Analysis Visually learn, Test, Automate
  • 30. Security. Analytics. Insight.30 HBI Metric Analysis - If you like Black Backgrounds
  • 31. Security. Analytics. Insight.31 We have tried many thing: • Social Network Analysis • Seasonality detection • Entropy over time • Frequent pattern mining • Clustering All kinds of challenges. Simple works! Let’s Get Mathematical U−matrix 4.28e−05 0.0461 0.0921
  • 33. Security. Analytics. Insight.33 Lateral Movement - Cross Network Communications Challenges • Scale • You will find one of everything • Defining white-lists and keeping them up to date (i.e., network and asset hygiene) VPN DMZ Office GIA Unknown Internet AWS
  • 34. Security. Analytics. Insight.34 http://secviz.org List: secviz.org/mailinglist Twitter: @secviz Share, discuss, challenge, and learn about security visualization. Security Visualization Community
  • 35. Security. Analytics. Insight.35 BlackHat Workshop Visual Analytics Delivering Actionable Security Intelligence July 30,31 & August 1,2 - Las Vegas, USA big data | analytics | visualization http://secviz.org
  • 36. Security. Analytics. Insight.36 After some exploration … raffael.marty@pixlcloud.com http://slideshare.net/zrlram http://secviz.org and @secviz Further resources: