SlideShare a Scribd company logo

Measuring the IQ of your Threat Intelligence Feeds (#tiqtest)

•
•
Alex Pinto
Alex Pinto

Follow along with the R Markdown file at http://rpubs.com/alexcpsec/tiq-test-Summer2014-2 Source code available at: https://github.com/mlsecproject/tiq-test https://github.com/mlsecproject/tiq-test-Summer2014 https://github.com/mlsecproject/combine --------- Full Abstract: Threat Intelligence feeds are now being touted as the saving grace for SIEM and log management deployments, and as a way to supercharge incident detection and even response practices. We have heard similar promises before as an industry, so it is only fair to try to investigate. Since the actual number of breaches and attacks worldwide is unknown, it is impossible to measure how good threat intelligence feeds really are, right? Enter a new scientific breakthrough developed over the last 300 years: statistics! This presentation will consist of a data-driven analysis of a cross-section of threat intelligence feeds (both open-source and commercial) to measure their statistical bias, overlap, and representability of the unknown population of breaches worldwide. Are they a statistical good measure of the population of "bad stuff" happening out there? Is there even such a thing? How tuned to your specific threat surface are those feeds anyway? Regardless, can we actually make good use of them even if the threats they describe have no overlap with the actual incidents you have been seeing in your environment? We will provide an open-source tool for attendees to extract, normalize and export data from threat intelligence feeds to use in their internal projects and systems. It will be pre-configured with current OSINT network feed and easily extensible for private or commercial feeds. All the statistical code written and research data used (from the open-source feeds) will be made available in the spirit of reproducible research. The tool itself will be able to be used by attendees to perform the same type of tests on their own data. Join Alex and Kyle on a journey through the actual real-world usability of threat intelligence to find out which mix of open source and private feeds are right for your organization.

Data & Analytics
1 of 42
Download to read offline
Measuring  the  IQ  of  your  Threat   Intelligence  Feeds  (#TIQtest)   Alex  Pinto   MLSec  Project     @alexcpsec   @MLSecProject! Kyle  Maxwell   Researcher   @kylemaxwell!
Alex  Pinto   •  Science  guy  at  MLSec  Project   •  ML  trainer   •  Network  security  aficionado     •  Tortured  by  SIEMs  as  a  child   •  Hacker  Spirit  Animal™:   CAFFEINATED  CAPYBARA! whoami(s)   Kyle  Maxwell   •  Researcher  at  [REDACTED]   •  Math  Smuggler   •  Recovering  Incident  Responder   •  GPL  zealot   •  Hacker  Spirit  Animal™:   AXIOMATIC  ARMADILLO! (hUps://secure.flickr.com/photos/kobashi_san/)   (hUp://www.langorigami.com/art/gallery/ gallery.php?tag=mammals&name=armadillo)  
•  Threat  Intel  102   •  Measuring  Intelligence   •  Data  Preparaaon   •  Tesang  the  Data   •  Tools:   •  COMBINE   •  TIQ-­‐TEST   •  Some  parang  ideas   Agenda   (hUp://www.savagechickens.com/2008/12/iq-­‐test.html)  
Threat  Intel  102:  Capability  and   Intent   •  What  are  they  able  to  do?   •  What  are  they  intending  to  do?!
Threat  Intel  102:  Cage  Matches   •  Signatures  vs  Indicators   •  Data  vs  Intelligence   •  Tacacal  vs  Strategic   •  Atomic  vs  Composite  
Threat  Intel  102:  Pyramid  of  Pain   “Simple”  and  “easy”  aren’t  always   (David  Bianco  –  Pyramid  of  Pain)  
What  about  IP  addresses?   •  Approximately  same  value  as  hostnames  (APT  vs  DGA)   •  Finite  resource  (unal  IPv6,  that  is)   •  Managed  /  controlled  by  orgs   •  Difficulty  /  economic  incenaves  /  implied  “cost”   •  Also,  recyclable     (hUps://xkcd.com/865/)  
Given  IP  addresses  harvested  from   TI  feeds,  can  we  measure  how   much  they  “help”  our  defense   metrics?!
Introducing  TIQ-­‐TEST   •  All  these  tests  are  available  as  R  funcaons  at     •  hUps://github.com/mlsecproject/aq-­‐test   •  Have  fun,  prove  me  wrong,  suggest  stuff   •  Tools  that  implement  those  tests   •  Sample  data  +  R  Markdown  file   •  The  excuse  to  learn  a  staasacal  language  you  were  waiang   for!  
Data  Sources  –  Types  of  data   •  Extract  the  “raw”  informaaon  from  indicator  feeds   •  Both  IP  addresses  and  hostnames  were  extracted  
Data  Sources  –  Feeds  Selected   •  Data  was  separated  into  “inbound”  and  “outbound”  
Data  PreparaLon  and  Cleansing   •  Convert  the  hostname  data  to  IP  addresses:   •  Acave  IP  addresses  for  the  respecave  date  (“A”  query)   •  Passive  DNS  from  Farsight  Security  (DNSDB)   •  We  removed  non-­‐public  IPs  from  the  dataset  (RFC1918)     •  Yeah,  we  know  it  is  a  “parking  technique”   (hUps://xkcd.com/742/)  
Data  PreparaLon  and  Cleansing   •  For  each  IP  record  (including  the  ones  from  hostnames):   •  Add  asnumber  and  asname  (from  MaxMind  ASN  DB)   •  Add  country  (from  MaxMind  GeoLite  DB)   •  Add  rhost  (again  from  DNSDB)  –  most  popular  “PTR”   •  The  experiments  will  be  around  ASNs  and  Geolocaaon  
Data  PreparaLon  and  Cleansing   •  However,  we  will  NOT  be  using  maps.  Just  let  it  go.  
Data  PreparaLon  and  Cleansing   •  Small  enriched  sample:!
TesLng  the  Data   •  Let’s  generate  some  interesang  metrics:   •  NOVELTY  –  How  oxen  do  they  update  themselves?   •  OVERLAP  –  How  do  they  compare  to  what  you  got?   •  POPULATION  –  what  is  in  them  anyway?   •  Populaaon  is  tricky:   •  Could  mean  the  enare  world  (all  IPv4  space)   •  Should  ideally  mean  YOUR  world  
But  WHAT  IS  THE  IQ?!??1?   •  We  will  withhold  judgment   •  The  best  data  composiaon  is  the  best  one  for  you   •  We  will  do  our  best  to  explain  results  so  you  can  decide.   •  Maybe  on  further  (or  more  private)  research…      
Novelty  Test  –  measuring  added   and  dropped  indicators!
Novelty  Test  (1)  
Novelty  Test  (2)  
Overlap  Test  –  More  data  is  beUer,   but  make  sure  it  is  not  the  same   data!
Overlap  Test  
Overlap  Test  
Overlap  Test  
PopulaLon  Test   •  Let  us  use  the  ASN  and   GeoIP  databases  that  we   used  to  enrich  our  data  as  a   reference  of  the  “true”   populaaon.     •  But,  but,  human  beings  are   unpredictable!  We  will   never  be  able  to  forecast   this!      
Can  we  get  a  beSer  look?   •  Don’t  like  squinang  either   •  Staasacal  inference-­‐based  comparison  models   (hypothesis  tesang)   •  Exact  binomial  tests  (when  we  have  the  “true”  pop)   •  Chi-­‐squared  proporaon  tests  (similar  to   independence  tests)    
Can  we  get  a  beSer  look?   •  We  can  beUer  esamate,  with  confidence  intervals,  our   measures  of  error.   •  Also,  p-­‐values!  (with  apologies  to  Alex  HuUon)   •  We  promise  to  be  very  conservaave  in  using  them.    
Hacker  Spirit  Animal™  Guide   •  US  –  Eagle   •  CA  –  Moose   •  FR  –  Frog   •  GB  –  Bulldog   •  AU  –  Koala   •  BR  –  Capybara  /  Toucan   •  Texas  –  Armadillo   •  Disclaimer:  we  do  not  endorse  Geolocaaon-­‐based  aUribuaon  
Trend  Comparison  
Introducing  COMBINE   •  Harvesang  feeds   takes  some  work.   •  Most  of  us  let   somebody  else  do  it   without  thinking   about  what  it   actually  takes.  
Introducing  COMBINE   hUps://github.com/mlsecproject/combine  
Introducing  COMBINE   •  Components:   1.  Reaper  gathers  the  threat  data  directly  from  feeds.   2.  Thresher  normalizes  it  into  a  simplisac  data  model.   3.  Winnower  opaonally  performs  basic  validaaon  or   enrichment.   4.  Baler  transforms  the  data  into  CybOX,  CSV,  JSON,  and   CIM.  (Only  CSV  and  JSON  work  right  now).  Could  also   write  others  fairly  easily.  (nudge  nudge,  wink  wink)  
Introducing  COMBINE   •  Always  trying  to  feed   it  more.  Lots  of   possibiliaes,   including  your  own   data  sources.   •  We  clearly  do  NOT   endorse  any  included   feeds.    
Introducing  COMBINE   •  Enrichments  -­‐  think   metadata.     •  AS,  geolocaaon   •  DNS  resoluaons   courtesy  of  Farsight   DNSDB     •  Ask  them  for  an   API  key  to  test  it,  tell   them  Alex  Pinto   sent  you  ;)    
MLSec  Project   •  Both  projects  have  been  released  as  GPLv3  by  MLSec  Project   •  Will  replace  the  internal  versions  we  have  on  the  main  code     •  Looking  for  paracipants  and  data  sharing  agreements   •  Liked  TIQ-­‐TEST?  We  can  benchmark  your  private  feeds  using   these  and  other  techniques     •  Visit  hSps://www.mlsecproject.org  ,  message  @MLSecProject   or  just  e-­‐mail  me.!
Take  Aways   •  Analyze  your  data.     •  Extract  value  from  it!   •  Try  before  you  buy!  Different  test   results  mean  different  things  to   different  orgs.   •  Use  the  tools!  Suggest  new  tests!   •  Share  data  with  us!  We  take  good   care  of  it,  make  sure  it  gets   proper  exercise.    
Thanks!   •  Q&A?   •  Feedback!   Alex  Pinto     @alexcpsec   @MLSecProject   ”The  measure  of  intelligence  is  the  ability  to  change."                  -­‐  Albert  Einstein     Kyle  Maxwell   @kylemaxwell  

Recommended

Bsides 2019 - Intelligent Threat Hunting
Bsides 2019 - Intelligent Threat HuntingBsides 2019 - Intelligent Threat Hunting
Bsides 2019 - Intelligent Threat HuntingDhruv Majumdar
 
Threat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-onThreat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-onSplunk
 
Sharpening your Threat-Hunting Program with ATTACK Framework
Sharpening your Threat-Hunting Program with ATTACK FrameworkSharpening your Threat-Hunting Program with ATTACK Framework
Sharpening your Threat-Hunting Program with ATTACK FrameworkMITRE - ATT&CKcon
 
What is Threat Hunting? - Panda Security
What is Threat Hunting? - Panda SecurityWhat is Threat Hunting? - Panda Security
What is Threat Hunting? - Panda SecurityPanda Security
 
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITREMITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITREMITRE - ATT&CKcon
 
Cyber Threat hunting workshop
Cyber Threat hunting workshopCyber Threat hunting workshop
Cyber Threat hunting workshopArpan Raval
 
How MITRE ATT&CK helps security operations
How MITRE ATT&CK helps security operationsHow MITRE ATT&CK helps security operations
How MITRE ATT&CK helps security operationsSergey Soldatov
 
Cyber kill chain
Cyber kill chainCyber kill chain
Cyber kill chainAnkita Ganguly
 

Recommended

Bsides 2019 - Intelligent Threat Hunting
Bsides 2019 - Intelligent Threat HuntingBsides 2019 - Intelligent Threat Hunting
Bsides 2019 - Intelligent Threat HuntingDhruv Majumdar
 
Threat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-onThreat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-onSplunk
 
Sharpening your Threat-Hunting Program with ATTACK Framework
Sharpening your Threat-Hunting Program with ATTACK FrameworkSharpening your Threat-Hunting Program with ATTACK Framework
Sharpening your Threat-Hunting Program with ATTACK FrameworkMITRE - ATT&CKcon
 
What is Threat Hunting? - Panda Security
What is Threat Hunting? - Panda SecurityWhat is Threat Hunting? - Panda Security
What is Threat Hunting? - Panda SecurityPanda Security
 
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITREMITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITREMITRE - ATT&CKcon
 
Cyber Threat hunting workshop
Cyber Threat hunting workshopCyber Threat hunting workshop
Cyber Threat hunting workshopArpan Raval
 
How MITRE ATT&CK helps security operations
How MITRE ATT&CK helps security operationsHow MITRE ATT&CK helps security operations
How MITRE ATT&CK helps security operationsSergey Soldatov
 
Cyber kill chain
Cyber kill chainCyber kill chain
Cyber kill chainAnkita Ganguly
 
MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...
MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...
MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...MITRE - ATT&CKcon
 
Threat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonThreat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonBen Boyd
 
MITRE ATT&CK framework
MITRE ATT&CK frameworkMITRE ATT&CK framework
MITRE ATT&CK frameworkBhushan Gurav
 
Cyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep SinghCyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep SinghOWASP Delhi
 
Threat hunting for Beginners
Threat hunting for BeginnersThreat hunting for Beginners
Threat hunting for BeginnersSKMohamedKasim
 
Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...
Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...
Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...Jorge Orchilles
 
Cyber Threat Intelligence
Cyber Threat IntelligenceCyber Threat Intelligence
Cyber Threat Intelligencemohamed nasri
 
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...Chris Gates
 
Threat Hunting
Threat HuntingThreat Hunting
Threat HuntingSplunk
 
Threat Modelling - It's not just for developers
Threat Modelling - It's not just for developersThreat Modelling - It's not just for developers
Threat Modelling - It's not just for developersMITRE ATT&CK
 
[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptx
[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptx[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptx
[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptxChi En (Ashley) Shen
 
Knowledge for the masses: Storytelling with ATT&CK
Knowledge for the masses: Storytelling with ATT&CKKnowledge for the masses: Storytelling with ATT&CK
Knowledge for the masses: Storytelling with ATT&CKMITRE ATT&CK
 
ATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat IntelligenceATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat IntelligenceMITRE - ATT&CKcon
 
Effective Threat Hunting with Tactical Threat Intelligence
Effective Threat Hunting with Tactical Threat IntelligenceEffective Threat Hunting with Tactical Threat Intelligence
Effective Threat Hunting with Tactical Threat IntelligenceDhruv Majumdar
 
State of the ATT&CK
State of the ATT&CKState of the ATT&CK
State of the ATT&CKMITRE ATT&CK
 
Putting MITRE ATT&CK into Action with What You Have, Where You Are
Putting MITRE ATT&CK into Action with What You Have, Where You ArePutting MITRE ATT&CK into Action with What You Have, Where You Are
Putting MITRE ATT&CK into Action with What You Have, Where You AreKatie Nickels
 
Cyber Threat Hunting Workshop
Cyber Threat Hunting WorkshopCyber Threat Hunting Workshop
Cyber Threat Hunting WorkshopDigit Oktavianto
 
Introduction to MITRE ATT&CK
Introduction to MITRE ATT&CKIntroduction to MITRE ATT&CK
Introduction to MITRE ATT&CKArpan Raval
 
Threat-Based Adversary Emulation with MITRE ATT&CK
Threat-Based Adversary Emulation with MITRE ATT&CKThreat-Based Adversary Emulation with MITRE ATT&CK
Threat-Based Adversary Emulation with MITRE ATT&CKKatie Nickels
 
Using IOCs to Design and Control Threat Activities During a Red Team Engagement
Using IOCs to Design and Control Threat Activities During a Red Team EngagementUsing IOCs to Design and Control Threat Activities During a Red Team Engagement
Using IOCs to Design and Control Threat Activities During a Red Team EngagementJoe Vest
 
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...Alex Pinto
 
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Alex Pinto
 

More Related Content

What's hot

MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...
MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...
MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...MITRE - ATT&CKcon
 
Threat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonThreat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonBen Boyd
 
MITRE ATT&CK framework
MITRE ATT&CK frameworkMITRE ATT&CK framework
MITRE ATT&CK frameworkBhushan Gurav
 
Cyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep SinghCyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep SinghOWASP Delhi
 
Threat hunting for Beginners
Threat hunting for BeginnersThreat hunting for Beginners
Threat hunting for BeginnersSKMohamedKasim
 
Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...
Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...
Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...Jorge Orchilles
 
Cyber Threat Intelligence
Cyber Threat IntelligenceCyber Threat Intelligence
Cyber Threat Intelligencemohamed nasri
 
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...Chris Gates
 
Threat Hunting
Threat HuntingThreat Hunting
Threat HuntingSplunk
 
Threat Modelling - It's not just for developers
Threat Modelling - It's not just for developersThreat Modelling - It's not just for developers
Threat Modelling - It's not just for developersMITRE ATT&CK
 
[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptx
[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptx[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptx
[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptxChi En (Ashley) Shen
 
Knowledge for the masses: Storytelling with ATT&CK
Knowledge for the masses: Storytelling with ATT&CKKnowledge for the masses: Storytelling with ATT&CK
Knowledge for the masses: Storytelling with ATT&CKMITRE ATT&CK
 
ATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat IntelligenceATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat IntelligenceMITRE - ATT&CKcon
 
Effective Threat Hunting with Tactical Threat Intelligence
Effective Threat Hunting with Tactical Threat IntelligenceEffective Threat Hunting with Tactical Threat Intelligence
Effective Threat Hunting with Tactical Threat IntelligenceDhruv Majumdar
 
State of the ATT&CK
State of the ATT&CKState of the ATT&CK
State of the ATT&CKMITRE ATT&CK
 
Putting MITRE ATT&CK into Action with What You Have, Where You Are
Putting MITRE ATT&CK into Action with What You Have, Where You ArePutting MITRE ATT&CK into Action with What You Have, Where You Are
Putting MITRE ATT&CK into Action with What You Have, Where You AreKatie Nickels
 
Cyber Threat Hunting Workshop
Cyber Threat Hunting WorkshopCyber Threat Hunting Workshop
Cyber Threat Hunting WorkshopDigit Oktavianto
 
Introduction to MITRE ATT&CK
Introduction to MITRE ATT&CKIntroduction to MITRE ATT&CK
Introduction to MITRE ATT&CKArpan Raval
 
Threat-Based Adversary Emulation with MITRE ATT&CK
Threat-Based Adversary Emulation with MITRE ATT&CKThreat-Based Adversary Emulation with MITRE ATT&CK
Threat-Based Adversary Emulation with MITRE ATT&CKKatie Nickels
 
Using IOCs to Design and Control Threat Activities During a Red Team Engagement
Using IOCs to Design and Control Threat Activities During a Red Team EngagementUsing IOCs to Design and Control Threat Activities During a Red Team Engagement
Using IOCs to Design and Control Threat Activities During a Red Team EngagementJoe Vest
 

What's hot (20)

MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...
MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...
MITRE ATT&CKcon 2.0: Prioritizing ATT&CK Informed Defenses the CIS Way; Phili...
 
Threat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonThreat hunting - Every day is hunting season
Threat hunting - Every day is hunting season
 
MITRE ATT&CK framework
MITRE ATT&CK frameworkMITRE ATT&CK framework
MITRE ATT&CK framework
 
Cyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep SinghCyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep Singh
 
Threat hunting for Beginners
Threat hunting for BeginnersThreat hunting for Beginners
Threat hunting for Beginners
 
Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...
Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...
Managing & Showing Value during Red Team Engagements & Purple Team Exercises ...
 
Cyber Threat Intelligence
Cyber Threat IntelligenceCyber Threat Intelligence
Cyber Threat Intelligence
 
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...
Purple Teaming the Cyber Kill Chain: Practical Exercises for Everyone Sector...
 
Threat Hunting
Threat HuntingThreat Hunting
Threat Hunting
 
Threat Modelling - It's not just for developers
Threat Modelling - It's not just for developersThreat Modelling - It's not just for developers
Threat Modelling - It's not just for developers
 
[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptx
[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptx[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptx
[HITCON 2020 CTI Village] Threat Hunting and Campaign Tracking Workshop.pptx
 
Knowledge for the masses: Storytelling with ATT&CK
Knowledge for the masses: Storytelling with ATT&CKKnowledge for the masses: Storytelling with ATT&CK
Knowledge for the masses: Storytelling with ATT&CK
 
ATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat IntelligenceATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat Intelligence
 
Effective Threat Hunting with Tactical Threat Intelligence
Effective Threat Hunting with Tactical Threat IntelligenceEffective Threat Hunting with Tactical Threat Intelligence
Effective Threat Hunting with Tactical Threat Intelligence
 
State of the ATT&CK
State of the ATT&CKState of the ATT&CK
State of the ATT&CK
 
Putting MITRE ATT&CK into Action with What You Have, Where You Are
Putting MITRE ATT&CK into Action with What You Have, Where You ArePutting MITRE ATT&CK into Action with What You Have, Where You Are
Putting MITRE ATT&CK into Action with What You Have, Where You Are
 
Cyber Threat Hunting Workshop
Cyber Threat Hunting WorkshopCyber Threat Hunting Workshop
Cyber Threat Hunting Workshop
 
Introduction to MITRE ATT&CK
Introduction to MITRE ATT&CKIntroduction to MITRE ATT&CK
Introduction to MITRE ATT&CK
 
Threat-Based Adversary Emulation with MITRE ATT&CK
Threat-Based Adversary Emulation with MITRE ATT&CKThreat-Based Adversary Emulation with MITRE ATT&CK
Threat-Based Adversary Emulation with MITRE ATT&CK
 
Using IOCs to Design and Control Threat Activities During a Red Team Engagement
Using IOCs to Design and Control Threat Activities During a Red Team EngagementUsing IOCs to Design and Control Threat Activities During a Red Team Engagement
Using IOCs to Design and Control Threat Activities During a Red Team Engagement
 

Viewers also liked

From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...Alex Pinto
 
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Alex Pinto
 
BSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information SecurityBSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information SecurityAlex Pinto
 
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...Alex Pinto
 
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and Sharing
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and SharingData-Driven Threat Intelligence: Metrics on Indicator Dissemination and Sharing
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and SharingAlex Pinto
 
Beyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based DetectionBeyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based DetectionAlex Pinto
 
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitian
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitianPoc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitian
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitianLiang Chen
 
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...North Texas Chapter of the ISSA
 
11 x1 t16 05 volumes (2013)
11 x1 t16 05 volumes (2013)11 x1 t16 05 volumes (2013)
11 x1 t16 05 volumes (2013)Nigel Simmons
 
11 x1 t01 03 factorising (2014)
11 x1 t01 03 factorising (2014)11 x1 t01 03 factorising (2014)
11 x1 t01 03 factorising (2014)Nigel Simmons
 
Ducky USB - Indicators of Compromise (IOCs)
Ducky USB - Indicators of Compromise (IOCs)Ducky USB - Indicators of Compromise (IOCs)
Ducky USB - Indicators of Compromise (IOCs)Brent Muir
 
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Alex Pinto
 
Tarea seminario 3
Tarea seminario 3Tarea seminario 3
Tarea seminario 3carsuro25
 
11 x1 t16 02 definite integral (2013)
11 x1 t16 02 definite integral (2013)11 x1 t16 02 definite integral (2013)
11 x1 t16 02 definite integral (2013)Nigel Simmons
 
The beauty of mathematics
The beauty of mathematicsThe beauty of mathematics
The beauty of mathematicsTarun Gehlot
 
Cyber Threat Intelligence: Who is Targeting your Information?
Cyber Threat Intelligence: Who is Targeting your Information? Cyber Threat Intelligence: Who is Targeting your Information?
Cyber Threat Intelligence: Who is Targeting your Information? Control Risks
 
Finite elements : basis functions
Finite elements : basis functionsFinite elements : basis functions
Finite elements : basis functionsTarun Gehlot
 
Graphing inverse functions
Graphing inverse functionsGraphing inverse functions
Graphing inverse functionsTarun Gehlot
 
Finite elements for 2‐d problems
Finite elements for 2‐d problemsFinite elements for 2‐d problems
Finite elements for 2‐d problemsTarun Gehlot
 
Goodbye slideshare UPDATE
Goodbye slideshare UPDATEGoodbye slideshare UPDATE
Goodbye slideshare UPDATENigel Simmons
 

Viewers also liked (20)

From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
 
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
 
BSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information SecurityBSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information Security
 
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
 
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and Sharing
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and SharingData-Driven Threat Intelligence: Metrics on Indicator Dissemination and Sharing
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and Sharing
 
Beyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based DetectionBeyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based Detection
 
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitian
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitianPoc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitian
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitian
 
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...
 
11 x1 t16 05 volumes (2013)
11 x1 t16 05 volumes (2013)11 x1 t16 05 volumes (2013)
11 x1 t16 05 volumes (2013)
 
11 x1 t01 03 factorising (2014)
11 x1 t01 03 factorising (2014)11 x1 t01 03 factorising (2014)
11 x1 t01 03 factorising (2014)
 
Ducky USB - Indicators of Compromise (IOCs)
Ducky USB - Indicators of Compromise (IOCs)Ducky USB - Indicators of Compromise (IOCs)
Ducky USB - Indicators of Compromise (IOCs)
 
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
 
Tarea seminario 3
Tarea seminario 3Tarea seminario 3
Tarea seminario 3
 
11 x1 t16 02 definite integral (2013)
11 x1 t16 02 definite integral (2013)11 x1 t16 02 definite integral (2013)
11 x1 t16 02 definite integral (2013)
 
The beauty of mathematics
The beauty of mathematicsThe beauty of mathematics
The beauty of mathematics
 
Cyber Threat Intelligence: Who is Targeting your Information?
Cyber Threat Intelligence: Who is Targeting your Information? Cyber Threat Intelligence: Who is Targeting your Information?
Cyber Threat Intelligence: Who is Targeting your Information?
 
Finite elements : basis functions
Finite elements : basis functionsFinite elements : basis functions
Finite elements : basis functions
 
Graphing inverse functions
Graphing inverse functionsGraphing inverse functions
Graphing inverse functions
 
Finite elements for 2‐d problems
Finite elements for 2‐d problemsFinite elements for 2‐d problems
Finite elements for 2‐d problems
 
Goodbye slideshare UPDATE
Goodbye slideshare UPDATEGoodbye slideshare UPDATE
Goodbye slideshare UPDATE
 

Similar to Measuring the IQ of your Threat Intelligence Feeds (#tiqtest)

Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...
Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...
Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...OpenSource Connections
 
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014The Hive
 
Agile Data: Building Hadoop Analytics Applications
Agile Data: Building Hadoop Analytics ApplicationsAgile Data: Building Hadoop Analytics Applications
Agile Data: Building Hadoop Analytics ApplicationsDataWorks Summit
 
Agile Data Science: Building Hadoop Analytics Applications
Agile Data Science: Building Hadoop Analytics ApplicationsAgile Data Science: Building Hadoop Analytics Applications
Agile Data Science: Building Hadoop Analytics ApplicationsRussell Jurney
 
BinaryPig - Scalable Malware Analytics in Hadoop
BinaryPig - Scalable Malware Analytics in HadoopBinaryPig - Scalable Malware Analytics in Hadoop
BinaryPig - Scalable Malware Analytics in HadoopJason Trost
 
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...Big Data Spain
 
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with Dask
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with DaskAUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with Dask
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with DaskVíctor Zabalza
 
Agile Data Science: Hadoop Analytics Applications
Agile Data Science: Hadoop Analytics ApplicationsAgile Data Science: Hadoop Analytics Applications
Agile Data Science: Hadoop Analytics ApplicationsRussell Jurney
 
The Background Noise of the Internet
The Background Noise of the InternetThe Background Noise of the Internet
The Background Noise of the InternetAndrew Morris
 
Protect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying TechniquesProtect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying TechniquesLeo Loobeek
 
10(?) holiday gifts for the SOC who has everything
10(?) holiday gifts for the SOC who has everything10(?) holiday gifts for the SOC who has everything
10(?) holiday gifts for the SOC who has everythingRyan Kovar
 
Data Science meets Software Development
Data Science meets Software DevelopmentData Science meets Software Development
Data Science meets Software DevelopmentAlexis Seigneurin
 
Searching Chinese Patents Presentation at Enterprise Data World
Searching Chinese Patents Presentation at Enterprise Data WorldSearching Chinese Patents Presentation at Enterprise Data World
Searching Chinese Patents Presentation at Enterprise Data WorldOpenSource Connections
 
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwon
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwonThe basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwon
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwonKenneth Kwon
 
SDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and modelsSDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and modelsKorea Sdec
 
Ncku csie talk about Spark
Ncku csie talk about SparkNcku csie talk about Spark
Ncku csie talk about SparkGiivee The
 
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...Lucidworks (Archived)
 
Data infrastructure architecture for medium size organization: tips for colle...
Data infrastructure architecture for medium size organization: tips for colle...Data infrastructure architecture for medium size organization: tips for colle...
Data infrastructure architecture for medium size organization: tips for colle...DataWorks Summit/Hadoop Summit
 
NotaCon 2011 - Networking for Pentesters
NotaCon 2011 - Networking for PentestersNotaCon 2011 - Networking for Pentesters
NotaCon 2011 - Networking for PentestersRob Fuller
 

Similar to Measuring the IQ of your Threat Intelligence Feeds (#tiqtest) (20)

Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...
Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...
Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...
 
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014
 
Agile Data: Building Hadoop Analytics Applications
Agile Data: Building Hadoop Analytics ApplicationsAgile Data: Building Hadoop Analytics Applications
Agile Data: Building Hadoop Analytics Applications
 
Agile Data Science: Building Hadoop Analytics Applications
Agile Data Science: Building Hadoop Analytics ApplicationsAgile Data Science: Building Hadoop Analytics Applications
Agile Data Science: Building Hadoop Analytics Applications
 
BinaryPig - Scalable Malware Analytics in Hadoop
BinaryPig - Scalable Malware Analytics in HadoopBinaryPig - Scalable Malware Analytics in Hadoop
BinaryPig - Scalable Malware Analytics in Hadoop
 
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...
 
Data Science
Data ScienceData Science
Data Science
 
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with Dask
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with DaskAUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with Dask
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with Dask
 
Agile Data Science: Hadoop Analytics Applications
Agile Data Science: Hadoop Analytics ApplicationsAgile Data Science: Hadoop Analytics Applications
Agile Data Science: Hadoop Analytics Applications
 
The Background Noise of the Internet
The Background Noise of the InternetThe Background Noise of the Internet
The Background Noise of the Internet
 
Protect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying TechniquesProtect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying Techniques
 
10(?) holiday gifts for the SOC who has everything
10(?) holiday gifts for the SOC who has everything10(?) holiday gifts for the SOC who has everything
10(?) holiday gifts for the SOC who has everything
 
Data Science meets Software Development
Data Science meets Software DevelopmentData Science meets Software Development
Data Science meets Software Development
 
Searching Chinese Patents Presentation at Enterprise Data World
Searching Chinese Patents Presentation at Enterprise Data WorldSearching Chinese Patents Presentation at Enterprise Data World
Searching Chinese Patents Presentation at Enterprise Data World
 
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwon
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwonThe basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwon
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwon
 
SDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and modelsSDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and models
 
Ncku csie talk about Spark
Ncku csie talk about SparkNcku csie talk about Spark
Ncku csie talk about Spark
 
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...
 
Data infrastructure architecture for medium size organization: tips for colle...
Data infrastructure architecture for medium size organization: tips for colle...Data infrastructure architecture for medium size organization: tips for colle...
Data infrastructure architecture for medium size organization: tips for colle...
 
NotaCon 2011 - Networking for Pentesters
NotaCon 2011 - Networking for PentestersNotaCon 2011 - Networking for Pentesters
NotaCon 2011 - Networking for Pentesters
 

Recently uploaded

Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAroojKhan71
 
Determinants of health, dimensions of health, positive health and spectrum of...
Determinants of health, dimensions of health, positive health and spectrum of...Determinants of health, dimensions of health, positive health and spectrum of...
Determinants of health, dimensions of health, positive health and spectrum of...shambhavirathore45
 
CebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptxCebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptxolyaivanovalion
 
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...SUHANI PANDEY
 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfLars Albertsson
 
Data-Analysis for Chicago Crime Data 2023
Data-Analysis for Chicago Crime Data 2023Data-Analysis for Chicago Crime Data 2023
Data-Analysis for Chicago Crime Data 2023ymrp368
 
Carero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptxCarero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptxolyaivanovalion
 
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Delhi Call girls
 
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 nightCheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 nightDelhi Call girls
 
BPAC WITH UFSBI GENERAL PRESENTATION 18_05_2017-1.pptx
BPAC WITH UFSBI GENERAL PRESENTATION 18_05_2017-1.pptxBPAC WITH UFSBI GENERAL PRESENTATION 18_05_2017-1.pptx
BPAC WITH UFSBI GENERAL PRESENTATION 18_05_2017-1.pptxMohammedJunaid861692
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysismanisha194592
 
Zuja dropshipping via API with DroFx.pptx
Zuja dropshipping via API with DroFx.pptxZuja dropshipping via API with DroFx.pptx
Zuja dropshipping via API with DroFx.pptxolyaivanovalion
 
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...amitlee9823
 
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779Delhi Call girls
 
Call me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% Secure
Call me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% SecureCall me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% Secure
Call me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% SecurePooja Nehwal
 
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...amitlee9823
 
Discover Why Less is More in B2B Research
Discover Why Less is More in B2B ResearchDiscover Why Less is More in B2B Research
Discover Why Less is More in B2B Researchmichael115558
 
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort ServiceBDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort ServiceDelhi Call girls
 

Recently uploaded (20)

Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
 
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get CytotecAbortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
 
Determinants of health, dimensions of health, positive health and spectrum of...
Determinants of health, dimensions of health, positive health and spectrum of...Determinants of health, dimensions of health, positive health and spectrum of...
Determinants of health, dimensions of health, positive health and spectrum of...
 
CebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptxCebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptx
 
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
 
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdf
 
Data-Analysis for Chicago Crime Data 2023
Data-Analysis for Chicago Crime Data 2023Data-Analysis for Chicago Crime Data 2023
Data-Analysis for Chicago Crime Data 2023
 
Carero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptxCarero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptx
 
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
 
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 nightCheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
 
BPAC WITH UFSBI GENERAL PRESENTATION 18_05_2017-1.pptx
BPAC WITH UFSBI GENERAL PRESENTATION 18_05_2017-1.pptxBPAC WITH UFSBI GENERAL PRESENTATION 18_05_2017-1.pptx
BPAC WITH UFSBI GENERAL PRESENTATION 18_05_2017-1.pptx
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysis
 
Zuja dropshipping via API with DroFx.pptx
Zuja dropshipping via API with DroFx.pptxZuja dropshipping via API with DroFx.pptx
Zuja dropshipping via API with DroFx.pptx
 
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
 
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
Best VIP Call Girls Noida Sector 22 Call Me: 8448380779
 
Call me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% Secure
Call me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% SecureCall me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% Secure
Call me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% Secure
 
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
 
Discover Why Less is More in B2B Research
Discover Why Less is More in B2B ResearchDiscover Why Less is More in B2B Research
Discover Why Less is More in B2B Research
 
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort ServiceBDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
 

Measuring the IQ of your Threat Intelligence Feeds (#tiqtest)

  • 1. Measuring  the  IQ  of  your  Threat   Intelligence  Feeds  (#TIQtest)   Alex  Pinto   MLSec  Project     @alexcpsec   @MLSecProject! Kyle  Maxwell   Researcher   @kylemaxwell!
  • 2. Alex  Pinto   •  Science  guy  at  MLSec  Project   •  ML  trainer   •  Network  security  aficionado     •  Tortured  by  SIEMs  as  a  child   •  Hacker  Spirit  Animal™:   CAFFEINATED  CAPYBARA! whoami(s)   Kyle  Maxwell   •  Researcher  at  [REDACTED]   •  Math  Smuggler   •  Recovering  Incident  Responder   •  GPL  zealot   •  Hacker  Spirit  Animal™:   AXIOMATIC  ARMADILLO! (hUps://secure.flickr.com/photos/kobashi_san/)   (hUp://www.langorigami.com/art/gallery/ gallery.php?tag=mammals&name=armadillo)  
  • 3. •  Threat  Intel  102   •  Measuring  Intelligence   •  Data  Preparaaon   •  Tesang  the  Data   •  Tools:   •  COMBINE   •  TIQ-­‐TEST   •  Some  parang  ideas   Agenda   (hUp://www.savagechickens.com/2008/12/iq-­‐test.html)  
  • 4. Threat  Intel  102:  Capability  and   Intent   •  What  are  they  able  to  do?   •  What  are  they  intending  to  do?!
  • 5. Threat  Intel  102:  Cage  Matches   •  Signatures  vs  Indicators   •  Data  vs  Intelligence   •  Tacacal  vs  Strategic   •  Atomic  vs  Composite  
  • 6. Threat  Intel  102:  Pyramid  of  Pain   “Simple”  and  “easy”  aren’t  always   (David  Bianco  –  Pyramid  of  Pain)  
  • 7. What  about  IP  addresses?   •  Approximately  same  value  as  hostnames  (APT  vs  DGA)   •  Finite  resource  (unal  IPv6,  that  is)   •  Managed  /  controlled  by  orgs   •  Difficulty  /  economic  incenaves  /  implied  “cost”   •  Also,  recyclable     (hUps://xkcd.com/865/)  
  • 8. Given  IP  addresses  harvested  from   TI  feeds,  can  we  measure  how   much  they  “help”  our  defense   metrics?!
  • 9. Introducing  TIQ-­‐TEST   •  All  these  tests  are  available  as  R  funcaons  at     •  hUps://github.com/mlsecproject/aq-­‐test   •  Have  fun,  prove  me  wrong,  suggest  stuff   •  Tools  that  implement  those  tests   •  Sample  data  +  R  Markdown  file   •  The  excuse  to  learn  a  staasacal  language  you  were  waiang   for!  
  • 10. Data  Sources  –  Types  of  data   •  Extract  the  “raw”  informaaon  from  indicator  feeds   •  Both  IP  addresses  and  hostnames  were  extracted  
  • 11. Data  Sources  –  Feeds  Selected   •  Data  was  separated  into  “inbound”  and  “outbound”  
  • 12. Data  PreparaLon  and  Cleansing   •  Convert  the  hostname  data  to  IP  addresses:   •  Acave  IP  addresses  for  the  respecave  date  (“A”  query)   •  Passive  DNS  from  Farsight  Security  (DNSDB)   •  We  removed  non-­‐public  IPs  from  the  dataset  (RFC1918)     •  Yeah,  we  know  it  is  a  “parking  technique”   (hUps://xkcd.com/742/)  
  • 13. Data  PreparaLon  and  Cleansing   •  For  each  IP  record  (including  the  ones  from  hostnames):   •  Add  asnumber  and  asname  (from  MaxMind  ASN  DB)   •  Add  country  (from  MaxMind  GeoLite  DB)   •  Add  rhost  (again  from  DNSDB)  –  most  popular  “PTR”   •  The  experiments  will  be  around  ASNs  and  Geolocaaon  
  • 14. Data  PreparaLon  and  Cleansing   •  However,  we  will  NOT  be  using  maps.  Just  let  it  go.  
  • 15. Data  PreparaLon  and  Cleansing   •  Small  enriched  sample:!
  • 16. TesLng  the  Data   •  Let’s  generate  some  interesang  metrics:   •  NOVELTY  –  How  oxen  do  they  update  themselves?   •  OVERLAP  –  How  do  they  compare  to  what  you  got?   •  POPULATION  –  what  is  in  them  anyway?   •  Populaaon  is  tricky:   •  Could  mean  the  enare  world  (all  IPv4  space)   •  Should  ideally  mean  YOUR  world  
  • 17. But  WHAT  IS  THE  IQ?!??1?   •  We  will  withhold  judgment   •  The  best  data  composiaon  is  the  best  one  for  you   •  We  will  do  our  best  to  explain  results  so  you  can  decide.   •  Maybe  on  further  (or  more  private)  research…      
  • 18. Novelty  Test  –  measuring  added   and  dropped  indicators!
  • 21. Overlap  Test  –  More  data  is  beUer,   but  make  sure  it  is  not  the  same   data!
  • 25. PopulaLon  Test   •  Let  us  use  the  ASN  and   GeoIP  databases  that  we   used  to  enrich  our  data  as  a   reference  of  the  “true”   populaaon.     •  But,  but,  human  beings  are   unpredictable!  We  will   never  be  able  to  forecast   this!      
  • 26.
  • 27.
  • 28. Can  we  get  a  beSer  look?   •  Don’t  like  squinang  either   •  Staasacal  inference-­‐based  comparison  models   (hypothesis  tesang)   •  Exact  binomial  tests  (when  we  have  the  “true”  pop)   •  Chi-­‐squared  proporaon  tests  (similar  to   independence  tests)    
  • 29. Can  we  get  a  beSer  look?   •  We  can  beUer  esamate,  with  confidence  intervals,  our   measures  of  error.   •  Also,  p-­‐values!  (with  apologies  to  Alex  HuUon)   •  We  promise  to  be  very  conservaave  in  using  them.    
  • 30.
  • 31. Hacker  Spirit  Animal™  Guide   •  US  –  Eagle   •  CA  –  Moose   •  FR  –  Frog   •  GB  –  Bulldog   •  AU  –  Koala   •  BR  –  Capybara  /  Toucan   •  Texas  –  Armadillo   •  Disclaimer:  we  do  not  endorse  Geolocaaon-­‐based  aUribuaon  
  • 33.
  • 34.
  • 35. Introducing  COMBINE   •  Harvesang  feeds   takes  some  work.   •  Most  of  us  let   somebody  else  do  it   without  thinking   about  what  it   actually  takes.  
  • 37. Introducing  COMBINE   •  Components:   1.  Reaper  gathers  the  threat  data  directly  from  feeds.   2.  Thresher  normalizes  it  into  a  simplisac  data  model.   3.  Winnower  opaonally  performs  basic  validaaon  or   enrichment.   4.  Baler  transforms  the  data  into  CybOX,  CSV,  JSON,  and   CIM.  (Only  CSV  and  JSON  work  right  now).  Could  also   write  others  fairly  easily.  (nudge  nudge,  wink  wink)  
  • 38. Introducing  COMBINE   •  Always  trying  to  feed   it  more.  Lots  of   possibiliaes,   including  your  own   data  sources.   •  We  clearly  do  NOT   endorse  any  included   feeds.    
  • 39. Introducing  COMBINE   •  Enrichments  -­‐  think   metadata.     •  AS,  geolocaaon   •  DNS  resoluaons   courtesy  of  Farsight   DNSDB     •  Ask  them  for  an   API  key  to  test  it,  tell   them  Alex  Pinto   sent  you  ;)    
  • 40. MLSec  Project   •  Both  projects  have  been  released  as  GPLv3  by  MLSec  Project   •  Will  replace  the  internal  versions  we  have  on  the  main  code     •  Looking  for  paracipants  and  data  sharing  agreements   •  Liked  TIQ-­‐TEST?  We  can  benchmark  your  private  feeds  using   these  and  other  techniques     •  Visit  hSps://www.mlsecproject.org  ,  message  @MLSecProject   or  just  e-­‐mail  me.!
  • 41. Take  Aways   •  Analyze  your  data.     •  Extract  value  from  it!   •  Try  before  you  buy!  Different  test   results  mean  different  things  to   different  orgs.   •  Use  the  tools!  Suggest  new  tests!   •  Share  data  with  us!  We  take  good   care  of  it,  make  sure  it  gets   proper  exercise.    
  • 42. Thanks!   •  Q&A?   •  Feedback!   Alex  Pinto     @alexcpsec   @MLSecProject   ”The  measure  of  intelligence  is  the  ability  to  change."                  -­‐  Albert  Einstein     Kyle  Maxwell   @kylemaxwell