SlideShare a Scribd company logo

Art into Science 2017 - Investigation Theory: A Cognitive Approach

•Download as PPTX, PDF•
•
C
chrissanders88

This presentation was delivered at Art into Science 2017 in Austin, TX. I discuss the ongoing cognitive crisis in information security, and present original research methods and results related to the investigation process.

Technology
1 of 23
Investigation Theory A Cognitive Approach Chris Sanders
Chris Sanders (@chrissanders88)  Analyst @ FireEye  Founder @ Rural Tech Fund  PhD Researcher  GSE # 64  BBQ Pit Master  Author:  Practical Packet Analysis  Applied NSM  Investigation Theory Course
Symptoms of a Cognitive Crisis 1. Demand for expertise greatly outweights supply 2. Most information cannot be trusted or validated 3. Inability to mobilize and tackle big systemic issues
Ethnography of the SOC “An analyst’s job is highly dynamic and requires dealing with constantly evolving threats. Doing the job is more art than science. Ad hoc, on-the-job training for new analysts is the norm." Sundaramurthy, S. C., McHugh, J., Ou, X., Rajagopalan, S. R., & Wesch, M. (2014). An anthropological approach to studying CSIRTs. Network, 100, 2.
Ethnography of the SOC “The profession [security] is so nascent that the how-tos have not been fully realized even by the people who have the knowledge…the process required to connect the dots is unclear even to analysts. Sundaramurthy, S. C., McHugh, J., Ou, X., Rajagopalan, S. R., & Wesch, M. (2014). An anthropological approach to studying CSIRTs. Network, 100, 2.
Symptoms of a Cognitive Crisis 1. Demand for expertise greatly outweights supply 2. Most information cannot be trusted or validated 3. Inability to mobilize and tackle big systemic issues
The Cognitive Revolution 1. Understand the processes used to draw conclusions 2. Develop repeatable methods and techniques 3. Build and advocate training that teaches practitioners how to think
What separates novice and expert analysts?
Mapping the Investigation  Sample:  Novice and expert analysts  Methodology:  30+ case studies  Stimulated recall interviews  Focus on individual investigations of varying types  Perform key phrase analysis – analyze results
Key Phrase Mapping  Dual Process Theory  Intuition: Implicit, unconscious, fast  Reflection: Explicit, controlled, slow Intuition Experimentation Restructuring Imagination Incubation Metacognition Evaluation Goal Setting Making Plans Reflection Analytically Viewing Data Rule-Based Reasoning Considering Alternatives
Results Novices Experts Intuition Metacognition Reflection
Analyzing the Flow of the Investigation
Investigations as Mental Labyrinths  The investigation is the core construct of information security.  How do we study them when everyone has a different toolset?  Follow the Data! Alert OSINT Reputation File Hash Sandbox Behaviors AV Detections (VT) Imphash More File Hashes Friendly Host Network PCAP Host Windows Logs Security Log System Log App LogRegistry File System Hostile Host Network PCAP Flow
Studying the Investigation Process
Studying the Investigation Process
What data did analysts look at first? 72% 16% 12% Observed PCAP Flow OSINT Data Suggests:  Analysts prefer a higher context data set…  …even if other data sets are available  …even if lower context data sets can lead to a resolution.
Did the first move affect analysis speed? Data Suggests:  While PCAP provides richer context, it may slow down the investigation if that’s where you start  Starting with a lower context data source can increase speed when working with higher context data 16 10 9 PCAP Flow OSINT Avg Time to Close
What happens when Bro data replaces PCAP? 46% 25% 29% Observed (Bro) Bro Flow OSINT 72% 16% 12% Observed (PCAP) PCAP Flow OSINT
What happens when Bro data replaces PCAP? 16 10 9 PCAP Flow OSINT Avg Time to Close (PCAP) 10 10 11 Bro Flow OSINT Avg Time to Close (Bro) Data Suggests:  Better organization of high context data sources can yield improvements in analysts performance
What data sources were viewed most and least frequently? Data Suggests:  Network data is used more frequently than host data…  …even when host data can be used exclusively to resolve.  …even when easy access is provided to host sources.  Revisting data is more prevalent on higher context data sources Data Sources Viewed Data Sources Revisited PCA P 84% Flow 11% OSIN T 5%
How many steps were taken to make a disposition judgement? Data Suggests:  At some point, the number of data sources you investigate impacts the speed of the investigation  Understanding where data exists and when to use it can impact analysis speed 6 12 9 3 0 5 10 15 6-10 11-15 16-20 21-25 Number of Steps 9 12 14 24 0 5 10 15 20 25 30 6-10 11-15 16-20 21-25 Avg Time to Close
Did analysts investigate friendly or hostile systems first? 9% 91% Observed Friendly Hostile Data Suggests:  Analysts are more compelled to investigate unknown external threats than internal systems  Analysts don’t fully understand their own techniques 41% 59% Friendly Friendly Hostile
Thank You! Mail: chris@chrissanders.org Twitter: @chrissanders88 Blog: chrissanders.org Training: chrissanders.org/training

Recommended

Threats that Matter - Murray State University 2017
Threats that Matter - Murray State University 2017Threats that Matter - Murray State University 2017
Threats that Matter - Murray State University 2017
chrissanders88
 
Abstract Tools for Effective Threat Hunting
Abstract Tools for Effective Threat HuntingAbstract Tools for Effective Threat Hunting
Abstract Tools for Effective Threat Hunting
chrissanders88
 
SOC2016 - The Investigation Labyrinth
SOC2016 - The Investigation LabyrinthSOC2016 - The Investigation Labyrinth
SOC2016 - The Investigation Labyrinth
chrissanders88
 
Building a Threat Hunting Practice in the Cloud
Building a Threat Hunting Practice in the CloudBuilding a Threat Hunting Practice in the Cloud
Building a Threat Hunting Practice in the Cloud
ProtectWise
 
BSA2016 - Honeypots for Network Security Monitoring
BSA2016 - Honeypots for Network Security MonitoringBSA2016 - Honeypots for Network Security Monitoring
BSA2016 - Honeypots for Network Security Monitoring
chrissanders88
 
Threat Hunting 102: Beyond the Basics
Threat Hunting 102: Beyond the BasicsThreat Hunting 102: Beyond the Basics
Threat Hunting 102: Beyond the Basics
Cybereason
 
Building a Successful Threat Hunting Program
Building a Successful Threat Hunting ProgramBuilding a Successful Threat Hunting Program
Building a Successful Threat Hunting Program
Carl C. Manion
 
Threat Hunting
Threat HuntingThreat Hunting
Threat Hunting
Splunk
 

Recommended

Threats that Matter - Murray State University 2017
Threats that Matter - Murray State University 2017Threats that Matter - Murray State University 2017
Threats that Matter - Murray State University 2017
chrissanders88
 
Abstract Tools for Effective Threat Hunting
Abstract Tools for Effective Threat HuntingAbstract Tools for Effective Threat Hunting
Abstract Tools for Effective Threat Hunting
chrissanders88
 
SOC2016 - The Investigation Labyrinth
SOC2016 - The Investigation LabyrinthSOC2016 - The Investigation Labyrinth
SOC2016 - The Investigation Labyrinth
chrissanders88
 
Building a Threat Hunting Practice in the Cloud
Building a Threat Hunting Practice in the CloudBuilding a Threat Hunting Practice in the Cloud
Building a Threat Hunting Practice in the Cloud
ProtectWise
 
BSA2016 - Honeypots for Network Security Monitoring
BSA2016 - Honeypots for Network Security MonitoringBSA2016 - Honeypots for Network Security Monitoring
BSA2016 - Honeypots for Network Security Monitoring
chrissanders88
 
Threat Hunting 102: Beyond the Basics
Threat Hunting 102: Beyond the BasicsThreat Hunting 102: Beyond the Basics
Threat Hunting 102: Beyond the Basics
Cybereason
 
Building a Successful Threat Hunting Program
Building a Successful Threat Hunting ProgramBuilding a Successful Threat Hunting Program
Building a Successful Threat Hunting Program
Carl C. Manion
 
Threat Hunting
Threat HuntingThreat Hunting
Threat Hunting
Splunk
 
Minding the Metacognitive Gap - BSides NOLA
Minding the Metacognitive Gap - BSides NOLAMinding the Metacognitive Gap - BSides NOLA
Minding the Metacognitive Gap - BSides NOLA
chrissanders88
 
Enabling effective hunt teaming and incident response
Enabling effective hunt teaming and incident responseEnabling effective hunt teaming and incident response
Enabling effective hunt teaming and incident response
jeffmcjunkin
 
See Clearly and Respond Quickly from the Network to the Endpoint
See Clearly and Respond Quickly from the Network to the EndpointSee Clearly and Respond Quickly from the Network to the Endpoint
See Clearly and Respond Quickly from the Network to the Endpoint
ProtectWise
 
Threat Hunting Report
Threat Hunting Report Threat Hunting Report
Threat Hunting Report
Morane Decriem
 
Cyber Threat Hunting with Phirelight
Cyber Threat Hunting with PhirelightCyber Threat Hunting with Phirelight
Cyber Threat Hunting with Phirelight
Hostway|HOSTING
 
Threat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-onThreat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-on
Splunk
 
Managing Indicator Deprecation in ThreatConnect
Managing Indicator Deprecation in ThreatConnectManaging Indicator Deprecation in ThreatConnect
Managing Indicator Deprecation in ThreatConnect
ThreatConnect
 
Threat Hunting with Splunk
Threat Hunting with SplunkThreat Hunting with Splunk
Threat Hunting with Splunk
Splunk
 
Threat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-onThreat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-on
Splunk
 
Episode IV: A New Scope
Episode IV: A New ScopeEpisode IV: A New Scope
Episode IV: A New Scope
ThreatConnect
 
Threat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonThreat hunting - Every day is hunting season
Threat hunting - Every day is hunting season
Ben Boyd
 
Threat Hunting Workshop
Threat Hunting WorkshopThreat Hunting Workshop
Threat Hunting Workshop
Splunk
 
(SACON) Shomiron das gupta - threat hunting use cases
(SACON) Shomiron das gupta - threat hunting use cases(SACON) Shomiron das gupta - threat hunting use cases
(SACON) Shomiron das gupta - threat hunting use cases
Priyanka Aash
 
Advanced Threat Hunting - Botconf 2017
Advanced Threat Hunting - Botconf 2017Advanced Threat Hunting - Botconf 2017
Advanced Threat Hunting - Botconf 2017
Kevin Finley
 
MITRE ATT&CKcon 2.0: Prioritizing Data Sources for Minimum Viable Detection; ...
MITRE ATT&CKcon 2.0: Prioritizing Data Sources for Minimum Viable Detection; ...MITRE ATT&CKcon 2.0: Prioritizing Data Sources for Minimum Viable Detection; ...
MITRE ATT&CKcon 2.0: Prioritizing Data Sources for Minimum Viable Detection; ...
MITRE - ATT&CKcon
 
Threat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement MatriceThreat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement Matrice
Vishal Kumar
 
Threat Hunting with Splunk
Threat Hunting with SplunkThreat Hunting with Splunk
Threat Hunting with Splunk
Splunk
 
Threat Hunting with Data Science
Threat Hunting with Data ScienceThreat Hunting with Data Science
Threat Hunting with Data Science
Austin Taylor
 
Threat hunting and achieving security maturity
Threat hunting and achieving security maturityThreat hunting and achieving security maturity
Threat hunting and achieving security maturity
DNIF
 
Intelligence driven defense webinar
Intelligence driven defense webinarIntelligence driven defense webinar
Intelligence driven defense webinar
ThreatConnect
 
Applied Detection and Analysis Using Flow Data - MIRCon 2014
Applied Detection and Analysis Using Flow Data - MIRCon 2014Applied Detection and Analysis Using Flow Data - MIRCon 2014
Applied Detection and Analysis Using Flow Data - MIRCon 2014
chrissanders88
 
Using Canary Honeypots for Network Security Monitoring
Using Canary Honeypots for Network Security MonitoringUsing Canary Honeypots for Network Security Monitoring
Using Canary Honeypots for Network Security Monitoring
chrissanders88
 

More Related Content

What's hot

Minding the Metacognitive Gap - BSides NOLA
Minding the Metacognitive Gap - BSides NOLAMinding the Metacognitive Gap - BSides NOLA
Minding the Metacognitive Gap - BSides NOLA
chrissanders88
 
Advanced Threat Hunting - Botconf 2017
Advanced Threat Hunting - Botconf 2017Advanced Threat Hunting - Botconf 2017
Advanced Threat Hunting - Botconf 2017
Kevin Finley
 
Threat Hunting with Splunk
Threat Hunting with SplunkThreat Hunting with Splunk
Threat Hunting with Splunk
Splunk
 
Threat Hunting with Data Science
Threat Hunting with Data ScienceThreat Hunting with Data Science
Threat Hunting with Data Science
Austin Taylor
 
Intelligence driven defense webinar
Intelligence driven defense webinarIntelligence driven defense webinar
Intelligence driven defense webinar
ThreatConnect
 

What's hot (20)

Minding the Metacognitive Gap - BSides NOLA
Minding the Metacognitive Gap - BSides NOLAMinding the Metacognitive Gap - BSides NOLA
Minding the Metacognitive Gap - BSides NOLA
 
Enabling effective hunt teaming and incident response
Enabling effective hunt teaming and incident responseEnabling effective hunt teaming and incident response
Enabling effective hunt teaming and incident response
 
See Clearly and Respond Quickly from the Network to the Endpoint
See Clearly and Respond Quickly from the Network to the EndpointSee Clearly and Respond Quickly from the Network to the Endpoint
See Clearly and Respond Quickly from the Network to the Endpoint
 
Threat Hunting Report
Threat Hunting Report Threat Hunting Report
Threat Hunting Report
 
Cyber Threat Hunting with Phirelight
Cyber Threat Hunting with PhirelightCyber Threat Hunting with Phirelight
Cyber Threat Hunting with Phirelight
 
Threat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-onThreat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-on
 
Managing Indicator Deprecation in ThreatConnect
Managing Indicator Deprecation in ThreatConnectManaging Indicator Deprecation in ThreatConnect
Managing Indicator Deprecation in ThreatConnect
 
Threat Hunting with Splunk
Threat Hunting with SplunkThreat Hunting with Splunk
Threat Hunting with Splunk
 
Threat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-onThreat Hunting with Splunk Hands-on
Threat Hunting with Splunk Hands-on
 
Episode IV: A New Scope
Episode IV: A New ScopeEpisode IV: A New Scope
Episode IV: A New Scope
 
Threat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonThreat hunting - Every day is hunting season
Threat hunting - Every day is hunting season
 
Threat Hunting Workshop
Threat Hunting WorkshopThreat Hunting Workshop
Threat Hunting Workshop
 
(SACON) Shomiron das gupta - threat hunting use cases
(SACON) Shomiron das gupta - threat hunting use cases(SACON) Shomiron das gupta - threat hunting use cases
(SACON) Shomiron das gupta - threat hunting use cases
 
Advanced Threat Hunting - Botconf 2017
Advanced Threat Hunting - Botconf 2017Advanced Threat Hunting - Botconf 2017
Advanced Threat Hunting - Botconf 2017
 
MITRE ATT&CKcon 2.0: Prioritizing Data Sources for Minimum Viable Detection; ...
MITRE ATT&CKcon 2.0: Prioritizing Data Sources for Minimum Viable Detection; ...MITRE ATT&CKcon 2.0: Prioritizing Data Sources for Minimum Viable Detection; ...
MITRE ATT&CKcon 2.0: Prioritizing Data Sources for Minimum Viable Detection; ...
 
Threat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement MatriceThreat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement Matrice
 
Threat Hunting with Splunk
Threat Hunting with SplunkThreat Hunting with Splunk
Threat Hunting with Splunk
 
Threat Hunting with Data Science
Threat Hunting with Data ScienceThreat Hunting with Data Science
Threat Hunting with Data Science
 
Threat hunting and achieving security maturity
Threat hunting and achieving security maturityThreat hunting and achieving security maturity
Threat hunting and achieving security maturity
 
Intelligence driven defense webinar
Intelligence driven defense webinarIntelligence driven defense webinar
Intelligence driven defense webinar
 

Viewers also liked

Cognitive approach & therapies
Cognitive approach & therapiesCognitive approach & therapies
Cognitive approach & therapies
sssfcpsychology
 
BSides Augusta 2015 - Building a Better Analyst Using Cognitive Psychology
BSides Augusta 2015 - Building a Better Analyst Using Cognitive PsychologyBSides Augusta 2015 - Building a Better Analyst Using Cognitive Psychology
BSides Augusta 2015 - Building a Better Analyst Using Cognitive Psychology
chrissanders88
 
Cronograma de actividades 8°- 1er periodo 2017(corregido)
Cronograma de actividades 8°- 1er periodo 2017(corregido)Cronograma de actividades 8°- 1er periodo 2017(corregido)
Cronograma de actividades 8°- 1er periodo 2017(corregido)
jhonathanmaradey
 
Real-time data analysis using ELK
Real-time data analysis using ELKReal-time data analysis using ELK
Real-time data analysis using ELK
Jettro Coenradie
 

Viewers also liked (20)

Applied Detection and Analysis Using Flow Data - MIRCon 2014
Applied Detection and Analysis Using Flow Data - MIRCon 2014Applied Detection and Analysis Using Flow Data - MIRCon 2014
Applied Detection and Analysis Using Flow Data - MIRCon 2014
 
Using Canary Honeypots for Network Security Monitoring
Using Canary Honeypots for Network Security MonitoringUsing Canary Honeypots for Network Security Monitoring
Using Canary Honeypots for Network Security Monitoring
 
Applied Detection and Analysis with Flow Data - SO Con 2014
Applied Detection and Analysis with Flow Data - SO Con 2014Applied Detection and Analysis with Flow Data - SO Con 2014
Applied Detection and Analysis with Flow Data - SO Con 2014
 
CISSA Lightning Talk - Building a Malware Analysis Lab on a Budget
CISSA Lightning Talk - Building a Malware Analysis Lab on a BudgetCISSA Lightning Talk - Building a Malware Analysis Lab on a Budget
CISSA Lightning Talk - Building a Malware Analysis Lab on a Budget
 
Developing Analytic Technique and Defeating Cognitive Bias in Security
Developing Analytic Technique and Defeating Cognitive Bias in SecurityDeveloping Analytic Technique and Defeating Cognitive Bias in Security
Developing Analytic Technique and Defeating Cognitive Bias in Security
 
Windows Threat Hunting
Windows Threat HuntingWindows Threat Hunting
Windows Threat Hunting
 
Threat Hunting with Splunk
Threat Hunting with SplunkThreat Hunting with Splunk
Threat Hunting with Splunk
 
Splunk for Developers Breakout Session
Splunk for Developers Breakout SessionSplunk for Developers Breakout Session
Splunk for Developers Breakout Session
 
Splunk for Security - Hands-On
Splunk for Security - Hands-On Splunk for Security - Hands-On
Splunk for Security - Hands-On
 
Web Application Security 101 - 12 Logging
Web Application Security 101 - 12 LoggingWeb Application Security 101 - 12 Logging
Web Application Security 101 - 12 Logging
 
09 application security fundamentals - part 2 - security mechanisms - logging
09 application security fundamentals - part 2 - security mechanisms - logging09 application security fundamentals - part 2 - security mechanisms - logging
09 application security fundamentals - part 2 - security mechanisms - logging
 
Cognitive approach & therapies
Cognitive approach & therapiesCognitive approach & therapies
Cognitive approach & therapies
 
Workshop threat-hunting
Workshop threat-huntingWorkshop threat-hunting
Workshop threat-hunting
 
BSides Augusta 2015 - Building a Better Analyst Using Cognitive Psychology
BSides Augusta 2015 - Building a Better Analyst Using Cognitive PsychologyBSides Augusta 2015 - Building a Better Analyst Using Cognitive Psychology
BSides Augusta 2015 - Building a Better Analyst Using Cognitive Psychology
 
Docker Indy Meetup Monitoring 30-Aug-2016
Docker Indy Meetup Monitoring 30-Aug-2016Docker Indy Meetup Monitoring 30-Aug-2016
Docker Indy Meetup Monitoring 30-Aug-2016
 
Cronograma de actividades 8°- 1er periodo 2017(corregido)
Cronograma de actividades 8°- 1er periodo 2017(corregido)Cronograma de actividades 8°- 1er periodo 2017(corregido)
Cronograma de actividades 8°- 1er periodo 2017(corregido)
 
Elastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & KibanaElastic - ELK, Logstash & Kibana
Elastic - ELK, Logstash & Kibana
 
Threat Hunting with Splunk
Threat Hunting with SplunkThreat Hunting with Splunk
Threat Hunting with Splunk
 
SNMP Demystified Part-I
SNMP Demystified Part-ISNMP Demystified Part-I
SNMP Demystified Part-I
 
Real-time data analysis using ELK
Real-time data analysis using ELKReal-time data analysis using ELK
Real-time data analysis using ELK
 

Similar to Art into Science 2017 - Investigation Theory: A Cognitive Approach

ODSC East 2017: Data Science Models For Good
ODSC East 2017: Data Science Models For GoodODSC East 2017: Data Science Models For Good
ODSC East 2017: Data Science Models For Good
Karry Lu
 
Emerging Data Citation Infrastructure
Emerging Data Citation InfrastructureEmerging Data Citation Infrastructure
Emerging Data Citation Infrastructure
Micah Altman
 
Data management plans
Data management plansData management plans
Data management plans
Brad Houston
 
FAIRness Assessment of the Library of Integrated Network-based Cellular Signa...
FAIRness Assessment of the Library of Integrated Network-based Cellular Signa...FAIRness Assessment of the Library of Integrated Network-based Cellular Signa...
FAIRness Assessment of the Library of Integrated Network-based Cellular Signa...
Kathleen Jagodnik
 
Data management plans (dmp) for nsf
Data management plans (dmp) for nsfData management plans (dmp) for nsf
Data management plans (dmp) for nsf
Brad Houston
 
Data management plans (dmp) for nsf
Data management plans (dmp) for nsfData management plans (dmp) for nsf
Data management plans (dmp) for nsf
Brad Houston
 
Making an impact with data science
Making an impact with data scienceMaking an impact with data science
Making an impact with data science
Jordan Engbers
 
Data management plans
Data management plansData management plans
Data management plans
Brad Houston
 
kantorNSF-NIJ-ISI-03-06-04.ppt
kantorNSF-NIJ-ISI-03-06-04.pptkantorNSF-NIJ-ISI-03-06-04.ppt
kantorNSF-NIJ-ISI-03-06-04.ppt
butest
 

Similar to Art into Science 2017 - Investigation Theory: A Cognitive Approach (20)

ODSC East 2017: Data Science Models For Good
ODSC East 2017: Data Science Models For GoodODSC East 2017: Data Science Models For Good
ODSC East 2017: Data Science Models For Good
 
Emerging Data Citation Infrastructure
Emerging Data Citation InfrastructureEmerging Data Citation Infrastructure
Emerging Data Citation Infrastructure
 
Rearch methodology
Rearch methodologyRearch methodology
Rearch methodology
 
PEARC17: ARCC Identity and Access Management, Security and related topics. Cy...
PEARC17: ARCC Identity and Access Management, Security and related topics. Cy...PEARC17: ARCC Identity and Access Management, Security and related topics. Cy...
PEARC17: ARCC Identity and Access Management, Security and related topics. Cy...
 
Twitter sentiment classifications 1
Twitter sentiment classifications 1Twitter sentiment classifications 1
Twitter sentiment classifications 1
 
Data management plans
Data management plansData management plans
Data management plans
 
Study on Cyber Security:Establishing a Sustainable Cyber Security Framework f...
Study on Cyber Security:Establishing a Sustainable Cyber Security Framework f...Study on Cyber Security:Establishing a Sustainable Cyber Security Framework f...
Study on Cyber Security:Establishing a Sustainable Cyber Security Framework f...
 
Hattrick-Simpers MRS Webinar on AI in Materials
Hattrick-Simpers MRS Webinar on AI in MaterialsHattrick-Simpers MRS Webinar on AI in Materials
Hattrick-Simpers MRS Webinar on AI in Materials
 
A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...
A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...
A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...
 
The Myths + Realities of Machine-Learning Cybersecurity
The Myths + Realities of Machine-Learning CybersecurityThe Myths + Realities of Machine-Learning Cybersecurity
The Myths + Realities of Machine-Learning Cybersecurity
 
informatics_future.pdf
informatics_future.pdfinformatics_future.pdf
informatics_future.pdf
 
FAIRness Assessment of the Library of Integrated Network-based Cellular Signa...
FAIRness Assessment of the Library of Integrated Network-based Cellular Signa...FAIRness Assessment of the Library of Integrated Network-based Cellular Signa...
FAIRness Assessment of the Library of Integrated Network-based Cellular Signa...
 
Data management plans (dmp) for nsf
Data management plans (dmp) for nsfData management plans (dmp) for nsf
Data management plans (dmp) for nsf
 
Data management plans (dmp) for nsf
Data management plans (dmp) for nsfData management plans (dmp) for nsf
Data management plans (dmp) for nsf
 
Making an impact with data science
Making an impact with data scienceMaking an impact with data science
Making an impact with data science
 
Data management plans
Data management plansData management plans
Data management plans
 
kantorNSF-NIJ-ISI-03-06-04.ppt
kantorNSF-NIJ-ISI-03-06-04.pptkantorNSF-NIJ-ISI-03-06-04.ppt
kantorNSF-NIJ-ISI-03-06-04.ppt
 
Jonathan Breeze, Symplectic
Jonathan Breeze, SymplecticJonathan Breeze, Symplectic
Jonathan Breeze, Symplectic
 
BLC & Digital Science: Jonathan Breeze, Symplectic
BLC & Digital Science: Jonathan Breeze, SymplecticBLC & Digital Science: Jonathan Breeze, Symplectic
BLC & Digital Science: Jonathan Breeze, Symplectic
 
Competitive Intelligence: An Island of Structure in an Unstructured Ocean
Competitive Intelligence: An Island of Structure in an Unstructured OceanCompetitive Intelligence: An Island of Structure in an Unstructured Ocean
Competitive Intelligence: An Island of Structure in an Unstructured Ocean
 

Recently uploaded

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Angeliki Cooney
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Orbitshub
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
UiPathCommunity
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Recently uploaded (20)

Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 

Art into Science 2017 - Investigation Theory: A Cognitive Approach

  • 2. Chris Sanders (@chrissanders88)  Analyst @ FireEye  Founder @ Rural Tech Fund  PhD Researcher  GSE # 64  BBQ Pit Master  Author:  Practical Packet Analysis  Applied NSM  Investigation Theory Course
  • 3. Symptoms of a Cognitive Crisis 1. Demand for expertise greatly outweights supply 2. Most information cannot be trusted or validated 3. Inability to mobilize and tackle big systemic issues
  • 4. Ethnography of the SOC “An analyst’s job is highly dynamic and requires dealing with constantly evolving threats. Doing the job is more art than science. Ad hoc, on-the-job training for new analysts is the norm." Sundaramurthy, S. C., McHugh, J., Ou, X., Rajagopalan, S. R., & Wesch, M. (2014). An anthropological approach to studying CSIRTs. Network, 100, 2.
  • 5. Ethnography of the SOC “The profession [security] is so nascent that the how-tos have not been fully realized even by the people who have the knowledge…the process required to connect the dots is unclear even to analysts. Sundaramurthy, S. C., McHugh, J., Ou, X., Rajagopalan, S. R., & Wesch, M. (2014). An anthropological approach to studying CSIRTs. Network, 100, 2.
  • 6. Symptoms of a Cognitive Crisis 1. Demand for expertise greatly outweights supply 2. Most information cannot be trusted or validated 3. Inability to mobilize and tackle big systemic issues
  • 7. The Cognitive Revolution 1. Understand the processes used to draw conclusions 2. Develop repeatable methods and techniques 3. Build and advocate training that teaches practitioners how to think
  • 9. Mapping the Investigation  Sample:  Novice and expert analysts  Methodology:  30+ case studies  Stimulated recall interviews  Focus on individual investigations of varying types  Perform key phrase analysis – analyze results
  • 10. Key Phrase Mapping  Dual Process Theory  Intuition: Implicit, unconscious, fast  Reflection: Explicit, controlled, slow Intuition Experimentation Restructuring Imagination Incubation Metacognition Evaluation Goal Setting Making Plans Reflection Analytically Viewing Data Rule-Based Reasoning Considering Alternatives
  • 12. Analyzing the Flow of the Investigation
  • 13. Investigations as Mental Labyrinths  The investigation is the core construct of information security.  How do we study them when everyone has a different toolset?  Follow the Data! Alert OSINT Reputation File Hash Sandbox Behaviors AV Detections (VT) Imphash More File Hashes Friendly Host Network PCAP Host Windows Logs Security Log System Log App LogRegistry File System Hostile Host Network PCAP Flow
  • 16. What data did analysts look at first? 72% 16% 12% Observed PCAP Flow OSINT Data Suggests:  Analysts prefer a higher context data set…  …even if other data sets are available  …even if lower context data sets can lead to a resolution.
  • 17. Did the first move affect analysis speed? Data Suggests:  While PCAP provides richer context, it may slow down the investigation if that’s where you start  Starting with a lower context data source can increase speed when working with higher context data 16 10 9 PCAP Flow OSINT Avg Time to Close
  • 18. What happens when Bro data replaces PCAP? 46% 25% 29% Observed (Bro) Bro Flow OSINT 72% 16% 12% Observed (PCAP) PCAP Flow OSINT
  • 19. What happens when Bro data replaces PCAP? 16 10 9 PCAP Flow OSINT Avg Time to Close (PCAP) 10 10 11 Bro Flow OSINT Avg Time to Close (Bro) Data Suggests:  Better organization of high context data sources can yield improvements in analysts performance
  • 20. What data sources were viewed most and least frequently? Data Suggests:  Network data is used more frequently than host data…  …even when host data can be used exclusively to resolve.  …even when easy access is provided to host sources.  Revisting data is more prevalent on higher context data sources Data Sources Viewed Data Sources Revisited PCA P 84% Flow 11% OSIN T 5%
  • 21. How many steps were taken to make a disposition judgement? Data Suggests:  At some point, the number of data sources you investigate impacts the speed of the investigation  Understanding where data exists and when to use it can impact analysis speed 6 12 9 3 0 5 10 15 6-10 11-15 16-20 21-25 Number of Steps 9 12 14 24 0 5 10 15 20 25 30 6-10 11-15 16-20 21-25 Avg Time to Close
  • 22. Did analysts investigate friendly or hostile systems first? 9% 91% Observed Friendly Hostile Data Suggests:  Analysts are more compelled to investigate unknown external threats than internal systems  Analysts don’t fully understand their own techniques 41% 59% Friendly Friendly Hostile
  • 23. Thank You! Mail: chris@chrissanders.org Twitter: @chrissanders88 Blog: chrissanders.org Training: chrissanders.org/training

Editor's Notes

  1. Every town had one doctor and they were also your vet Many home remedies spawn from this time – milk as a treatment for stomach ulcers is an example Major health crises were frequent and impossible to control
  2. Anthroplogists Ethnography
  3. Is this an individual thing, or is it a systemic problem?
  4. Every town had one doctor and they were also your vet Many home remedies spawn from this time – milk as a treatment for stomach ulcers is an example Major health crises were frequent and impossible to control
  5. We ended up with an investigation game
  6. Sidebar: Analysts looked at the PCAP 100% of the time, even if it wasn’t necessary.
  7. This points to tendencies gained from training. Most shops don’t have easy access to host data.
  8. Anecdotal – Experts I knew took less than 10 steps. Anecdotal – Novices I knew took > 15.