SlideShare a Scribd company logo
1 of 26
MOLOCH:
SEARCHFOR
FULLPACKET
CAPTURE
It is a Great Horned Owl
Project Logo
WHYTHE OWL?
Owls are silent hunters that go after RATs. We think that’s pretty
cool.
3
HISTORY
LESSON
WHYAOL BUILT MOLOCH
WHAT IS MOLOCH?
5
Moloch is an open source, scalable IPv4 packet capture indexing and
database system, built using open source technologies.
• A simple web GUI is provided for browsing,
searching, viewing and exporting PCAP data.
• Web APIs are accessible if you wish to design your
own GUI or directly grab PCAP with various
command line tools for further analysis or
processing.
• Find it on AOL’s GitHub page:
https://github.com/aol/moloch
It’s like AOL Search for PCAP repositories!
WHAT IS MOLOCH NOT?
6
NOT IDS: NO ALERTS
NOT IPV6 (Today)
NOT SLOW
NOT CLOSED
NOT EXPENSIVE
WHYUSE MOLOCH?
7
Real-time capture of network traffic for forensic and
investigative purposes
• Combine the power of Moloch with other indicators (intelligence
feeds, alerting from IDS/anti-virus) to empower your analysts to
quickly and effectively review actions on the network to
determine the validity/threat.
• Review past network traffic for post compromise investigations.
Static PCAP repository
• Import large collections of PCAP that were created by malware.
• Import collections of PCAP from Capture The Flag events.
• Custom tagging of data at time of import.
THE PIECES OF MOLOCH
8
CAPTURE
• A C application that sniffs the network interface, parses the
traffic, and creates the Session Profile Information (SPI data)
and writes it to disk.
DATABASE
• Elasticsearch is used for storing and searching through the SPI
data generated by the capture component.
VIEWER
• A web interface that allows for GUI and API access from remote
hosts to browse or query SPI data and retrieve stored PCAP.
THE PIECES OF MOLOCH:
CAPTURE
9
Libnids based daemon written in C
Can be used to sniff network interface for live capture
Can be called from CLI to do manual imports
Parses layers 3-7 to create SPI data
• Spits them out to the Elasticsearch cluster. A lot like making owl
pellets!
THE PIECES OF MOLOCH:
DATABASE
11
Elasticsearch (http://www.elasticsearch.org)
• Powered by Apache Lucene (http://lucene.apache.org)
• Requests over HTTP(s)
• Results returned in JSON
Nosql
• Network traffic doesn’t fit the mold for relational DBs.
Documented oriented
• Great for lots and lots of network sessions.
Automatic sharding across multiple hosts
• At the time, we skipped SOLR because it couldn’t run distributed.
Fast, scalable, all that goodness
THE PIECES OF MOLOCH:
VIEWER
12
Node.js based application
• Event driven server side JavaScript platform.
• Based on Chrome’s JavaScript runtime.
• Comes with its own HTTP server and easy JSON for
communication.
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP.
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs,
consoles, and command line tools is easy.
• Easy automation to retrieve PCAP or sessions of interest.
THE PIECES OF MOLOCH:
VIEWER
13
Nodejs based application
• Event driven server side JavaScript platform
• Based on Chrome’s JavaScript runtime
• Comes with its own HTTP server and easy JSON for
communication
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs,
consoles, command line tools is easy.
• Easy automation to retrieve pcap or sessions of interest.
THE PIECES OF MOLOCH:
VIEWER
14
Nodejs based application
• Event driven server side JavaScript platform
• Based on Chrome’s JavaScript runtime
• Comes with its own HTTP server and easy JSON for
communication
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs,
consoles, command line tools is easy.
• Easy automation to retrieve pcap or sessions of interest.
ARCHITECTUREOF MOLOCH:
DATAFLOW
15
ARCHITECTUREOF MOLOCH:
MULTINODE WITH CLUSTER
16
ARCHITECTUREOF MOLOCH:
SCALE
17
Packets Captured Kilobytes Saved Sessions Saved
Example: Moloch Capture
Documents Disk Storage (MB)
Example: Elasticsearch
MOLOCH: SPI-DATATYPES
SESSION PROFILE
INFORMATION
18
IP
• Source
• Destination
• Ports
• Protocol
HTTP
• Method
• Status Codes
• Headers
• Content Type
DNS
• IP Address
• Hostnames
MOLOCH: SPI-DATATYPES
SESSION PROFILE
INFORMATION
19
SSL/TLS
• Cert Elements:
• Common Name
• Serial Number
• Alt Names
SSH
• Client Name
• Public Key
• Port
IRC
• Channel Name
• Hostname
MOLOCH: CAPTURE
CREATING SPI DATA
20
MOLOCH: CAPTURE
CREATING SPI DATA
21
MOLOCH: CAPTURE
CREATING SPI DATA
22
MOLOCH: CAPTURE
CREATING SPI DATA
23
MOLOCH: CAPTURE
CREATING SPI DATA
24
MOLOCH: DEMO
25
MOLOCH: QUESTIONS?
26

More Related Content

What's hot

4_Session 1- Universal ZTNA.pptx
4_Session 1- Universal ZTNA.pptx4_Session 1- Universal ZTNA.pptx
4_Session 1- Universal ZTNA.pptx
aungyekhant1
 
LTM essentials
LTM essentialsLTM essentials
LTM essentials
bharadwajv
 
Palo alto networks next generation firewalls
Palo alto networks next generation firewallsPalo alto networks next generation firewalls
Palo alto networks next generation firewalls
Castleforce
 

What's hot (20)

Layer 3 redundancy hsrp
Layer 3 redundancy   hsrpLayer 3 redundancy   hsrp
Layer 3 redundancy hsrp
 
Forescout exam
Forescout examForescout exam
Forescout exam
 
IDENTITY ACCESS MANAGEMENT
IDENTITY ACCESS MANAGEMENTIDENTITY ACCESS MANAGEMENT
IDENTITY ACCESS MANAGEMENT
 
From Cisco ACS to ISE
From Cisco ACS to ISE From Cisco ACS to ISE
From Cisco ACS to ISE
 
4_Session 1- Universal ZTNA.pptx
4_Session 1- Universal ZTNA.pptx4_Session 1- Universal ZTNA.pptx
4_Session 1- Universal ZTNA.pptx
 
Palo Alto Networks 28.5.2013
Palo Alto Networks 28.5.2013Palo Alto Networks 28.5.2013
Palo Alto Networks 28.5.2013
 
Whitepaper IBM Guardium Data Activity Monitor
Whitepaper IBM Guardium Data Activity MonitorWhitepaper IBM Guardium Data Activity Monitor
Whitepaper IBM Guardium Data Activity Monitor
 
Introduction to LTE
Introduction to LTEIntroduction to LTE
Introduction to LTE
 
LTM essentials
LTM essentialsLTM essentials
LTM essentials
 
Datapower Steven Cawn
Datapower Steven CawnDatapower Steven Cawn
Datapower Steven Cawn
 
EMEA Airheads- Layer-3 Redundancy for Mobility Master - ArubaOS 8.x
EMEA Airheads- Layer-3 Redundancy for Mobility Master - ArubaOS 8.xEMEA Airheads- Layer-3 Redundancy for Mobility Master - ArubaOS 8.x
EMEA Airheads- Layer-3 Redundancy for Mobility Master - ArubaOS 8.x
 
SD WAN VS MPLS – Which is better for your Business?
SD WAN VS MPLS – Which is better for your Business?SD WAN VS MPLS – Which is better for your Business?
SD WAN VS MPLS – Which is better for your Business?
 
Analyzing and optimizing mpls technology at Reliance Jio
Analyzing and optimizing mpls technology at Reliance JioAnalyzing and optimizing mpls technology at Reliance Jio
Analyzing and optimizing mpls technology at Reliance Jio
 
CCNA Security 02- fundamentals of network security
CCNA Security 02-  fundamentals of network securityCCNA Security 02-  fundamentals of network security
CCNA Security 02- fundamentals of network security
 
What is zero trust model (ztm)
What is zero trust model (ztm)What is zero trust model (ztm)
What is zero trust model (ztm)
 
Storm-Control
Storm-ControlStorm-Control
Storm-Control
 
Palo alto networks next generation firewalls
Palo alto networks next generation firewallsPalo alto networks next generation firewalls
Palo alto networks next generation firewalls
 
Conditional Access System
Conditional Access SystemConditional Access System
Conditional Access System
 
Advanced OpenVPN Concepts - pfSense Hangout September 2014
Advanced OpenVPN Concepts - pfSense Hangout September 2014Advanced OpenVPN Concepts - pfSense Hangout September 2014
Advanced OpenVPN Concepts - pfSense Hangout September 2014
 
Fortinet security fabric
Fortinet security fabricFortinet security fabric
Fortinet security fabric
 

Viewers also liked

Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Alex Pinto
 

Viewers also liked (11)

Cyber after Snowden (OA Cyber Summit)
Cyber after Snowden (OA Cyber Summit)Cyber after Snowden (OA Cyber Summit)
Cyber after Snowden (OA Cyber Summit)
 
Workshop - Linux Memory Analysis with Volatility
Workshop - Linux Memory Analysis with VolatilityWorkshop - Linux Memory Analysis with Volatility
Workshop - Linux Memory Analysis with Volatility
 
Creating Your Own Threat Intel Through Hunting & Visualization
Creating Your Own Threat Intel Through Hunting & VisualizationCreating Your Own Threat Intel Through Hunting & Visualization
Creating Your Own Threat Intel Through Hunting & Visualization
 
Big Data Visualization
Big Data VisualizationBig Data Visualization
Big Data Visualization
 
Introduction to Python for Security Professionals
Introduction to Python for Security ProfessionalsIntroduction to Python for Security Professionals
Introduction to Python for Security Professionals
 
Workshop: Big Data Visualization for Security
Workshop: Big Data Visualization for SecurityWorkshop: Big Data Visualization for Security
Workshop: Big Data Visualization for Security
 
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
 
The Cyber Threat Intelligence Matrix
The Cyber Threat Intelligence MatrixThe Cyber Threat Intelligence Matrix
The Cyber Threat Intelligence Matrix
 
No Easy Breach DerbyCon 2016
No Easy Breach DerbyCon 2016No Easy Breach DerbyCon 2016
No Easy Breach DerbyCon 2016
 
Taking the Attacker Eviction Red Pill (v2.0)
Taking the Attacker Eviction Red Pill (v2.0)Taking the Attacker Eviction Red Pill (v2.0)
Taking the Attacker Eviction Red Pill (v2.0)
 
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
 

Similar to MOLOCH: Search for Full Packet Capture (OA Cyber Summit)

開放原始碼 Ch1.2 intro - oss - apahce foundry (ver 2.0)
開放原始碼 Ch1.2   intro - oss - apahce foundry (ver 2.0)開放原始碼 Ch1.2   intro - oss - apahce foundry (ver 2.0)
開放原始碼 Ch1.2 intro - oss - apahce foundry (ver 2.0)
My own sweet home!
 
AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101
Timothy Spann
 
Building high performance microservices in finance with Apache Thrift
Building high performance microservices in finance with Apache ThriftBuilding high performance microservices in finance with Apache Thrift
Building high performance microservices in finance with Apache Thrift
RX-M Enterprises LLC
 
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Timothy Spann
 

Similar to MOLOCH: Search for Full Packet Capture (OA Cyber Summit) (20)

Deploy and Access WebSphere Liberty and StrongLoop REST Endpoints on IBM Bluemix
Deploy and Access WebSphere Liberty and StrongLoop REST Endpoints on IBM BluemixDeploy and Access WebSphere Liberty and StrongLoop REST Endpoints on IBM Bluemix
Deploy and Access WebSphere Liberty and StrongLoop REST Endpoints on IBM Bluemix
 
Introduction to Kong API Gateway
Introduction to Kong API GatewayIntroduction to Kong API Gateway
Introduction to Kong API Gateway
 
Building Content-Rich Java Apps in the Cloud with the Alfresco API
Building Content-Rich Java Apps in the Cloud with the Alfresco APIBuilding Content-Rich Java Apps in the Cloud with the Alfresco API
Building Content-Rich Java Apps in the Cloud with the Alfresco API
 
Monitor OpenStack Environments from the bottom up and front to back
Monitor OpenStack Environments from the bottom up and front to backMonitor OpenStack Environments from the bottom up and front to back
Monitor OpenStack Environments from the bottom up and front to back
 
Apache Beam @ GCPUG.TW Flink.TW 20161006
Apache Beam @ GCPUG.TW Flink.TW 20161006Apache Beam @ GCPUG.TW Flink.TW 20161006
Apache Beam @ GCPUG.TW Flink.TW 20161006
 
Rest API with Swagger and NodeJS
Rest API with Swagger and NodeJSRest API with Swagger and NodeJS
Rest API with Swagger and NodeJS
 
RESTful web APIs (build, document, manage)
RESTful web APIs (build, document, manage)RESTful web APIs (build, document, manage)
RESTful web APIs (build, document, manage)
 
Go mico
Go micoGo mico
Go mico
 
Cisco CSIRT Case Study: Forensic Investigations with NetFlow
Cisco CSIRT Case Study: Forensic Investigations with NetFlowCisco CSIRT Case Study: Forensic Investigations with NetFlow
Cisco CSIRT Case Study: Forensic Investigations with NetFlow
 
開放原始碼 Ch1.2 intro - oss - apahce foundry (ver 2.0)
開放原始碼 Ch1.2   intro - oss - apahce foundry (ver 2.0)開放原始碼 Ch1.2   intro - oss - apahce foundry (ver 2.0)
開放原始碼 Ch1.2 intro - oss - apahce foundry (ver 2.0)
 
Overview of Apache Flink: Next-Gen Big Data Analytics Framework
Overview of Apache Flink: Next-Gen Big Data Analytics FrameworkOverview of Apache Flink: Next-Gen Big Data Analytics Framework
Overview of Apache Flink: Next-Gen Big Data Analytics Framework
 
Integrating Alfresco with Portals
Integrating Alfresco with PortalsIntegrating Alfresco with Portals
Integrating Alfresco with Portals
 
DBCC 2021 - FLiP Stack for Cloud Data Lakes
DBCC 2021 - FLiP Stack for Cloud Data LakesDBCC 2021 - FLiP Stack for Cloud Data Lakes
DBCC 2021 - FLiP Stack for Cloud Data Lakes
 
Apache Cordova 4.x
Apache Cordova 4.xApache Cordova 4.x
Apache Cordova 4.x
 
Using the Splunk Java SDK
Using the Splunk Java SDKUsing the Splunk Java SDK
Using the Splunk Java SDK
 
Five Fabulous Sinks for Your Kafka Data. #3 will surprise you! (Rachel Pedres...
Five Fabulous Sinks for Your Kafka Data. #3 will surprise you! (Rachel Pedres...Five Fabulous Sinks for Your Kafka Data. #3 will surprise you! (Rachel Pedres...
Five Fabulous Sinks for Your Kafka Data. #3 will surprise you! (Rachel Pedres...
 
AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101
 
Building high performance microservices in finance with Apache Thrift
Building high performance microservices in finance with Apache ThriftBuilding high performance microservices in finance with Apache Thrift
Building high performance microservices in finance with Apache Thrift
 
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
 
Deploying Confluent Platform for Production
Deploying Confluent Platform for ProductionDeploying Confluent Platform for Production
Deploying Confluent Platform for Production
 

More from Open Analytics

Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
Open Analytics
 
Using Real-Time Data to Drive Optimization & Personalization
Using Real-Time Data to Drive Optimization & PersonalizationUsing Real-Time Data to Drive Optimization & Personalization
Using Real-Time Data to Drive Optimization & Personalization
Open Analytics
 
Competing in the Digital Economy
Competing in the Digital EconomyCompeting in the Digital Economy
Competing in the Digital Economy
Open Analytics
 
Piwik: An Analytics Alternative (Chicago Summit)
Piwik: An Analytics Alternative (Chicago Summit)Piwik: An Analytics Alternative (Chicago Summit)
Piwik: An Analytics Alternative (Chicago Summit)
Open Analytics
 
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
Open Analytics
 
Crossing the Chasm (Ikanow - Chicago Summit)
Crossing the Chasm (Ikanow - Chicago Summit)Crossing the Chasm (Ikanow - Chicago Summit)
Crossing the Chasm (Ikanow - Chicago Summit)
Open Analytics
 
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
Open Analytics
 
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
Open Analytics
 
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
Open Analytics
 
From Insight to Impact (Chicago Summit - Keynote)
From Insight to Impact (Chicago Summit - Keynote)From Insight to Impact (Chicago Summit - Keynote)
From Insight to Impact (Chicago Summit - Keynote)
Open Analytics
 
Easybib Open Analytics NYC
Easybib Open Analytics NYCEasybib Open Analytics NYC
Easybib Open Analytics NYC
Open Analytics
 
MarkLogic - Open Analytics Meetup
MarkLogic - Open Analytics MeetupMarkLogic - Open Analytics Meetup
MarkLogic - Open Analytics Meetup
Open Analytics
 
The caprate presentation_july2013_open analytics dc meetup
The caprate presentation_july2013_open analytics dc meetupThe caprate presentation_july2013_open analytics dc meetup
The caprate presentation_july2013_open analytics dc meetup
Open Analytics
 
Verifeed open analytics_3min deck_071713_final
Verifeed open analytics_3min deck_071713_finalVerifeed open analytics_3min deck_071713_final
Verifeed open analytics_3min deck_071713_final
Open Analytics
 

More from Open Analytics (20)

Utilizing cyber intelligence to combat cyber adversaries (OA Cyber Summit)
Utilizing cyber intelligence to combat cyber adversaries (OA Cyber Summit)Utilizing cyber intelligence to combat cyber adversaries (OA Cyber Summit)
Utilizing cyber intelligence to combat cyber adversaries (OA Cyber Summit)
 
CDM….Where do you start? (OA Cyber Summit)
CDM….Where do you start? (OA Cyber Summit)CDM….Where do you start? (OA Cyber Summit)
CDM….Where do you start? (OA Cyber Summit)
 
An Immigrant’s view of Cyberspace (OA Cyber Summit)
An Immigrant’s view of Cyberspace (OA Cyber Summit)An Immigrant’s view of Cyberspace (OA Cyber Summit)
An Immigrant’s view of Cyberspace (OA Cyber Summit)
 
Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
 
Using Real-Time Data to Drive Optimization & Personalization
Using Real-Time Data to Drive Optimization & PersonalizationUsing Real-Time Data to Drive Optimization & Personalization
Using Real-Time Data to Drive Optimization & Personalization
 
M&A Trends in Telco Analytics
M&A Trends in Telco AnalyticsM&A Trends in Telco Analytics
M&A Trends in Telco Analytics
 
Competing in the Digital Economy
Competing in the Digital EconomyCompeting in the Digital Economy
Competing in the Digital Economy
 
Piwik: An Analytics Alternative (Chicago Summit)
Piwik: An Analytics Alternative (Chicago Summit)Piwik: An Analytics Alternative (Chicago Summit)
Piwik: An Analytics Alternative (Chicago Summit)
 
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
 
Crossing the Chasm (Ikanow - Chicago Summit)
Crossing the Chasm (Ikanow - Chicago Summit)Crossing the Chasm (Ikanow - Chicago Summit)
Crossing the Chasm (Ikanow - Chicago Summit)
 
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
 
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
 
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
 
From Insight to Impact (Chicago Summit - Keynote)
From Insight to Impact (Chicago Summit - Keynote)From Insight to Impact (Chicago Summit - Keynote)
From Insight to Impact (Chicago Summit - Keynote)
 
Easybib Open Analytics NYC
Easybib Open Analytics NYCEasybib Open Analytics NYC
Easybib Open Analytics NYC
 
MarkLogic - Open Analytics Meetup
MarkLogic - Open Analytics MeetupMarkLogic - Open Analytics Meetup
MarkLogic - Open Analytics Meetup
 
The caprate presentation_july2013_open analytics dc meetup
The caprate presentation_july2013_open analytics dc meetupThe caprate presentation_july2013_open analytics dc meetup
The caprate presentation_july2013_open analytics dc meetup
 
Verifeed open analytics_3min deck_071713_final
Verifeed open analytics_3min deck_071713_finalVerifeed open analytics_3min deck_071713_final
Verifeed open analytics_3min deck_071713_final
 
HDScores OA DC Pitch
HDScores OA DC PitchHDScores OA DC Pitch
HDScores OA DC Pitch
 
Oas schwartz 16
Oas schwartz 16Oas schwartz 16
Oas schwartz 16
 

Recently uploaded

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Recently uploaded (20)

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
A Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source MilvusA Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source Milvus
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 

MOLOCH: Search for Full Packet Capture (OA Cyber Summit)

  • 2. It is a Great Horned Owl Project Logo
  • 3. WHYTHE OWL? Owls are silent hunters that go after RATs. We think that’s pretty cool. 3
  • 5. WHAT IS MOLOCH? 5 Moloch is an open source, scalable IPv4 packet capture indexing and database system, built using open source technologies. • A simple web GUI is provided for browsing, searching, viewing and exporting PCAP data. • Web APIs are accessible if you wish to design your own GUI or directly grab PCAP with various command line tools for further analysis or processing. • Find it on AOL’s GitHub page: https://github.com/aol/moloch It’s like AOL Search for PCAP repositories!
  • 6. WHAT IS MOLOCH NOT? 6 NOT IDS: NO ALERTS NOT IPV6 (Today) NOT SLOW NOT CLOSED NOT EXPENSIVE
  • 7. WHYUSE MOLOCH? 7 Real-time capture of network traffic for forensic and investigative purposes • Combine the power of Moloch with other indicators (intelligence feeds, alerting from IDS/anti-virus) to empower your analysts to quickly and effectively review actions on the network to determine the validity/threat. • Review past network traffic for post compromise investigations. Static PCAP repository • Import large collections of PCAP that were created by malware. • Import collections of PCAP from Capture The Flag events. • Custom tagging of data at time of import.
  • 8. THE PIECES OF MOLOCH 8 CAPTURE • A C application that sniffs the network interface, parses the traffic, and creates the Session Profile Information (SPI data) and writes it to disk. DATABASE • Elasticsearch is used for storing and searching through the SPI data generated by the capture component. VIEWER • A web interface that allows for GUI and API access from remote hosts to browse or query SPI data and retrieve stored PCAP.
  • 9. THE PIECES OF MOLOCH: CAPTURE 9 Libnids based daemon written in C Can be used to sniff network interface for live capture Can be called from CLI to do manual imports Parses layers 3-7 to create SPI data • Spits them out to the Elasticsearch cluster. A lot like making owl pellets!
  • 10.
  • 11. THE PIECES OF MOLOCH: DATABASE 11 Elasticsearch (http://www.elasticsearch.org) • Powered by Apache Lucene (http://lucene.apache.org) • Requests over HTTP(s) • Results returned in JSON Nosql • Network traffic doesn’t fit the mold for relational DBs. Documented oriented • Great for lots and lots of network sessions. Automatic sharding across multiple hosts • At the time, we skipped SOLR because it couldn’t run distributed. Fast, scalable, all that goodness
  • 12. THE PIECES OF MOLOCH: VIEWER 12 Node.js based application • Event driven server side JavaScript platform. • Based on Chrome’s JavaScript runtime. • Comes with its own HTTP server and easy JSON for communication. Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP. GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, and command line tools is easy. • Easy automation to retrieve PCAP or sessions of interest.
  • 13. THE PIECES OF MOLOCH: VIEWER 13 Nodejs based application • Event driven server side JavaScript platform • Based on Chrome’s JavaScript runtime • Comes with its own HTTP server and easy JSON for communication Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy. • Easy automation to retrieve pcap or sessions of interest.
  • 14. THE PIECES OF MOLOCH: VIEWER 14 Nodejs based application • Event driven server side JavaScript platform • Based on Chrome’s JavaScript runtime • Comes with its own HTTP server and easy JSON for communication Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy. • Easy automation to retrieve pcap or sessions of interest.
  • 17. ARCHITECTUREOF MOLOCH: SCALE 17 Packets Captured Kilobytes Saved Sessions Saved Example: Moloch Capture Documents Disk Storage (MB) Example: Elasticsearch
  • 18. MOLOCH: SPI-DATATYPES SESSION PROFILE INFORMATION 18 IP • Source • Destination • Ports • Protocol HTTP • Method • Status Codes • Headers • Content Type DNS • IP Address • Hostnames
  • 19. MOLOCH: SPI-DATATYPES SESSION PROFILE INFORMATION 19 SSL/TLS • Cert Elements: • Common Name • Serial Number • Alt Names SSH • Client Name • Public Key • Port IRC • Channel Name • Hostname

Editor's Notes

  1. Example of typical cover slide.