SlideShare a Scribd company logo
1 of 18
Download to read offline
LogStash: Concept Run-Through
What is LogStash?
What is LogStash?
Definition
Logstash is an open
source, server-side data
processing pipeline that
ingests data from a
multitude of sources
simultaneously,
transforms it, and then
sends it to a specified
output.
What is LogStash?
Definition
Jordan Sissel created the first version of LogStash in 2013, when he frequently found himself in the thick of
aggregating and managing log files.
Logstash continued to grow with the other components of the ELK stack and is now part of a comprehensive
platform for log data and analysis, providing companies with valuable insights into how their customers
interact with support system, e-commerce sites and so on.
What is LogStash?
Importance of LogStash
Open source data collection engine
Centralize data processing of all types
Normalized varying schema for
business critical data
Support for multiple and custom
formats
Extensibility via plugins
History of LogStash
What is LogStash?
What Can LogStash Do?
Core Features
What Can LogStash Do?
Data ingestion
workhorse
Events enrichment
and transformation
Extensible plugin
ecosystem
It is highly available,
scalable and elastic in
nature.
Pluggable pipeline
architecture
Horizontally
scalable data
processing pipeline
Strong
Elasticsearch and
Kibana synergy
Handles data of all
shapes and sizes
LogStash Versatility
What Can LogStash Do?
Analysis
Archiving
Monitoring
Alerting
Search
What Constitutes LogStash?
Key Components and Terminology
What Constitutes LogStash?
INPUTS
FILTERS
OUTPUTS
Specify the source of events
LogStash can handle variety of
sources
Most common ones are:
Logs
Network
Web
Data stores and streams
Sensors and IoT
Key Components and Terminology
What Constitutes LogStash?
INPUTS
Key Components and Terminology
What Constitutes LogStash?
FILTERS
Responsible for parsing the
incoming events
May enrich the events
Most common ones are:
grok
mutate
drop
Key Components and Terminology
What Constitutes LogStash?
OUTPUTS
Final stage of the pipeline
Sends the enriched output to a
specified destination
LogStash can handle variety
of destinations
ElasticSearch
AWS S3 buckets
Files
How Does LogStash Work?
Files
AWS S3
Elastic Search
Grok
GeoIP
Drop
Anonymize
Inputs Filters Outputs
LogStash Pipeline
How Does LogStash Work?
LogStash: Concept Run-Through

More Related Content

What's hot

How to visualize Cosmos DB graph data
How to visualize Cosmos DB graph dataHow to visualize Cosmos DB graph data
How to visualize Cosmos DB graph dataLinkurious
 
Building Data Lakes with Apache Airflow
Building Data Lakes with Apache AirflowBuilding Data Lakes with Apache Airflow
Building Data Lakes with Apache AirflowGary Stafford
 
The Yellowbrick Impact for MicroStrategy
The Yellowbrick Impact for MicroStrategyThe Yellowbrick Impact for MicroStrategy
The Yellowbrick Impact for MicroStrategyYellowbrick Data
 
PSSUG Nov 2012: Big Data with SQL Server
PSSUG Nov 2012: Big Data with SQL ServerPSSUG Nov 2012: Big Data with SQL Server
PSSUG Nov 2012: Big Data with SQL ServerMark Kromer
 
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...Shawn Jones
 
Azure data catalog your data your way eugene polonichko dataconf 21 04 18
Azure data catalog your data your way eugene polonichko dataconf 21 04 18Azure data catalog your data your way eugene polonichko dataconf 21 04 18
Azure data catalog your data your way eugene polonichko dataconf 21 04 18Olga Zinkevych
 
Data & AI Platform Concepts
Data & AI Platform ConceptsData & AI Platform Concepts
Data & AI Platform ConceptsAnkit Rathi
 
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...Edureka!
 
Unlocking Geospatial Analytics Use Cases with CARTO and Databricks
Unlocking Geospatial Analytics Use Cases with CARTO and DatabricksUnlocking Geospatial Analytics Use Cases with CARTO and Databricks
Unlocking Geospatial Analytics Use Cases with CARTO and DatabricksDatabricks
 
Collecting and Making Sense of Diverse Data at WayUp
Collecting and Making Sense of Diverse Data at WayUpCollecting and Making Sense of Diverse Data at WayUp
Collecting and Making Sense of Diverse Data at WayUpHarlan Harris
 
Cortana Analytics Workshop: Azure Data Catalog
Cortana Analytics Workshop: Azure Data CatalogCortana Analytics Workshop: Azure Data Catalog
Cortana Analytics Workshop: Azure Data CatalogMSAdvAnalytics
 
ODA Use-Case: XaitPorter Appliance
ODA Use-Case: XaitPorter ApplianceODA Use-Case: XaitPorter Appliance
ODA Use-Case: XaitPorter ApplianceRoy Olsen
 
Load data from xml to Snowflake in minutes
Load data from xml to Snowflake in minutesLoad data from xml to Snowflake in minutes
Load data from xml to Snowflake in minutessyed_javed
 
Bridging to a hybrid cloud data services architecture
Bridging to a hybrid cloud data services architectureBridging to a hybrid cloud data services architecture
Bridging to a hybrid cloud data services architectureIBM Analytics
 
Leveraging cloud database connectors to automate analytics in alteryx
Leveraging cloud database connectors to automate analytics in alteryxLeveraging cloud database connectors to automate analytics in alteryx
Leveraging cloud database connectors to automate analytics in alteryxGrazitti Interactive
 
Data Mesh @ Yelp - 2019
Data Mesh @ Yelp - 2019Data Mesh @ Yelp - 2019
Data Mesh @ Yelp - 2019Steven Moy
 
New York Elastic{ON} Tour Opening Keynote
New York Elastic{ON} Tour Opening KeynoteNew York Elastic{ON} Tour Opening Keynote
New York Elastic{ON} Tour Opening KeynoteElasticsearch
 
Business Insight
Business InsightBusiness Insight
Business InsightMicrosoft
 
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...Ashnikbiz
 

What's hot (20)

How to visualize Cosmos DB graph data
How to visualize Cosmos DB graph dataHow to visualize Cosmos DB graph data
How to visualize Cosmos DB graph data
 
Building Data Lakes with Apache Airflow
Building Data Lakes with Apache AirflowBuilding Data Lakes with Apache Airflow
Building Data Lakes with Apache Airflow
 
The Yellowbrick Impact for MicroStrategy
The Yellowbrick Impact for MicroStrategyThe Yellowbrick Impact for MicroStrategy
The Yellowbrick Impact for MicroStrategy
 
PSSUG Nov 2012: Big Data with SQL Server
PSSUG Nov 2012: Big Data with SQL ServerPSSUG Nov 2012: Big Data with SQL Server
PSSUG Nov 2012: Big Data with SQL Server
 
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
 
Azure data catalog your data your way eugene polonichko dataconf 21 04 18
Azure data catalog your data your way eugene polonichko dataconf 21 04 18Azure data catalog your data your way eugene polonichko dataconf 21 04 18
Azure data catalog your data your way eugene polonichko dataconf 21 04 18
 
Data & AI Platform Concepts
Data & AI Platform ConceptsData & AI Platform Concepts
Data & AI Platform Concepts
 
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
 
Unlocking Geospatial Analytics Use Cases with CARTO and Databricks
Unlocking Geospatial Analytics Use Cases with CARTO and DatabricksUnlocking Geospatial Analytics Use Cases with CARTO and Databricks
Unlocking Geospatial Analytics Use Cases with CARTO and Databricks
 
Collecting and Making Sense of Diverse Data at WayUp
Collecting and Making Sense of Diverse Data at WayUpCollecting and Making Sense of Diverse Data at WayUp
Collecting and Making Sense of Diverse Data at WayUp
 
Cortana Analytics Workshop: Azure Data Catalog
Cortana Analytics Workshop: Azure Data CatalogCortana Analytics Workshop: Azure Data Catalog
Cortana Analytics Workshop: Azure Data Catalog
 
ODA Use-Case: XaitPorter Appliance
ODA Use-Case: XaitPorter ApplianceODA Use-Case: XaitPorter Appliance
ODA Use-Case: XaitPorter Appliance
 
Load data from xml to Snowflake in minutes
Load data from xml to Snowflake in minutesLoad data from xml to Snowflake in minutes
Load data from xml to Snowflake in minutes
 
Bridging to a hybrid cloud data services architecture
Bridging to a hybrid cloud data services architectureBridging to a hybrid cloud data services architecture
Bridging to a hybrid cloud data services architecture
 
Leveraging cloud database connectors to automate analytics in alteryx
Leveraging cloud database connectors to automate analytics in alteryxLeveraging cloud database connectors to automate analytics in alteryx
Leveraging cloud database connectors to automate analytics in alteryx
 
Data Mesh @ Yelp - 2019
Data Mesh @ Yelp - 2019Data Mesh @ Yelp - 2019
Data Mesh @ Yelp - 2019
 
New York Elastic{ON} Tour Opening Keynote
New York Elastic{ON} Tour Opening KeynoteNew York Elastic{ON} Tour Opening Keynote
New York Elastic{ON} Tour Opening Keynote
 
Business Insight
Business InsightBusiness Insight
Business Insight
 
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...
 
StreamSet ETL tool
StreamSet  ETL toolStreamSet  ETL tool
StreamSet ETL tool
 

Similar to LogStash: Concept Run-Through

Log management with_logstash_and_elastic_search
Log management with_logstash_and_elastic_searchLog management with_logstash_and_elastic_search
Log management with_logstash_and_elastic_searchRishav Rohit
 
SAP BODS -quick guide.docx
SAP BODS -quick guide.docxSAP BODS -quick guide.docx
SAP BODS -quick guide.docxKen T
 
Centralized logging
Centralized loggingCentralized logging
Centralized loggingblessYahu
 
Centralization of all log (application, docker, security, ...)
Centralization of all log (application, docker, security, ...)Centralization of all log (application, docker, security, ...)
Centralization of all log (application, docker, security, ...)Thierry Gayet
 
Comprehensive Guide for Microsoft Fabric to Master Data Analytics
Comprehensive Guide for Microsoft Fabric to Master Data AnalyticsComprehensive Guide for Microsoft Fabric to Master Data Analytics
Comprehensive Guide for Microsoft Fabric to Master Data AnalyticsSparity1
 
Webinar: Open Source Business Intelligence Intro
Webinar: Open Source Business Intelligence IntroWebinar: Open Source Business Intelligence Intro
Webinar: Open Source Business Intelligence IntroSpagoWorld
 
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016StampedeCon
 
Enabling SQL Access to Data Lakes
Enabling SQL Access to Data LakesEnabling SQL Access to Data Lakes
Enabling SQL Access to Data LakesVasu S
 
Business Intelligence for users - Sharperlight
Business Intelligence for users - SharperlightBusiness Intelligence for users - Sharperlight
Business Intelligence for users - SharperlightMichell8240
 
the Data World Distilled
the Data World Distilledthe Data World Distilled
the Data World DistilledRTTS
 
LinkedInSaxoBankDataWorkbench
LinkedInSaxoBankDataWorkbenchLinkedInSaxoBankDataWorkbench
LinkedInSaxoBankDataWorkbenchSheetal Pratik
 
OLAP & DATA WAREHOUSE
OLAP & DATA WAREHOUSEOLAP & DATA WAREHOUSE
OLAP & DATA WAREHOUSEZalpa Rathod
 
Date warehousing concepts
Date warehousing conceptsDate warehousing concepts
Date warehousing conceptspcherukumalla
 
Data junction tool
Data junction toolData junction tool
Data junction toolSara shall
 
OLAP & Data Warehouse
OLAP & Data WarehouseOLAP & Data Warehouse
OLAP & Data WarehouseZalpa Rathod
 
Datawarehousing & DSS
Datawarehousing & DSSDatawarehousing & DSS
Datawarehousing & DSSDeepali Raut
 

Similar to LogStash: Concept Run-Through (20)

Log management with_logstash_and_elastic_search
Log management with_logstash_and_elastic_searchLog management with_logstash_and_elastic_search
Log management with_logstash_and_elastic_search
 
SAP BODS -quick guide.docx
SAP BODS -quick guide.docxSAP BODS -quick guide.docx
SAP BODS -quick guide.docx
 
Centralized logging
Centralized loggingCentralized logging
Centralized logging
 
FluentD vs. Logstash
FluentD vs. LogstashFluentD vs. Logstash
FluentD vs. Logstash
 
ExecutiveWhitePaper
ExecutiveWhitePaperExecutiveWhitePaper
ExecutiveWhitePaper
 
Centralization of all log (application, docker, security, ...)
Centralization of all log (application, docker, security, ...)Centralization of all log (application, docker, security, ...)
Centralization of all log (application, docker, security, ...)
 
Comprehensive Guide for Microsoft Fabric to Master Data Analytics
Comprehensive Guide for Microsoft Fabric to Master Data AnalyticsComprehensive Guide for Microsoft Fabric to Master Data Analytics
Comprehensive Guide for Microsoft Fabric to Master Data Analytics
 
Webinar: Open Source Business Intelligence Intro
Webinar: Open Source Business Intelligence IntroWebinar: Open Source Business Intelligence Intro
Webinar: Open Source Business Intelligence Intro
 
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
 
Enabling SQL Access to Data Lakes
Enabling SQL Access to Data LakesEnabling SQL Access to Data Lakes
Enabling SQL Access to Data Lakes
 
Business Intelligence for users - Sharperlight
Business Intelligence for users - SharperlightBusiness Intelligence for users - Sharperlight
Business Intelligence for users - Sharperlight
 
the Data World Distilled
the Data World Distilledthe Data World Distilled
the Data World Distilled
 
LinkedInSaxoBankDataWorkbench
LinkedInSaxoBankDataWorkbenchLinkedInSaxoBankDataWorkbench
LinkedInSaxoBankDataWorkbench
 
OLAP & DATA WAREHOUSE
OLAP & DATA WAREHOUSEOLAP & DATA WAREHOUSE
OLAP & DATA WAREHOUSE
 
Date warehousing concepts
Date warehousing conceptsDate warehousing concepts
Date warehousing concepts
 
Data junction tool
Data junction toolData junction tool
Data junction tool
 
CTP Data Warehouse
CTP Data WarehouseCTP Data Warehouse
CTP Data Warehouse
 
OLAP & Data Warehouse
OLAP & Data WarehouseOLAP & Data Warehouse
OLAP & Data Warehouse
 
Archonnex at ICPSR
Archonnex at ICPSRArchonnex at ICPSR
Archonnex at ICPSR
 
Datawarehousing & DSS
Datawarehousing & DSSDatawarehousing & DSS
Datawarehousing & DSS
 

More from Manuj Aggarwal

IaaS Cloud Computing With OpenStack - Master Class (Handout)
IaaS Cloud Computing With OpenStack - Master Class (Handout)IaaS Cloud Computing With OpenStack - Master Class (Handout)
IaaS Cloud Computing With OpenStack - Master Class (Handout)Manuj Aggarwal
 
Manage Azure Cloud with ARM Templates
Manage Azure Cloud with ARM TemplatesManage Azure Cloud with ARM Templates
Manage Azure Cloud with ARM TemplatesManuj Aggarwal
 

More from Manuj Aggarwal (6)

IaaS Cloud Computing With OpenStack - Master Class (Handout)
IaaS Cloud Computing With OpenStack - Master Class (Handout)IaaS Cloud Computing With OpenStack - Master Class (Handout)
IaaS Cloud Computing With OpenStack - Master Class (Handout)
 
Features of AWS - IAM
Features of AWS - IAMFeatures of AWS - IAM
Features of AWS - IAM
 
Manage Azure Cloud with ARM Templates
Manage Azure Cloud with ARM TemplatesManage Azure Cloud with ARM Templates
Manage Azure Cloud with ARM Templates
 
What is Open VPN
What is Open VPN What is Open VPN
What is Open VPN
 
Why Use PfSense ?
Why Use PfSense ?Why Use PfSense ?
Why Use PfSense ?
 
LogStash in action
LogStash in actionLogStash in action
LogStash in action
 

Recently uploaded

Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024StefanoLambiase
 
What are the key points to focus on before starting to learn ETL Development....
What are the key points to focus on before starting to learn ETL Development....What are the key points to focus on before starting to learn ETL Development....
What are the key points to focus on before starting to learn ETL Development....kzayra69
 
PREDICTING RIVER WATER QUALITY ppt presentation
PREDICTING  RIVER  WATER QUALITY  ppt presentationPREDICTING  RIVER  WATER QUALITY  ppt presentation
PREDICTING RIVER WATER QUALITY ppt presentationvaddepallysandeep122
 
Xen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfXen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfStefano Stabellini
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureDinusha Kumarasiri
 
Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Velvetech LLC
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作qr0udbr0
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationBradBedford3
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Angel Borroy López
 
React Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaReact Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaHanief Utama
 
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...confluent
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfAlina Yurenko
 
Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)Hr365.us smith
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Matt Ray
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfMarharyta Nedzelska
 
Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Mater
 
Buds n Tech IT Solutions: Top-Notch Web Services in Noida
Buds n Tech IT Solutions: Top-Notch Web Services in NoidaBuds n Tech IT Solutions: Top-Notch Web Services in Noida
Buds n Tech IT Solutions: Top-Notch Web Services in Noidabntitsolutionsrishis
 
Introduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdfIntroduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdfFerryKemperman
 

Recently uploaded (20)

Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
 
What are the key points to focus on before starting to learn ETL Development....
What are the key points to focus on before starting to learn ETL Development....What are the key points to focus on before starting to learn ETL Development....
What are the key points to focus on before starting to learn ETL Development....
 
PREDICTING RIVER WATER QUALITY ppt presentation
PREDICTING  RIVER  WATER QUALITY  ppt presentationPREDICTING  RIVER  WATER QUALITY  ppt presentation
PREDICTING RIVER WATER QUALITY ppt presentation
 
Xen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfXen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdf
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with Azure
 
Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...
 
Advantages of Odoo ERP 17 for Your Business
Advantages of Odoo ERP 17 for Your BusinessAdvantages of Odoo ERP 17 for Your Business
Advantages of Odoo ERP 17 for Your Business
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion Application
 
2.pdf Ejercicios de programación competitiva
2.pdf Ejercicios de programación competitiva2.pdf Ejercicios de programación competitiva
2.pdf Ejercicios de programación competitiva
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
 
React Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaReact Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief Utama
 
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
 
Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdf
 
Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)
 
Buds n Tech IT Solutions: Top-Notch Web Services in Noida
Buds n Tech IT Solutions: Top-Notch Web Services in NoidaBuds n Tech IT Solutions: Top-Notch Web Services in Noida
Buds n Tech IT Solutions: Top-Notch Web Services in Noida
 
Introduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdfIntroduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdf
 

LogStash: Concept Run-Through

  • 1.
  • 4. What is LogStash? Definition Logstash is an open source, server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to a specified output.
  • 5. What is LogStash? Definition Jordan Sissel created the first version of LogStash in 2013, when he frequently found himself in the thick of aggregating and managing log files. Logstash continued to grow with the other components of the ELK stack and is now part of a comprehensive platform for log data and analysis, providing companies with valuable insights into how their customers interact with support system, e-commerce sites and so on.
  • 6. What is LogStash? Importance of LogStash Open source data collection engine Centralize data processing of all types Normalized varying schema for business critical data Support for multiple and custom formats Extensibility via plugins
  • 9. Core Features What Can LogStash Do? Data ingestion workhorse Events enrichment and transformation Extensible plugin ecosystem It is highly available, scalable and elastic in nature. Pluggable pipeline architecture Horizontally scalable data processing pipeline Strong Elasticsearch and Kibana synergy Handles data of all shapes and sizes
  • 10. LogStash Versatility What Can LogStash Do? Analysis Archiving Monitoring Alerting Search
  • 12. Key Components and Terminology What Constitutes LogStash? INPUTS FILTERS OUTPUTS
  • 13. Specify the source of events LogStash can handle variety of sources Most common ones are: Logs Network Web Data stores and streams Sensors and IoT Key Components and Terminology What Constitutes LogStash? INPUTS
  • 14. Key Components and Terminology What Constitutes LogStash? FILTERS Responsible for parsing the incoming events May enrich the events Most common ones are: grok mutate drop
  • 15. Key Components and Terminology What Constitutes LogStash? OUTPUTS Final stage of the pipeline Sends the enriched output to a specified destination LogStash can handle variety of destinations ElasticSearch AWS S3 buckets Files
  • 17. Files AWS S3 Elastic Search Grok GeoIP Drop Anonymize Inputs Filters Outputs LogStash Pipeline How Does LogStash Work?