How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf
•Download as PPTX, PDF•
0 likes•865 views
This document provides an overview of introducing network telemetry using streaming protocols like gNMI with Telegraf. It discusses gNMI as a streaming telemetry protocol, using Telegraf to collect metrics from network devices via gNMI and SNMP, and how to normalize and enrich the collected data through Telegraf processors before outputting to a time-series database. It also includes a demo of collecting interface counters from devices supporting gNMI and SNMP, and processing the data in Telegraf.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf
Similar to How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf (20)
More from InfluxData
More from InfluxData (20)
Recently uploaded
Recently uploaded (20)
How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf
- 1. www.networktocode.com Network to Code Confidential How to Introduce Telemetry Streaming (gNMI) in Your Network with SNMP with Telegraf November 2020
- 2. www.networktocode.com Network to Code Confidential Who is Network to Code? /company/network-to-code/ @networktocode • Founded in mid 2014 • Network Automation Solution Provider – Next-gen consulting and integration company – Focus on automation/programmability and reducing operational inefficiencies – Services only business model • Vendor Independent – Infrastructure (Cisco, Juniper, Arista, HP, Cumulus, F5 etc.) – Open Source Tools (Ansible, Salt, Python, StackStorm, Puppet, Terraform, Telegraf, InfluxDB, Prometheus, Grafana, ElasticSearch, Kibana) – IT Operations Software – ServiceNOW, Remedy, IBM www.networktocode.com
- 3. www.networktocode.com Network to Code Confidential About David & Josh David ● Dublin, IE ● Sr Network Automation Consultant, Network to Code ● Automating Networks since 2013 ● Telegraf user since 2019 Josh ● Minnesota, US ● Network Automation Engineer, Network to Code ● Automating Networks since 2015 ● Telegraf user since 2018 @vanderaaj@davidban77 @jvanderaa@davidban77
- 4. www.networktocode.com Network to Code Confidential Agenda One of the best times to be gathering telemetry data from network devices! • Network Streaming Telemetry Stack • gNMI • Gathering of Data from Network Devices with Telegraf • Enriching and Modifying metrics • Demo • Wrap Up & Tips and Tricks
- 5. www.networktocode.com Network to Code Confidential Telemetry Stack Collect Enrichment Store Dashboard Network Devices
- 6. www.networktocode.com Network to Code Confidential gNMI - Streaming Telemetry
- 7. www.networktocode.com Network to Code Confidential What is gNMI • Model driven configuration and retrieval of operational data over gRPC (Remote Procedure Calls) • Protocol Buffers • Subscription model
- 8. www.networktocode.com Network to Code Confidential gNMI Plugin Note • Previously Cisco gNMI Telemetry (cisco_telemetry_gnmi) • Renamed to just gNMI in Telegraf 1.15 • https://github.com/influxdata/telegraf/tree/rel ease-1.16/plugins/inputs/gnmi
- 9. www.networktocode.com Network to Code Confidential Telegraf for Network Telemetry
- 10. www.networktocode.com Network to Code Confidential What is Telegraf? • Telegraf is a plugin-driven server agent for collecting & reporting metrics written in Go • Pulls metrics from system running on, 3rd party APIs • Output plugins to send to various data stores
- 11. www.networktocode.com Network to Code Confidential Telegraf - gNMI 11 [[inputs.gnmi]] addresses = ["device1"] username = "<USERNAME>" password = "<PASSWORD>" [[inputs.gnmi.subscription]] origin = "openconfig-interfaces" path = "/interfaces/interface" subscription_mode = "sample" sample_interval = "60s" Collect Interfaces/interface counters Base config : Device + Credential
- 12. www.networktocode.com Network to Code Confidential Want to monitor all of this, streaming telemetry would be terrific
- 13. www.networktocode.com Network to Code Confidential Several devices in your environment do not yet support Streaming Telemetry!
- 14. www.networktocode.com Network to Code Confidential Hybrid Collection
- 15. www.networktocode.com Network to Code Confidential Telegraf - SNMP 15 [[inputs.snmp]] agents = ["device1"] version = 2 community = "<SNMP Community>" interval = "60s" timeout = "10s" retries = 3 [[inputs.snmp.field]] name = "hostname" oid = "RFC1213-MIB::sysName.0" is_tag = true [[inputs.snmp.table]] name = "interface" inherit_tags = [ "hostname" ] oid = "IF-MIB::ifXTable" [[inputs.snmp.table.field]] name = "name" oid = "IF-MIB::ifDescr" is_tag = true Base config : Device, Credential, interval Convert sysName.0 as tag hostname Collect IF-MIB::ifXTable Table Convert ifDescr as Tag name
- 16. www.networktocode.com Network to Code Confidential Challenge with Multiple Input Plugins 16 snmp gnmi interface_counters,agent_host=router1,ifName=Ethernet7 ifOutOctets=3814i,ifInOctets=469i 160519 interface,source=192.0.2.10,name=Ethernet7 in_octets=2176i 1605192 interface,source=192.0.2.10,name=Ethernet7 out_octets=316i 1605192 Measurement Tag Set Field Set TimestampInflux Line Protocol:
- 17. www.networktocode.com Network to Code Confidential Telegraf Pipeline
- 18. www.networktocode.com Network to Code Confidential Telegraf Pipeline snmp Output Plugin 40+ availableprometheus_client gnmi Execd regex Input Plugins 190+ available Processors Plugins 25+ available influxdb rename Collect Data Enrichment Output
- 19. www.networktocode.com Network to Code Confidential Data Enrichment
- 20. www.networktocode.com Network to Code Confidential Data Normalization with Telegraf - single device 20 snmp Output Plugin - Send to InfluxDB Normalize Data influxdb_v2 gnmi execd regex rename rename rename Collect Data + Add device level Tags Enrich Data (add interface role)
- 21. www.networktocode.com Network to Code Confidential # Processor to rename the snmp fields [[processors.rename]] [[processors.rename.replace]] field = "ifHCInOctets" dest = "in_octets" [[processors.rename.replace]] field = "ifHCOutOctets" dest = "out_octets" [[processors.rename.replace]] tag = "agent_host" dest = "source" Telegraf - Rename Data 21 Rename the field ifHCInOctets and ifHCOutOctets to match gNMI Rename the tag agent_host to match gNMI
- 22. www.networktocode.com Network to Code Confidential Telegraf - Add metadata 22 [[processors.regex]] [processors.regex.tagpass] host = device1 [[processors.regex.tags]] key = "name" pattern = "^ge-0/0/0$" replacement = "management" result_key = "interfacerole" [[processors.regex.tags]] key = "name" pattern = "^ge-0/0/1$" replacement = "data" result_key = "interfacerole" Apply these rules only for measurement with a tag host=device1 If a tag name=ge-0/0/0 is present, add a tag interfacerole=management If a tag name=ge-0/0/1 is present, add a tag interfacerole=data
- 23. www.networktocode.com Network to Code Confidential Demo Environment
- 24. www.networktocode.com Network to Code Confidential Demo Environment
- 25. www.networktocode.com Network to Code Confidential Recap • Collect Data from devices • Normalize the data • Enrich the data • Output the data • Dashboard of the data
- 26. www.networktocode.com Network to Code Confidential What’s Next & Tips and Tricks
- 27. www.networktocode.com Network to Code Confidential What we are trying to do next? • Flux language • InfluxDB 2.0 • InfluxDB IOx • Additional plugins - Input and Processor • Starlark processor
- 28. www.networktocode.com Network to Code Confidential Telegraf Tips & Tricks • Automate configuration with templating - Ansible is a good choice • Run in Docker • Install SNMP MIB’s Locally • Use files output for troubleshooting • SNMP/gNMI won’t work? Look at writing your own executable with the execd plugins • Input Internal Plugin (about the Telegraf Plugin) • Consider running on device natively • Use the reachability plugins (Network Response, DNS) • Look at the Plugin Documentation (search “Telegraf Plugins”)
- 29. www.networktocode.com Network to Code Confidential Monitor Your Network With gNMI, SNMP, and Grafana http://blog.networktocode.com/post/monitor_your_network_with_gnmi_snmp_and_grafana/ Network Telemetry for SNMP Devices http://blog.networktocode.com/post/network_telemetry_for_snmp_devices/ Monitoring Websites with Telegraf and Prometheus https://blog.networktocode.com/post/monitoring_websites_with_telegraf_and_prometheus/ Network to Code Blog Content
- 30. www.networktocode.com Network to Code Confidential Thank You! /company/network-to-code/ @networktocode www.networktocode.com slack.networktocode.com
- 31. www.networktocode.com Network to Code Confidential Appendix Items
- 32. www.networktocode.com Network to Code Confidential [global_tags] device_role = "spine" [agent] hostname = "telegraf-r1" Telegraf-R1: Base Input Configuration [[inputs.snmp]] agents = ["r1"] version = 2 community = "${SNMP_COMMUNITY}" interval = "60s" timeout = "10s" retries = 3 [[inputs.snmp.table]] name = "interface" oid = "IF-MIB::ifXTable" [[inputs.snmp.table.field]] name = "name" oid = "IF-MIB::ifDescr" is_tag = true Cont.
- 33. www.networktocode.com Network to Code Confidential Telegraf -R1 Data Normalization and Enrichment [[processors.rename]] [[processors.rename.replace]] field = "ifHCInOctets" dest = "in_octets" [[processors.rename.replace]] field = "ifHCOutOctets" dest = "out_octets" [[processors.rename.replace]] tag = "agent_host" dest = "device" [[processors.regex]] namepass = ["interface"] [[processors.regex.tags]] key = "name" pattern = "^GigaS+0$" replacement = "management" result_key = "intf_role" [[processors.regex.tags]] key = "name" pattern = "^GigaS+0/1$" replacement = "spine" result_key = "intf_role" [[processors.regex.tags]] key = "name" pattern = "^GigaS+0/2$" replacement = "backbone" result_key = "intf_role" Cont.
- 34. www.networktocode.com Network to Code Confidential Telegraf -R1 Output InfluxDB and STDOUT [[outputs.influxdb]] urls = ["http://influxdb:8086"] [[outputs.file]] files = ["stdout"] data_format = "influx"
- 35. www.networktocode.com Network to Code Confidential [global_tags] device_role = "spine" [agent] hostname = "telegraf-r2" Telegraf-R2: Base Input Configuration Cont. [[inputs.gnmi]] addresses = ["r2:6030"] username = "${GNMI_USER}" password = "${GNMI_PASSWORD}" redial = "10s" [[inputs.gnmi.subscription]] name = "interface" origin = "openconfig-interfaces" path = "/interfaces/interface/state/counters" subscription_mode = "sample" sample_interval = "5s"
- 36. www.networktocode.com Network to Code Confidential Telegraf -R2 Data Normalization and Enrichment Cont. [[processors.rename]] [[processors.rename.replace]] tag = "source" dest = "device" [[processors.regex]] namepass = ["interface"] [[processors.regex.tags]] key = "name" pattern = "^Managementd+$" replacement = "management" result_key = "intf_role" [[processors.regex.tags]] key = "name" pattern = "^Ethernet1$" replacement = "spine" result_key = "intf_role" [[processors.regex.tags]] key = "name" pattern = "^Ethernet2$" replacement = "backbone" result_key = "intf_role"
- 37. www.networktocode.com Network to Code Confidential Telegraf -R2 Output InfluxDB and STDOUT [[outputs.influxdb]] urls = ["http://influxdb:8086"] [[outputs.file]] files = ["stdout"] data_format = "influx"