The document discusses New Relic Flex, which allows users to collect custom metrics from various sources and send them to New Relic. It provides examples of how Flex can be used to monitor specific metrics from Linux systems, scripts, and JSON outputs. The document then outlines lessons for installing the New Relic Infra agent, configuring a YAML file to collect disk usage data, writing NRQL queries to analyze and visualize the data, and troubleshooting Flex.

1. ©2008–22 New Relic, Inc. All rights reserved
@LeonAdato
bit.ly/miami-signup
IT’S FREE!!
FOREVER
You also need it
for this workshop

2. © 2022 New Relic, Inc. All rights reserved.
Closing gaps in visibility
With New Relic Flex
Leon Adato
$date();

3. © 2022 New Relic, Inc. All rights reserved.
Leon Adato
• Principal DevRel Advocate @ New Relic
• Started in tech ~33 yrs ago
• Focus on monitoring for ~20 yrs
• Tivoli, BMC, OpenView, Nagios, SolarWinds
Perl scripts, batch files,
and New Relic (duh)
• Built monitoring for small, medium,
and ludicrous size orgs.
• @LeonAdato almost everywhere

4. © 2022 New Relic, Inc. All rights reserved.
This is an Oyster Talk™
● Installing the Infrastructure agent
● YAML and the Flex directory
● Flex and command line output
● Flex and script output
● Flex and JSON output
● Troubleshooting

5. ©2008–22 New Relic, Inc. All rights reserved
@LeonAdato

6. © 2022 New Relic, Inc. All rights reserved.
What isn’t (always) covered
● Standard commercial off-the-shelf (COTS) software
○ Exchange, Sharepoint, Blackberry, SAP, Storage
management, etc.
● Customized standard software
○ SAP, Wordpress, Drupal, etc)
● Custom data found in custom-built applications

7. © 2022 New Relic, Inc. All rights reserved.
Why you need* custom monitoring values
If what you need isn’t built into your monitoring / observability tool, you can:
● Place a feature request with the vendor, asking to add the metric.
○ And then wait. And pray. And maybe scream if you think it will help**
● Ask the community if/how they've solved the issue
○ IF there's an active community.
● Start shopping for yet-another-monitoring-tool
* probably
** spoiler: It won’t.

8. © 2022 New Relic, Inc. All rights reserved.
That sounds dumb
Or - and just hear me out - you could use your
monitoring / observability tool’s BUILT-IN ability to
collect custom metrics.
DUH!

9. © 2022 New Relic, Inc. All rights reserved.
What is Flex?
● Application-agnostic integration into New Relic
● Samples & more
https://github.com/newrelic/nri-flex

10. © 2022 New Relic, Inc. All rights reserved.
Agent
YAML
How does Flex ACTUALLY work?
Data!

11. © 2022 New Relic, Inc. All rights reserved.
YAML starter kit for the uninitiated
integrations:
- name: nri-flex
config:
name: NameyMcNameyFace
apis:
- event_type: <something here>
<lots more stuff here>

12. © 2022 New Relic, Inc. All rights reserved.
What can Flex do?
Collect metrics from a variety of outputs, and seamlessly send them
to New Relic for display on dashboards, use in alerts, etc. It can grab
data from
● A utility’s <stdout>.
● The output or return values of a script or program.
● JSON data generated by a local application.
● JSON data returned from a web page or external API call.
● Data scraped from a web page.

13. © 2022 New Relic, Inc. All rights reserved.
OK, but what is that in real life?
● LinuxAteMyRam.com - why sysadmins drink
● The IOT coffee pot - why security admins drink
● Everyone blaming the network - why network engineers drink
● A bunch of Oracle servers, and no way to get specific metrics.
● A different customer had >4k batch jobs per day, and needed
specific information on completion, size, return codes, etc.
● Or even finding the IIS space station and checking if your
current weather conditions will let you see it.

14. © 2022 New Relic, Inc. All rights reserved.
LESSON 1: Install the New Relic infra agent

15. © 2022 New Relic, Inc. All rights reserved.
LESSON 2: Your First YAML
integrations:
- name: nri-flex
config:
name: linuxFileSystemIntegration
apis:
- name: TestFlex
commands:
- run: 'df -PT -B1 -x tmpfs -x xfs -x vxfs -x btrfs -x ext -x ext2 -x ext3'
split: horizontal
split_by: s+
row_start: 1
set_header: [fs,fsType,capacityBytes,usedBytes,availableBytes,usedPerc,mountedOn]
perc_to_decimal: true

16. © 2022 New Relic, Inc. All rights reserved.
LESSON 3: The Necessary NRQL
FROM TestFlexSample SELECT average(availableBytes) TIMESERIES

17. © 2022 New Relic, Inc. All rights reserved.
LESSON 4: Flex with script output
integrations:
- name: nri-flex
timeout: 5m
interval: 10m
config:
name: linuxspeedtest
apis:
- name: pyspeedtest
commands:
- run: sudo python3 /flexlab/NR_checkspeed.py
split: horizontal
split_by: s+
set_header: [latency,download,upload]
timeout: 300000

18. © 2022 New Relic, Inc. All rights reserved.
LESSON 5: More NRQL
FROM pyspeedtestSample
SELECT average(download), average(upload)
TIMESERIES

19. © 2022 New Relic, Inc. All rights reserved.
LESSON 6: Flex with JSON output
integrations:
- name: nri-flex
timeout: 5m
interval: 10m
config:
name: linuxspeedtest
apis:
- name: speedtest
commands:
- run: speedtest --accept-license --accept-gcp -f json
timeout: 300000
remove_keys:
- timestamp

20. © 2022 New Relic, Inc. All rights reserved.
LESSON 7: Still more NRQL
FROM speedtestSample
select average(download.bandwidth)/125000,
average(upload.bandwidth)/125000
TIMESERIES

21. © 2022 New Relic, Inc. All rights reserved.
LESSON 8: Troubleshooting*
sudo /var/db/newrelic-infra/newrelic-integrations/bin/nri-flex
--verbose --pretty --config_file <path>/<your yaml>.yml
Edit /etc/newrelic-infra.yml
log:
file: '/var/log/newrelic-infra/newrelic-infra.log'
level: debug
forward: true
stdout: false
* and other swear words