### Part 1 (20 min) - Avoiding Bad Stats
Bad and even fake statistics are commonly found in mainstream media, but did you know that they're even more common in InfoSec? Cybersecurity vendors and media can often be found using statistics that are poorly interpreted, come from bad data, or are even entirely fabricated! I'll cover some high-profile examples of bad and fake stats. Then, I'll walk through some strategies and tools you can use to spot and debunk bad stats yourself! This skill isn't just useful for your InfoSec day job, either - these same approaches will work for bad stats you come across in any field.
### Part 2 (20 min) - The Benefits of Playing Live Trivia with Friends
Now that you understand how to spot and validate bad stats, I'll talk about how doing weekly live trivia with friends can improve your self-awareness, confidence, and humility. We'll talk about how trivia can help you spot and avoid your own cognitive biases and some fallacies that often lead us down dangerous paths.
On knowing what normal looks like, this can be tricky, because it requires some expertise in a field, and familiarity with general stats in your field.For example, to know a stat on the number of unfilled jobs in cybersecurity seems off, you need a general understanding of how large the cybersecurity field is, and some awareness of how much hiring is going on in the public and private space, in your country and globally.You don't need to know all that to investigate a stat or claim though! Most of the data you need is available on the Internet!The Gell-Mann Amnesia Effect was something that Michael Crichton came up with - yeah, the guy that created Jurassic Park and ERHe used this term to describe the phenomenon of experts believing news articles written on topics outside of their fields of expertise, yet acknowledging that articles written in the same publication within their fields of expertise are error-ridden and full of misunderstandingThe lesson here, is that it's important to have a healthy dose of skepticism while consuming media, reports, papers, and research.
On knowing what normal looks like, this can be tricky, because it requires some expertise in a field, and familiarity with general stats in your field.For example, to know a stat on the number of unfilled jobs in cybersecurity seems off, you need a general understanding of how large the cybersecurity field is, and some awareness of how much hiring is going on in the public and private space, in your country and globally.You don't need to know all that to investigate a stat or claim though! Most of the data you need is available on the Internet!The Gell-Mann Amnesia Effect was something that Michael Crichton came up with - yeah, the guy that created Jurassic Park and ERHe used this term to describe the phenomenon of experts believing news articles written on topics outside of their fields of expertise, yet acknowledging that articles written in the same publication within their fields of expertise are error-ridden and full of misunderstandingThe lesson here, is that it's important to have a healthy dose of skepticism while consuming media, reports, papers, and research.
On knowing what normal looks like, this can be tricky, because it requires some expertise in a field, and familiarity with general stats in your field.For example, to know a stat on the number of unfilled jobs in cybersecurity seems off, you need a general understanding of how large the cybersecurity field is, and some awareness of how much hiring is going on in the public and private space, in your country and globally.You don't need to know all that to investigate a stat or claim though! Most of the data you need is available on the Internet!The Gell-Mann Amnesia Effect was something that Michael Crichton came up with - yeah, the guy that created Jurassic Park and ERHe used this term to describe the phenomenon of experts believing news articles written on topics outside of their fields of expertise, yet acknowledging that articles written in the same publication within their fields of expertise are error-ridden and full of misunderstandingThe lesson here, is that it's important to have a healthy dose of skepticism while consuming media, reports, papers, and research.
On knowing what normal looks like, this can be tricky, because it requires some expertise in a field, and familiarity with general stats in your field.For example, to know a stat on the number of unfilled jobs in cybersecurity seems off, you need a general understanding of how large the cybersecurity field is, and some awareness of how much hiring is going on in the public and private space, in your country and globally.You don't need to know all that to investigate a stat or claim though! Most of the data you need is available on the Internet!The Gell-Mann Amnesia Effect was something that Michael Crichton came up with - yeah, the guy that created Jurassic Park and ERHe used this term to describe the phenomenon of experts believing news articles written on topics outside of their fields of expertise, yet acknowledging that articles written in the same publication within their fields of expertise are error-ridden and full of misunderstandingThe lesson here, is that it's important to have a healthy dose of skepticism while consuming media, reports, papers, and research.
Let's take a closer look at one of these stats.
First off, TRUST YOUR GUT. If it sounds like BS, it's worth investigating.
In 2017, I set out to document every company that had ever been destroyed by a breach.
How many did I find?
Only 23 over a 20 year period.
All small businesses; maybe 4 had over 100 employees, but all less than 500
Is it possible I missed some, or some didn't get reported? Sure!
Is it possible I'm missing 150 PER YEAR? Probably not.
This answers a more broad and general assumption in our industry: the assumption that security incidents MUST be ending companies
Ramon Ray: founder and owner of smallbiztechnology.com and Smart Hustle.
He was still using the stat as recently as late 2021, five years after NextGov interviewed him for their article debunking this claim.
And that's at the core of what we're talking about here:
there are a lot of folks out there that won't let the truth get in the way of a good sound byte
Let's take a closer look at one of these stats.
First off, TRUST YOUR GUT. If it sounds like BS, it's worth investigating.
In 2017, I set out to document every company that had ever been destroyed by a breach.
How many did I find?
Only 23 over a 20 year period.
All small businesses; maybe 4 had over 100 employees, but all less than 500
Is it possible I missed some, or some didn't get reported? Sure!
Is it possible I'm missing 150 PER YEAR? Probably not.
This answers a more broad and general assumption in our industry: the assumption that security incidents MUST be ending companies
Ramon Ray: founder and owner of smallbiztechnology.com and Smart Hustle.
He was still using the stat as recently as late 2021, five years after NextGov interviewed him for their article debunking this claim.
And that's at the core of what we're talking about here:
there are a lot of folks out there that won't let the truth get in the way of a good sound byte
Let's take a closer look at one of these stats.
First off, TRUST YOUR GUT. If it sounds like BS, it's worth investigating.
In 2017, I set out to document every company that had ever been destroyed by a breach.
How many did I find?
Only 23 over a 20 year period.
All small businesses; maybe 4 had over 100 employees, but all less than 500
Is it possible I missed some, or some didn't get reported? Sure!
Is it possible I'm missing 150 PER YEAR? Probably not.
This answers a more broad and general assumption in our industry: the assumption that security incidents MUST be ending companies
Ramon Ray: founder and owner of smallbiztechnology.com and Smart Hustle.
He was still using the stat as recently as late 2021, five years after NextGov interviewed him for their article debunking this claim.
And that's at the core of what we're talking about here:
there are a lot of folks out there that won't let the truth get in the way of a good sound byte
Let's take a closer look at one of these stats.
First off, TRUST YOUR GUT. If it sounds like BS, it's worth investigating.
In 2017, I set out to document every company that had ever been destroyed by a breach.
How many did I find?
Only 23 over a 20 year period.
All small businesses; maybe 4 had over 100 employees, but all less than 500
Is it possible I missed some, or some didn't get reported? Sure!
Is it possible I'm missing 150 PER YEAR? Probably not.
This answers a more broad and general assumption in our industry: the assumption that security incidents MUST be ending companies
Ramon Ray: founder and owner of smallbiztechnology.com and Smart Hustle.
He was still using the stat as recently as late 2021, five years after NextGov interviewed him for their article debunking this claim.
And that's at the core of what we're talking about here:
there are a lot of folks out there that won't let the truth get in the way of a good sound byte
Let's take a closer look at one of these stats.
First off, TRUST YOUR GUT. If it sounds like BS, it's worth investigating.
In 2017, I set out to document every company that had ever been destroyed by a breach.
How many did I find?
Only 23 over a 20 year period.
All small businesses; maybe 4 had over 100 employees, but all less than 500
Is it possible I missed some, or some didn't get reported? Sure!
Is it possible I'm missing 150 PER YEAR? Probably not.
This answers a more broad and general assumption in our industry: the assumption that security incidents MUST be ending companies
Ramon Ray: founder and owner of smallbiztechnology.com and Smart Hustle.
He was still using the stat as recently as late 2021, five years after NextGov interviewed him for their article debunking this claim.
And that's at the core of what we're talking about here:
there are a lot of folks out there that won't let the truth get in the way of a good sound byte
Let's take a closer look at one of these stats.
First off, TRUST YOUR GUT. If it sounds like BS, it's worth investigating.
In 2017, I set out to document every company that had ever been destroyed by a breach.
How many did I find?
Only 23 over a 20 year period.
All small businesses; maybe 4 had over 100 employees, but all less than 500
Is it possible I missed some, or some didn't get reported? Sure!
Is it possible I'm missing 150 PER YEAR? Probably not.
This answers a more broad and general assumption in our industry: the assumption that security incidents MUST be ending companies
Ramon Ray: founder and owner of smallbiztechnology.com and Smart Hustle.
He was still using the stat as recently as late 2021, five years after NextGov interviewed him for their article debunking this claim.
And that's at the core of what we're talking about here:
there are a lot of folks out there that won't let the truth get in the way of a good sound byte
Let's take a closer look at one of these stats.
First off, TRUST YOUR GUT. If it sounds like BS, it's worth investigating.
In 2017, I set out to document every company that had ever been destroyed by a breach.
How many did I find?
Only 23 over a 20 year period.
All small businesses; maybe 4 had over 100 employees, but all less than 500
Is it possible I missed some, or some didn't get reported? Sure!
Is it possible I'm missing 150 PER YEAR? Probably not.
This answers a more broad and general assumption in our industry: the assumption that security incidents MUST be ending companies
Ramon Ray: founder and owner of smallbiztechnology.com and Smart Hustle.
He was still using the stat as recently as late 2021, five years after NextGov interviewed him for their article debunking this claim.
And that's at the core of what we're talking about here:
there are a lot of folks out there that won't let the truth get in the way of a good sound byte
Let's take a closer look at one of these stats.
First off, TRUST YOUR GUT. If it sounds like BS, it's worth investigating.
In 2017, I set out to document every company that had ever been destroyed by a breach.
How many did I find?
Only 23 over a 20 year period.
All small businesses; maybe 4 had over 100 employees, but all less than 500
Is it possible I missed some, or some didn't get reported? Sure!
Is it possible I'm missing 150 PER YEAR? Probably not.
This answers a more broad and general assumption in our industry: the assumption that security incidents MUST be ending companies
Ramon Ray: founder and owner of smallbiztechnology.com and Smart Hustle.
He was still using the stat as recently as late 2021, five years after NextGov interviewed him for their article debunking this claim.
And that's at the core of what we're talking about here:
there are a lot of folks out there that won't let the truth get in the way of a good sound byte
99.5% of losses are unaccounted for - could we make a case that these are all indirect losses? I don't think so...
99.5% of losses are unaccounted for - could we make a case that these are all indirect losses? I don't think so...
99.5% of losses are unaccounted for - could we make a case that these are all indirect losses? I don't think so...
99.5% of losses are unaccounted for - could we make a case that these are all indirect losses? I don't think so...
99.5% of losses are unaccounted for - could we make a case that these are all indirect losses? I don't think so...
99.5% of losses are unaccounted for - could we make a case that these are all indirect losses? I don't think so...
Even though the tweet engagement was almost non-existent
Cylance held a grudge for years
They mis-represented a Verizon DBIR stat
You're required to get permission from VZ before using their stats, for exactly this reason
They assumed the top of the graph = 100%, when it actually = ~35%
I DMed some of the DBIR folks and had the raw data from the graph in my hands in under 30 minutes
The danger here is that if you tell defenders that 90% of their problem is malware...
there's a really good chance they're going to find a way to justify pouring 90% of their resources into addressing it!
At the detriment of other areas that need budget and attention
When myths and lies prevail, they can cause us to choose the wrong path
Even though the tweet engagement was almost non-existent
Cylance held a grudge for years
They mis-represented a Verizon DBIR stat
You're required to get permission from VZ before using their stats, for exactly this reason
They assumed the top of the graph = 100%, when it actually = ~35%
I DMed some of the DBIR folks and had the raw data from the graph in my hands in under 30 minutes
The danger here is that if you tell defenders that 90% of their problem is malware...
there's a really good chance they're going to find a way to justify pouring 90% of their resources into addressing it!
At the detriment of other areas that need budget and attention
When myths and lies prevail, they can cause us to choose the wrong path