How to do threat modeling in the age of Agile and DevOps. A practical methodology for teams focusing on developers. Also, an introduction to PyTM as a tool for threat-modeling-with-code.
"Threat Model Every Story": Practical Continuous Threat Modeling Work for Your Team
1. Threat Model Every Story:
Practical Continuous Threat
Modeling Work for Your Team
or ”What Do You Mean Threat Model EVERY Story Who Has That Kind
of Time Go Away and Take Your Threat Model With you!?!”
2. The Bureau of Statistics informs:
No surveys were harmed in the making of this
talk. All data is purely anecdotal and open to
subjective interpretation based on the reader’s
experience.
3. • Lead Security Architect at Autodesk Inc.,
focusing on application security
• SAFECode collaborator, IEEE Center For
Security Design collaborator
• Curious argumentative all-around kvetcher
Who am I?
4. • Show of hands
– You threat model every day
– You want to add threat modeling to your practice
– You do research work on Threat Modeling
– You are in the wrong room and too shy to leave
after three slides into the presentation
Who are you?
5. • Level setting – threat modeling, what and why?
• My personal threat modeling journey
• Challenges encountered in this unfinished journey
• How I am trying to solve that – Continuous Threat
Modeling
– How can you use it?
– Tools
– References
What are we doing here today?
6. • A conceptual exercise that aims to identify security-
related flaws in the design of a system and identify
modifications or activities that will mitigate those flaws.
• Formally, it can be “A technique to identify the attacks a
system must resist and the defenses that will bring the
system to a desired state” (Brook Schoenfield)
• Four Fundamental Questions (Adam Shostack)
– What are we working on?
– What can go wrong?
– What are we going to do about it?
– Did we do a good job?
Threat Modeling – what & why
7. A Threat Modeling (ongoing) personal
journey
Role TM Methodology
developer Discuss the latest published exploits. Wonder aloud if they apply to my code.
developer STRIDE
developer/
architect
STRIDE/element
security
consultant
Threat Library
security
consultant
A frank conversation about the system and what it wants to be (“SME-led”)
security
consultant
TM “spikes”
8. But what was I looking for?
• Accessible: can a product team do it independently after
a brief period of instruction? Can they keep doing it?
• Scalable: can the same thing be done over many teams
and products? In a short time?
• Educational: can it teach instead of correct?
• Useful: are the results found useful for both product and
security?
• Agile: is it repeatable, does it negatively impact the
product team’s velocity?
• Representative: how does the system being modeled
compare with the model?
• Unconstrained: once the “known suspects” are
evaluated, is the team led to explore further?
11. Threat Model Every Story
• build a baseline - involving everyone. Use whatever technique works for
your team. At Autodesk we are currently focusing on a “subject based” list
of points of interest
• designate one or more “threat model curators” who will be responsible for
maintaining the canonical threat model document and the findings queue
• instruct your developers to evaluate each one of their stories with focus
on security:
– if the story has no “security value”, continue as usual
– if the story generates a security “notable event”, either fix it (and document as
a mitigated finding) or pop it up as a “threat model candidate finding” for the
curator to take notice of (at Autodesk we are doing this using labels on JIRA
tickets)
• make sure your curators are on top of the finding and candidate finding
queues
12. But…how do my developers know
what has “security value”?
Subject areas
Question and then
continue
questioning during
“official design
time” or when
building a baseline
Checklist
Verify that the
principles have
been followed at
implementation
time
15. Threat Model Every Story
• build a baseline - involving everyone. Use whatever technique works for
your team. At Autodesk we are currently focusing on a “subject based” list
of points of interest
• designate one or more “threat model curators” who will be responsible for
maintaining the canonical threat model document and the findings queue
• instruct your developers to evaluate each one of their stories with focus
on security:
– if the story has no “security value”, continue as usual
– if the story generates a security “notable event”, either fix it (and document as
a mitigated finding) or pop it up as a “threat model candidate finding” for the
curator to take notice of (at Autodesk we are doing this using labels on JIRA
tickets)
• make sure your curators are on top of the finding and candidate finding
queues
17. • “Uh...what?”
• “This is still too heavy”
• “But how do I know I did everything?”
• “I never saw a room of architects excited
about threat modeling before”
Reactions from product teams
18. Caveat Emptor: This Is Not Perfect
• Difficult to convince teams that the Subject List is
not a threat library and developers that the
Checklist is not a requirements list – not
exhaustive, just a starting point
• The resulting TM won’t be perfect – evolutionary
• A SME or security group is still necessary for
education
• GIGO – garbage-in, garbage-out
19. So…about that automation thing.
• What are the parts of Threat Modeling we can most easily
automate?
– Diagraming - cross-platform, over the network, simple and quick
yet representative
– Reporting - having a standard and keeping to it; information
passing
– Threat ranking - CVSS or some other agreed ranking system
– Low-hanging fruit - threats that can be immediately derived
from a formal description of the system should emerge
• Tooling should help discuss the system, keep the model as
close as possible to the reality of the system, disseminate
information, and not hinder collaboration:
20. What is available today?
• There are many threat modeling tools; some are platform-
dependent, like the MS Tool, others are web-based
• Some start the process with a questionnaire along the lines of
“what do you want to build” and generate a list of requirements
that the developers must follow
• Others get a description of the system and generate threats based
on characteristics of the design
• But … developers write code; why not have them feed the threat
model with something that looks like code?
• “TM-as-code” is in the same place “DevOps” was a couple of years
ago. There is talk of, people want to do it, but the definition of what
it actually means is murky
21. Three current practical approaches
ThreatSpec Fraser Scott
@zeroXten
Threat modeling IN
code
ThreatPlaybook Abhay Bhargav
@abhaybargav
Threat modeling
FROM code
PyTM Threat modeling
WITH code
22. PyTM – A Pythonic way of TM’ing
Matt Coles, @coles_matthewj Nick Ozmore, @nozmore
Rohit Shambhuni, @rshambho Izar Tarandach, @izar_t
30. PyTM – how is it being used?
• during team meetings to create the initial
diagram
• in discussions with the product team - “it is
missing this attribute”, “why is this a threat”,
“what if?”
• keep threat models in revision control, together
with the code they describe and generate
automated, standard threat model reports
31. PyTM – invitation to collaborate
• More threats
• More elements
• Documentation
• A better rule engine
• Integration with other tools
• Suggestions, bug fixes, requirements, sample use
cases
• Discussion!
Running for 8 or so months.
Results are promising but not enough empirical data yet.
not saying one thing is better than another
just offering the justification and the initial findings of the experiment
8 years at EMC
IBM
start-up game
consulting work
calibrate comments - need to know who you are
just a brief agenda
Ideally, traditionally performed at the design stage.
The objective is to generate findings, but may serve as a framework for other stages of the secure software development cycle.
Widely agreed to be an activity with high returns on investment: flaw correction, learning and documentation are immediate products, with training, testing and of course development also impacted.
The four questions start with understanding, then wondering, then inventing, then measuring.
As a developer I was all about the results. I just wanted to know which holes I had to plug.
As an architect, I focused on coverage. Every single aspect of the system had to be considered to exhaustion, lest something fall between the lines.
As a security consultant, it all became about scalability and being able to help the product team as much as possible, hopefully at some point growing security champions and architects that would take over the consulting. It was also about adapting to the rhythm of the team.
The issue with STRIDE and STRIDE/element is that they need someone who knows how things can be broken in order to figure out what is potentially possible as a flaw, or they need the use of attack trees that add complexity and weight to the process. As a learning tool they have their value since they lead to discussion of those findings, but they still depend on having someone, a SME, to do that teaching. They are not constrained, but at the same time a lack of structure may leave areas unresolved or unexamined. They don’t scale well due to the time needed, the knowledge needed, the need for a SME.
The threat library is an improvement on needing a SME, as the team looks at issues that have historically been a problem. On the other side, that tends to constrain the team to only those issues, as they look to discharge themselves of the requirement of a threat model, and the exercise becomes a compliance checklist rather than an examination. The issues are very descriptive and lead to over-focus. It scales better than STRIDE, but still needs the SME to explore beyond the library.
Having a tailored conversation on each product amounts to a SME-led threat model. It will surely generate the most correct findings, but at the same time it will not scale at all, it will not teach much to the product team, but in small shops it may be the easiest and best way to go
The idea of continuously improving a threat model is not new - it has been pushed, for example, by Microsoft for a long time - but it was in a vague sense, that “a threat model should be updated in every sprint” without giving a practical way to do so that didn’t fall foul of the other issues. It assumes that all the resources needed for a “threat modeling spike” are in place and willing to do what’s needed, and that the process is easily reduced to the scope of a sprint.
The saying that “threat modeling happens at the design stage” used to be fine when we were waterfalling, since you could threat model until you had an acceptable design that would then be implemented. Nowadays with agile methods and emergent designs we don’t have that luxury anymore.
Seymour Cray - The problem with programmers is that you can never tell what a programmer is doing until it is too late.
to the meat of the thing.
Richard Feynman: Do your own homework. To truly use first principles, don’t rely on experts or previous work. Approach new problems with the mindset of a novice. Truly understand the fundamental data, assumptions and reasoning yourself. Be curious.
This is where the education angle comes in.
We are training our developers wrong. We are providing huge amounts of information that do not correspond to huge amounts of useability. We are doing the busywork of teaching people how the RSA algorithm works without focusing on the aspects of choosing the right key length, algorithm and secret protection scheme that their system needs.
In order to create sensitivity to what “security notable events” are, we at Autodesk are experimenting with providing developers with a checklist that they use as part of the definition of done of their stories.
The subject areas are more important than the sample questions.
This checklist follows a “if this then that” model - the developer only needs to relate to those items that are relevant to the work at hand
The language on the “if” side is developer language. There is no need to decipher what the security team intends in order to figure out if something is relevant or not
The checklist is limited in length - one double sided printed page should be the limit so ideally it can be printed and kept at hand
The “then that” side is not prescriptive. It pushes the developer to search for the information that relates to whatever environment or framework they are using. This is for three reasons - to keep the list short, to make it somewhat open-ended, and to tickle the curiosity innate to most developers. Pointers are given but not “absolute solutions”
The checklist is backed by documentation and live support by the security team
It is made clear to the developer - once you don’t need the list anymore, throw it away.
The list focuses on teaching fundamentals, not formulas.
If we look at the threat modeling spike in detail what we see is that at the end of the sprint, the same process used to generate the baseline threat model should be again used to update it. The mitigator is that this time only those things that changed will need to be revisited. That is well and fine, but it still doesn’t answer the basic questions:
who is responsible for doing the update? the whole team? the owner of the tm
who will provide guidance? is a SME available?
when will the findings be fixed? is a finding enough to hold back a story? are they automatically addressed in the next sprint?
Tm-IN-code – threat modeling happens as code is written and mixes with the code, encapsulates the problem with the solution
Tm-from-code - deriving previously identified threats from other tools, validating or discovering threats present in code and providing a proper language to talk about these threats
Tm-with-code - we use code to express the system to be modeled and derive information about it