Abstract: The Building Security In Maturity Model (or BSIMM) BSIMM observes and measures what firms' software security initiatives are actually doing. John, who has helped several firms build or improve their security initiatives, will share sometimes surprising data about security initiatives big and small. His presentation will focus on what activities organizations use to "boot" security initiatives and which they presently focus on.

1. The OWASP Foundation
http://www.owasp.org
Building a Security Initiative
( Field +XP & Measures )
-jOHN (Steven)
Internal CTO, Cigital Inc.
@m1splacedsoul

2. The OWASP Foundation
http://www.owasp.org
This Presentation
…is about observed trends, DISCUSSION to follow
Wild West AppSec - State of assessment
Growing Up – Security Initiatives
BSIMM – Measuring Security Initiatives
What Most Firms Are „On Top‟ of…
What Firms Struggle with Today

3. The OWASP Foundation
http://www.owasp.org
’06: Shift Philosophy to HOW
 Cigital’s Touchpoints
 Microsoft’s SDL
 OWASP CLASP
(2001)

4. The OWASP Foundation
http://www.owasp.org
State of Assessment

5. The OWASP Foundation
http://www.owasp.org
Assessment is TOUGH
Dynamic Assessment (tools)
<= 10% statement coverage
IFF Authenticated
Manual Penetration Testing?
Including “Expert Crawling”
What about static analysis (tools)?
SCR?

6. The OWASP Foundation
http://www.owasp.org
Actual Results Breakdown




Static tool: 20%
Dynamic tool: 5%
Manual SCR: 15%
Architecture Risk
Analysis: 60%






Static tool: 12%
Dynamic Tool: 12%
Manual SCR: 21%
Manual Pen: 21%
ARA: 14%
Sec Testing: 20%

7. The OWASP Foundation
http://www.owasp.org
We Won‟t Test Our Way to
Security,
Orgs need Security Initiative

8. The OWASP Foundation
http://www.owasp.org

9. The OWASP Foundation
http://www.owasp.org
A software security initiative
more
A software security initiative is an:
 executive-backed,
 permanently-staffed,
 metrics-driven
investment in…
 software security policy and standards,
 “secure SDLC” gates, and
 governance knowledge, processes, and tools
to implement capabilities across a reasonable cross-section of the application
portfolio.

10. The OWASP Foundation
http://www.owasp.org
Security Initiative !=
Does * NOT * mean…
Heavy
Waterfall
Process
Microsoft SDL
Audit

11. The OWASP Foundation
http://www.owasp.org
Security Initiative ~=
May look very different than other
organizations’
Needs to match an organization’s
culture

12. The OWASP Foundation
http://www.owasp.org
Where Orgs Are
…and how do we know?
We‟ve measured.

13. The OWASP Foundation
http://www.owasp.org
Building BSIMM (2009)
 Big idea: Build a maturity model from actual data gathered from 9
well known large-scale software security initiatives
Create a software security framework
Interview nine firms in-person
Discover 110 activities through observation
Organize the activities in 3 levels
Build scorecard
 The model has been validated with data from 51 firms

14. The OWASP Foundation
http://www.owasp.org
Prescriptive vs. Descriptive
 Prescriptive models
describe what you should
do




SAFECode
SAMM
SDL
Touchpoints
 Every firm has a
methodology they follow
(often a hybrid)
 You need an SSDL
 Descriptive models describe
what is actually happening
 The BSIMM is a descriptive
model that can be used to
measure any number of
prescriptive SSDLs

15. The OWASP Foundation
http://www.owasp.org
Monkeys Eat Bananas
 BSIMM is not about good or bad
ways to eat bananas or banana
best practices
 BSIMM is about observations
 BSIMM is descriptive, not
prescriptive
 BSIMM describes and measures
multiple prescriptive approaches
15

16. The OWASP Foundation
http://www.owasp.org
Yeah but we‟re different
You *are* a special snowflake, just
like everyone else
All snowflakes are equally special
No matter how special a snowflake
you are, you‟ll still melt when it‟s
hot out.

17. The OWASP Foundation
http://www.owasp.org
…but they‟re HUGE right?

18. The OWASP Foundation
http://www.owasp.org
BSIMM Basics

19. The OWASP Foundation
http://www.owasp.org
A Software Security
Framework
 Four domains
 Twelve practices
 See informIT article on BSIMM website http://bsimm.com

20. The OWASP Foundation
http://www.owasp.org
Architecture Analysis Practice
Skeleton

21. The OWASP Foundation
http://www.owasp.org
…It could have been worse

22. The OWASP Foundation
http://www.owasp.org
Where Orgs Are
(Actually this time)

23. The OWASP Foundation
http://www.owasp.org
We Hold These Truths to be
Self-evident
 Someone (a security group) has to be responsible
 Software security is more than a set of security functions
 Not magic crypto fairy dust
 Non-functional aspects of design are essential
 Not silver-bullet security mechanisms
 Bugs and flaws are 50/50
 To end up with secure software, deep integration with the SDLC is
necessary

24. The OWASP Foundation
http://www.owasp.org
12 Common Activities
1.
SM1.4 Identify gate locations, gather necessary artifacts
2.
CP1.2 Identify PII obligations;
3.
T1.1 Provide awareness training;
4.
AM1.5 Gather attack intelligence;
5.
SFD1.1 Build and publish security features;
6.
SR1.1 Create security standards;
7.
AA1.1 Perform security feature review;
8.
CR1.4 Use automated tools along with manual review;
9.
ST1.1 Ensure quality assurance (QA) supports edge/boundary value condition
testing;
10.
PT1.1 Use external penetration testers to find problems;
11.
SE1.2 Ensure host and network security basics are in place; and
12.
CMVM1.2 Identify software defects found in operations monitoring and feed them
back to development.

25. The OWASP Foundation
http://www.owasp.org
Evolving Initiatives (2012)
 Build an SSG
 Something in Architecture
 Use automated tools @ scale
 Security Sign-off
3rd*
Party
Metrics
VA
Configuration*
Management
Vulnerability*
Management
CR*
Portal
Security*
Sign9off
Attack*
Intelligence
Assessment

26. The OWASP Foundation
http://www.owasp.org
Something in Architecture
US vs. Them *
Ugly babies *
Unfunded fixes *
Lock-in *

27. The OWASP Foundation
http://www.owasp.org
One Architecture Climb
3.2 Results 
Arch. Patterns
Year 5
2.3 Make SSG
Available
1.3 SSG
Reviews
2.2 Standardize
Descriptions
1.2 Perform
Review
1.1 Feature
Review
Year 3
Year 2
Year 1

28. The OWASP Foundation
http://www.owasp.org
Automation =
<anything> + Plumbing

29. The OWASP Foundation
http://www.owasp.org
Static Step by Step

30. The OWASP Foundation
http://www.owasp.org
Plumbing can mean email…

31. The OWASP Foundation
http://www.owasp.org
Real Sign-off

32. The OWASP Foundation
http://www.owasp.org
Evolving Initiatives (2014)
 Metrics driving budget
 Gather attack Intelligence
 Security comes to Agile
 Open source risk
 Something in Architecture, maybe threat modeling? (again)
 Security BAU
 Dev doing Security (particularly static testing)
 CM& VM plumbing (making previous ideas tools)
3rd*
Party
Metrics
VA
Configuration*
Management
Vulnerability*
Management
CR*
Portal
Security*
Sign9off
Attack*
Intelligence
Assessment

33. The OWASP Foundation
http://www.owasp.org
Metrics-driven Budget

34. The OWASP Foundation
http://www.owasp.org
Security Intelligence

35. The OWASP Foundation
http://www.owasp.org
Threat Traceability Matrix
Who
Where
What
How
So what?
Now what?
Threat
Attack
Surface
Asset/Privileg
e
Attack Vector
Impact
Mitigation

36. The OWASP Foundation
http://www.owasp.org
Addressing Threat Intel
helps the Something
(Anything)
in architecture

37. The OWASP Foundation
http://www.owasp.org
SSIs Fit Naturally into Agile
Top 2,3
Awareness (pre-training)
Top 10
Passwords, SSL
[Open Source] Automation
Configuration Mgmt, plumbing
Infrastructure Security
API
Threat Modeling
Risk Management
Security Libraries

38. The OWASP Foundation
http://www.owasp.org
Vuln + Config. Management
Build a pile, rank the pile
Rank applications w/in portfolio
Call a spade a spade
Standardize names for vulnerabilities
Normalize assessment / tool scoring
Prioritize
Calculate risk effectively
Go from “hated cop” to B.A.U.
Establish security gates
Integrate with normal change/bug management