Abhay Bhargav's DEFCON 25 Workshop on using Python to build AppSec Automation for integration with CI/CD Pipelines

1. Building AppSec Automation with
python
Abhay Bhargav - CTO, we45

2. Yours Truly
• Co-author of Secure Java For Web Application Development
• Author of PCI Compliance: A Definitive Guide
• Speaker at OWASP Conferences worldwide
• Avid Pythonista and AppSec Automation Junkie
• Specialization in Web Application Security and Security Testing
• Trainer and Workshop Lead for Security Training Workshops

3. Today's Session
• A Gentle Introduction to DevOps and DevSecOps
• Perspectives - AppSec Automation and its importance in DevSecOps
• DevSecOps Engineering Perspective
• Pentester Perspective

4. Tools we will be using today
• OWASP ZAP 2.6.0
• mitmproxy 0.17
• TinyDB (Document Database - Embedded/File based DB (JSON)
• pygal, tabulate - visualization libs and table generator
• iPython Notebook - Examples
• ZAP Python Scripting Framework
• Examples are in py2 - But can be ported to py3 with little effort

5. Things I have to say…
• This is **NOT** Production-Ready Code. This is meant to demonstrate concepts. Use
at your own risk.
• The apps and the VM we are using are vulnerable. Please DO NOT put it on a
network
• This is not the ONLY way to achieve these results. You have varied tools,etc.
Solutions are also across platforms and programming languages.
• A lot of what we are dealing with is unchartered territory, so documentation is
sparse. A lot of hacking and research is required

6. Vulnerable apps we will be using
• http://localhost => Django based app, multiple vulnerabilities
• http://localhost:5050 => Flask-based Web Service
• Specific Vulnerabilities we will be dealing with today:
• SQL Injection
• Insecure Direct Object Reference
• Weak HMAC Key - JSON Web Token
• none token flaw - JSON Web Token
• Server-Side Template Injection

7. Reading code…
• Most of it is in iPython Notebooks
• For everything else, you can use mousepad or vim (depending on your level of
comfort)

8. A Gentle Introduction to DevOps
• What is DevOps?
• Where does Security fit in?

9. What is DevOps?
• Key Objective - Harmonize IT Operations by
working with Developers and Ops
seamlessly
• Rely on processes and automation to
achieve higher throughput - Continuous
Delivery

10. Without DevOps
Requirements Design Develop Test Deploy

11. With DevOps (hopefully…)
Requirements Design Develop Test Deploy

12. Example pipeline
Developer Orchestration engine
Coding -
Modify and commit
Git:
Checkout for build
Jenkins:
Continuous
Integration
Docker:
Publish to
repo
success
Docker:
Deploy to
QA
Selenium:
Run tests
QA
success
Docker:
Deploy to
Prod

13. But….
Requirements Design Develop Test Deploy

14. Let’s do a security test
just before we go live….
The line that has ruined
Application Security for all of us.

15. In Short….

16. CI/CD Pipeline
Trigger ARA
Trigger manual code review
Email notifications
Configuration review
Trigger threat modelling
Run SAST tools
Automatic security testing
Gather metrics
Break the build
Compile and build code
SCA
Risk based security testing
Gather metrics
Break the build
Comprehensive SAST
Pre-commit checks
Commit-time checks
Build-time checks

17. CI/CD Pipeline
DAST/AST
Malicious code detection
Gather metrics
Break the build
Broader SAST
Pre-deployment checks
Post-deployment checks
Test-time checks
Commit-time checks
Continuous management
Provisioning runtime environment
Security scanning
Vulnerability scanning
Bug bounty program
Threat intelligence

18. The Need of the Hour….
• Continuous Application Security Practices to
keep pace with Continuous Delivery
• Dynamic Application Security Testing in
the Pipeline
• Static Application Security Testing in the
Pipeline

19. Security in DevOps
Plan
Code
Build
Test
Release
Deploy
Operate
Monitor
Threat
modeling
SAST
Security - Composition
DAST
IAST
Security in
IaC
Security monitoring
& attack detection

20. The Application Security Engineer Story
• How?
• Run DAST in the Pipeline?
• Correlate Results from DAST
• Compare Results from scans in time?

21. The Need of the Hour….
• Continuous Application Security Practices to
keep pace with Continuous Delivery
• Dynamic Application Security Testing in
the Pipeline
• Static Application Security Testing in the
Pipeline

22. Our Approach Today
• A View of DAST in the Pipeline
• Tool of Choice: OWASP ZAP
• with:
• Jenkins
• Customized Python Scripts
• ElasticSearch/Redis
• Objective: Explore Automated DAST Testing
Approaches with OWASP ZAP and its Python
API

23. Why OWASP ZAP?
• Free and Open Source Web Application
Vulnerability Scanner
• Feature-Rich, well supported, with several
contributors
• Community Support - Plugins, Add-ons, etc.
• Documentation - Better than most scanners
out there
• Great API and Scriptable Scanner

24. Security in DevOps
Plan
Code
Build
Test
Release
Deploy
Operate
Monitor
Threat
modeling
SAST
Security - Composition
DAST
IAST
Security in
IaC
Security monitoring
& attack detection

25. Stories for today….
• The Application Security Engineer/
DevSecOps Engineering Perspective
• The Automation-focused Pentester
Perspective

26. Key Questions - AppSec Engineering/DevSecOps
• How do we roll out Automated Security
Testing in the pipeline?
• Authenticated Scanning in the Pipeline -
for Apps/API, etc
• Account for changes in Attack Surface

27. Introduction to the OWASP ZAP API
• OWASP ZAP - Automation
• Concept Overview
• Useful Concepts and API
• OWASP ZAP Python API Deep-Dive
• Workshop Exercises

28. Concept Overview - OWASP ZAP
• Context
• Session
• Active Scan
• Passive Scan
• Scan Policy
• Alert

29. Workshop Exercise - Basic ZAP Functionality
• Concept overview:
• Context
• Sites
• Scan Policy
• Scripts
• Script Console

30. ZAP - Useful API Operations
from zapv2 import ZAPv2 as ZAP
zap.spider
#spider operations
zap.core
#App-wide operations
zap.ascan
#Active Scan
zap.pscan
#Passive Scan
zap.script
#Operations with ZAP Scripts
zap.context
#Context related operations

31. ZAP API Quicksearch operations
zap.spider.scan()
#initiate ZAP Spider Scan against target
zap.ascan.scan()
#initiate ZAP Active Scan against Target
zap.core.alerts()
#all alerts (scan results) from the ZAP Scanner
zap.core.urls()
#list of URLs from ZAP
zap.ascan.status(), zap.spider.status()
#real time status of the spider or ascan
zap.ascan.scan_progress()
#List of Vulnerabilities being tested for with number of payloads

32. Workshop Exercise: ZAP Unauth Scan
from zapv2 import ZAPv2 as ZAP
import time
zap = ZAP({'http': 'http://localhost:8090', 'https': 'http://localhost:8090'})
zap.urlopen('http://localhost/')
zap.spider.scan(target = 'http://localhost/')
while (int(zap.spider.status()) < 100):
print zap.spider.status()
time.sleep(5)
zap.ascan.scan(target = 'http://localhost/')
while (int(zap.ascan.status()) < 100):
print zap.ascan.status()
time.sleep(5)

33. Workshop Exercise - ZAP API Walkthrough
1. ZAP Tool walkthrough
2. Run Unauthenticated ZAP Scan - GUI or API
3. ipython walkthrough

34. Running Authenticated Scans in OWASP ZAP
• Approaches:
• Selenium-driven Scan Process
• Leveraging canned ZAP Sessions
• Zest Scripting

35. Selenium-Authenticated Scan
Run Selenium and ZAP in Headless Mode
Leverage Functional Scripts
Beats Spidering the app! :)

36. ZAP Session-Authenticated Scan
Programmatically invoked with ZAP API
Maintains state with Sessions/Tokens, etc

37. ZestScript Authenticated Scan
Programmatically invoked with ZAP API
Easily Customizable

38. Workshop Exercise - Automated, Authenticated ZAP Scans
1. Selenium-ZAP Scan - Follow the HTML Instructions
2. ZAP Session Scans - Follow the HTML Instructions
3. Zest ZAP Scans - Follow the HTML Instructions

39. Common Problems - ZAP Integration
• ZAP Selenium - Driver versions and Browsers tend to be difficult to handle
• Impact on Zest Scripts
• ZAP Authentication - Getting to work for complex flows is difficult
• ZAP Sessions - Maintain state, so you need to reload a new version of a session file, every time.

40. ZAP in the Continuous Delivery Pipeline

41. Workshop Exercise - Automated, Authenticated ZAP Scans
• Authenticated ZAP Scans - Jenkins Integration - Follow HTML Instructions

42. Correlating DAST Results
• The Common Weakness Enumeration (CWE)
system is the best we have for correlation right
now
• Problems:
• Several tools don’t give any/accurate CWEs
• Multiple CWE values tend to be difficult to
handle and correlate with - BurpSuite, etc

43. Correlation and Aggregation - Success Factors
• Breakdown the Vulnerability into your own structure
• Standardize this structure to capture all relevant details
• Store this relevant info in a DB - Any one would do, but Document DBs offer some benefits
IMHO

44. Workshop Exercise
1. Visualization and Correlation across Scans

45. AppSec Automation - A Pentester’s perspective
• How do we go beyond traditional DAST?
• Scale Custom/Business Logic Security
Flaws
• Create Custom Application Exploits for
non-standard/esoteric flaws
• Create a Library of attacks extending/
complementing DAST Scanners

46. Custom Exploits - Integration
• ~50% of security flaws are those that cannot be easily identified by tools - Authentication,
Authorization, Crypto flaws, etc.
• Pentests, Bug Bounties find custom exploits in apps.
• However, including them in the Continuous Security Testing Op is also important
• That’s where custom scripts come in….

47. Tools we will use
• OWASP ZAP 2.6.0
• mitmproxy 0.17

48. OWASP ZAP - Scripting Framework
• Python (Jython)
• Ruby
• Oracle Nashorn ECMAScript
• Zest Scripts

49. OWASP ZAP - Scripting Framework
• Active Rules => Scripts invoked during Active Scan
• Authentication Scripts => Scripts invoked to facilitate
authentication for a Context
• Fuzzer Processors => Scripts invoked after Fuzzers are run
with ZAP
• HTTPSender => Scripts invoked against every request/
response received by ZAP
• Proxy => Runs inline and acts on all requests and responses
• Targeted Rules => Invoked on specific urls or on manual
start only
• Standalone => Invoked manually
• Passive Rules => Passive Scanning Rules

50. Configuring ZAP to run with Python
• ZAP supports scripts written in Jython
• Python on Java JVM
• Not fully compatible with python libraries
• limitations on networking and i/o libraries in python
• Works when œPython Scripting’ add-on is installed in OWASP ZAP.
• Third Party Python Libs can be linked when refer to the jython site-packages directory

51. mitmproxy
• Primarily used as an extensible, interception proxy.
• Powerful Inline scripting framework
• Pure Python implementation :) - Highly extensible and scriptable
• Current version is 2.x on python 3 only

52. ZAP Scripting QuickSearch
msg
#the message object that is acted upon to parse/manipulate
msg.getRequestHeader()
#Request Header Object
msg.getRequestHeader().getURI()
#fetches the URI from the request header
msg.getRequestBody()
#Fetches the request body from the request
msg.getResponseBody()
#Fetches the request body from the request
msg.setRequestBody()
#Sets a different request body from the one in the original request

53. ZAP Active Rules Template
"""
The scanNode function will typically be called once for every page
The scan function will typically be called for every parameter in every
URL and Form for every page
"""
def scanNode(sas, msg):
#Invoke something for every page here
def scan(sas, msg, param, value):
#invoke something for every param here.
sas.raiseAlert(1, 1, 'Active Vulnerability title', 'Full description',
msg.getRequestHeader().getURI().toString(),
param, 'Your attack', 'Any other info', 'The solution
', '', 0, 0, msg);

54. ZAP Proxy Template
def proxyRequest(msg):
# Debugging can be done using print like this
print('proxyRequest called for url=' +
msg.getRequestHeader().getURI().toString());
return True;
def proxyResponse(msg):
# Debugging can be done using print like this
print('proxyResponse called for url=' +
msg.getRequestHeader().getURI().toString());
return True;

55. mitmproxy inline scripting
def request(context, flow):
flow.request.headers
#request headers object
flow.request.host
#host in the request
flow.request.path
#request path
flow.request.content
#request body
def response(context, flow):
flow.response.headers
# request headers object
flow.response.host
# host in the request
flow.response.path
# request path
flow.response.content
# request body

56. Workshop Exercises
1. ZAP POST Request Insecure Direct Object Reference Active Script
2. ZAP JSON Insecure Direct Object Reference Active Script
3. mitmproxy JWT Bruteforce Script
4. None JWT Flaw

57. Reach me
• Twitter: @abhaybhargav
• LinkedIn: www.linkedin.com/in/
abhaybhargav