OWASP Testing Guide (Recon)

1. OTG-RECON
OWASP Thailand Chapter (September 2015)

2. $whoami
>>> import speaker
>>> print(fullname)
Sedthakit Prasanphanich
>>> print(hobby)
[‘IT-Sec’, ‘Basketball’, ‘Reading’, ‘Games’, ‘Anime’]
>>> print(OWASP_role)
Thailand Chapter Manager (General Butler)
>>> print(work)
Web Security Engineer

3. Disclaimer
• This presentation assume that you have a little tiny bits of
“Information Gathering”.
• If you don’t have any ideas about the previous bullet,
please consult Google, Yahoo, Bing, Baidu (I’m not
joking),…
• If you don’t understand any jargon or gibberish word,
please ask me what it is!
• This presentation is not a bible. Need more info, Read the
OWASP Testing Guide!
• Found any errata or having any suggestion, Please
subscribe to the mailing list
(https://www.owasp.org/index.php/OWASP_Testing_Projec
t#tab=Feedback_and_Participation)

4. Agenda
• Smalltalk about Breaker, Security Tester, or
Hacker.
• OTG 101
• OTG-INFO-x WHERE x >= 0 && x <=10;

5. #define hacker

6. –Gehring, Verna (2004). The Internet In Public Life. Maryland: Rowman &
Littlefield Publishers. pp. 43–56.
“a person who enjoy the intellectual challenge of
creatively overcoming and circumventing
limitations of systems to achieve novel and
clever outcomes.”

7. –Richard M. Stallman, 2002, and 2000 respectively
“It means someone who enjoys playful
cleverness, especially in programming but other
media are also possible.”
“hacking means exploring the limits of what is
possible, in a spirit of playful cleverness.
Activities that display playful cleverness have
"hack value".

8. #import OTG 101

9. What is Web Application Security
Testing?
• a method of evaluating the security of a web
application by methodically validating and
verifying the effectiveness of application security
controls.

10. What is a Test?
• A test is an action to demonstrate that an
application meets the security requirements of its
stakeholders.

11. The Approach in Writing this
Guide
• Open: every security expert can participate with his or
her experience in the project. Everything is free.
• Collaborative: brainstorming is performed before the
articles are written so the team can share ideas and
develop a collective vision of the project. That means
rough consensus, a wider audience and increased
participation.

12. goto: Recon

13. Why information gathering so
important?
• All possible information is gathered about the
target
• very little footprints generally left by the attacker

14. –Sun Tzu
“Every battle is won before it is fought”

15. >>> i=1
>>> while i<11:
... if i==10:
... print "OTG-INFO-010"
... else:
... print "OTG-INFO-" + "00" + str(i)
... i+=1

16. Conduct search engine
discovery/reconnaissance for
information leakage (OTG-INFO-
001)

17. OTG-INFO-001
• To understand what sensitive design and
configuration information of the
application/system/organization is exposed both
directly (on the organization’s website) or
indirectly (on a third party website).

18. HOW TO TEST?
• Use a search engine to search for:
• Network diagrams and configurations
• Archived posts and emails by administrators and
other key staff
• Log on procedures and username formats
• Usernames and passwords
• Error message content
• Development, test, UAT and staging versions of the
website

19. Search Operators
gotta know ’em all
Ref: http://www.googleguide.com/advanced_operators_reference.html

20. Search Engines

21. • Baidu
• binsearch.info
• Bing
• Duck Duck Go
• ixquick/Startpage • Google
• Shodan
• PunkSpider
• Google
• Yahoo
• Yandexx
• etc.

22. Example
intitle:"index of"

23. Want to practice some
searching?
goto: http://www.agoogleaday.com/

24. Another Example

25. Lazy?

26. GHDB
The Google Hacking Database is list of useful search queries for Google.
Queries are put in several categories:
• Footholds
• Files containing usernames
• Sensitive Directories
• Web Server Detection
• Vulnerable Files
• Vulnerable Servers
• Error Messages
• Files containing juicy info
• Files containing passwords
• Sensitive Online Shopping Info

27. print(Remediation)
• Carefully consider the sensitivity of design and
configuration information before it is posted
online.
• Periodically review the sensitivity of existing
design and configuration information that is
posted online.

28. Fingerprint Web Server
(OTG-INFO-002)

29. OTG-INFO-002
• Knowing the version and type of a running web
server allows testers to determine known
vulnerabilities and the appropriate exploits to use
during testing.
• Find the version and type of a running web server
to determine known vulnerabilities and the
appropriate exploits to use during testing

30. How to Test
$ nc 202.41.76.251 80 HEAD / HTTP/1.0
HTTP/1.1 200 OK
Date: Mon, 16 Jun 2003 02:53:29 GMT
Server: Apache/1.3.3 (Unix) (Red Hat/Linux)
Last-Modified: Wed, 07 Oct 1998 11:18:14 GMT ETag: “1813-
49b-361b4df6”
Accept-Ranges: bytes
Content-Length: 1179
Connection: close
Content-Type: text/html

31. How about this?

32. 403 HTTP/1.1 Forbidden
Date: Mon, 16 Jun 2003 02:41: 27 GMT
Server: Unknown-Webserver/1.0
Connection: close
Content-Type: text/HTML; charset=iso-8859-1

33. Protocol Behaviour
• HTTP HEADER FIELD ORDERING
• MALFORMED REQUEST TEST

34. HTTP HEADER FIELD
ORDERING
• The first method consists of observing the
ordering of the several headers in the response.
Every web server has an inner ordering of the
header. Consider the following answers as an
example:

35. Comparison
Response from Apache 1.3.23
$ nc apache.example.com 80 HEAD
/ HTTP/1.0
HTTP/1.1 200 OK
Date: Sun, 15 Jun 2003 17:10: 49
GMT
Server: Apache/1.3.23
Last-Modified: Thu, 27 Feb 2003
03:48: 19 GMT ETag: 32417-c4-
3e5d8a83
Accept-Ranges: bytes
Content-Length: 196
Connection: close
Content-Type: text/HTML
Response from IIS 5.0
$ nc iis.example.com 80 HEAD /
HTTP/1.0
HTTP/1.1 200 OK
Server: Microsoft-IIS/5.0
Content-Location:
http:/iis.example.com/Default.htm
Date: Fri, 01 Jan 1999 20:13: 52
GMT
Content-Type: text/HTML
Accept-Ranges: bytes
Last-Modified: Fri, 01 Jan 1999
20:13: 52 GMT
ETag: W/e0d362a4c335be1: ae1
Content-Length: 133

36. Malformed requests test
• Another useful test to execute involves sending
malformed requests or requests of nonexistent
pages to the server. Consider the following HTTP
responses.

37. MALFORMED REQUEST
TEST
Response from Apache 1.3.23
$ nc apache.example.com 80 GET /
JUNK/1.0
HTTP/1.1 200 OK
Date: Sun, 15 Jun 2003 17:17: 47
GMT
Server: Apache/1.3.23
Last-Modified: Thu, 27 Feb 2003
03:48: 19 GMT ETag: 32417-c4-
3e5d8a83
Accept-Ranges: bytes
Content-Length: 196
Connection: close
Content-Type: text/HTML
Response from IIS 5.0
$ nc iis.example.com 80 GET /
JUNK/1.0
HTTP/1.1 400 Bad Request
Server: Microsoft-IIS/5.0
Date: Fri, 01 Jan 1999 20:14: 34
GMT Content-Type: text/HTML
Content-Length: 87

38. Can’t remember?

39. Automated Test
• HTTPrint (way too classics)
• Netcraft
• hmap
• nmap with ‘-sV’ options
• blindelephant (Version 2 is coming soon)
• Shodan
• etc.

40. print(Remediation)
• Custom compile Webserver (For #hardcores)
• Protect the presentation layer web server behind
a hardened reverse proxy.
• Obfuscate the presentation layer web server
headers.

41. Review Webserver Metafiles for
Information Leakage (OTG-INFO-
003)

42. OTG-INFO-003
• This section describes how to test the robots.txt file for
information leakage of the web application’s directory or folder
path(s).
• the list of directories that are to be avoided by Spiders,
Robots, or Crawlers can also be created as a dependency for
Map execution paths through application (OTG-INFO-007)
• Information leakage of the web application’s directory or folder
path(s).
• Create the list of directories that are to be avoided by Spiders,
Ro- bots, or Crawlers

43. Example of Robots.txt
killer-robot.txt
User-Agent: T-1000
User-Agent: T-800
Disallow: /+LarryPage
Disallow: /+SergeyBrin
Ref: http://www.google.com/killer-robots.txt

44. Fine, Real robots.txt
from http://www.google.com

45. What else?

46. Meta tag
• <META> tags are located within the HEAD section of each
HTML Document and should be consistent across a web site
in the likely event that the robot/spider/crawler start point does
not begin from a document link other than webroot i.e. a “deep
link”[5].
• Web spiders/robots/crawlers can intentionally ignore the
“<META NAME=”ROBOTS”” tag as the robots.txt file
convention is preferred. Hence, <META> Tags should not be
considered the primary mechanism, rather a complementary
control to robots.txt.

47. Meta Example
Example of Meta tags
Ref: https://anantshri.info/articles/web_app/image002.png

48. Enumerate Applications
on Webserver (OTG-
INFO-004)

49. OTG-INFO-004
A paramount step in testing for web application
vulnerabilities is to find out which particular
applications are hosted on a web server.
In addition, many applications are often
misconfigured or not updated, due to the perception
that they are only used “internally” and therefore no
threat exists.

50. TL;DR
• Enumerate the applications within scope that
exist on a web server
• Some server may have more than one
applications running

51. HOW TO TEST?
1. Different base URL
2. Non-standard ports
3. Virtual hosts

52. Base URL
• Take a deep breath, and
• keep typing in the url bar (address bar for the
retro)
• http://www.example.com/url1
• http://www.example.com/url2
• http://www.example.com/url3

53. not a lucky guy?

54. How to identify via base
URL?
• if mis-configured, and allow directory browsing,
we can find spot them.
• Dictionary searching (Forced Browsing in ZAP)
• Use Vulnerability scanner to help

55. Non-standard port
• Normally, HTTP, and HTTPS use TCP port 80,
443 respectively
• But, are you sure?

56. #nmap #FTW
Ref: https://nmap.org/movies/matrix/trinity-nmapscreen-
hd-crop-1200x728.jpg

57. Virtual Host

58. –The Three musketeers
“All for one, and one for all.”

59. $(whereis
my_neighbor_host)?
• DNS zone transfer (too classic)
• DNS inverse query (depends on target’s config)
• Robtex
• etc.

60. #Robtex #Example
Ref:
https://www.robtex.com/en/advisory/dns/org/icann/www/

61. Review webpage comments and
metadata
for information leakage (OTG-INFO-
005)

62. OTG-INFO-005
It is very common, and even recommended, for
programmers to include detailed comments and
metadata on their source code. However, comments and
metadata included into the HTML code might reveal
internal information that should not be available to
potential attackers. Comments and metadata review
should be done in order to determine if any information is
being leaked.

63. TL;DR again
• Just pay attention in any interesting Comment
and meta HTML tag

64. HOW TO TEST
• Check HTML Source code for any comment!
• Examine the Meta tag

65. #comment #CTF
Ref: http://ctf.infosecinstitute.com/levelone.php

66. Why meta tag again?
• some may provide the application profiling
• some alter HTTP response headers which can
used to conduct injection attacks

67. Identify application entry
points (OTG-INFO-006)

68. OTG-INFO-006
Enumerating the application and its attack surface
is a key precursor before any thorough testing can
be undertaken, as it allows the tester to identify
likely areas of weakness. This section aims to help
identify and map out areas within the application
that should be investigated once enumeration and
mapping have been completed.

69. TL;DR again
• Understand how requests are formed and typical
responses from the application

70. HOW TO TEST?
• Step 0: Understand HTTP Protocol
• Look into the Request and Response

71. Any tools to intercept?
• any intercepting proxy(Burp, Zed, fiddler, etc.)
• Tamper Data (Web Browser plugins)
• Live HTTP Header (Web Browser plugins)

72. Example
• GET example
GET
https:/x.x.x.x/shoppingApp/buyme
.asp?CUSTOMERID=100&ITEM
=z101a&PRICE=62.50&IP=x.x.x.
x
Host: x.x.x.x
Cookie:
SESSIONID=Z29vZCBqb2IgcGF
kYXdhIG15IHVzZXJuY-
W1lIGlzIGZvbyBhbmQgcGFzc3d
vcmQgaXMgYmFy
• POST Example
POST
https:/x.x.x.x/KevinNotSoGoodApp/authe
nticate.asp?- service=login
Host: x.x.x.x
Cookie:
SESSIONID=dGhpcyBpcyBhIGJhZCBhc
HAgdGhhdCB-
zZXRzIHByZWRpY3RhYmxlIGNvb2tpZX
MgYW5kIG1pbmUgaX- MgMTIzNA==
CustomCookie=00my00trusted00ip00is0
0x.x.x.x00
• Body of the POST message:
user=admin&pass=pass123&debug=true
&fromtrustIP=true =

73. Map execution paths
through application (OTG-
INFO-007)

74. James_Bond(){007}
Ref: http://1.bp.blogspot.com/-
pyBGbYAWLPI/UsWtagebozI/AAAAAAAAAvI/jAHHopVTrC4/s160
0/every-james-bond-ever.png

75. OTG-INFO-
$(James_Bond)
Before commencing security testing, understanding
the structure of the application is paramount.
Without a thorough understanding of the layout of
the application, it is unlkely that it will be tested
thoroughly.

76. Still want shorter
version?
• Map the target application and understand the
principal workflows.

77. HOW TO TEST?
There are several ways to approach the testing and
measurement of code coverage:
• Path - test each of the paths through an application
that includes combinatorial and boundary value
analysis testing for each decision path.
• Data flow (or taint analysis) - tests the assignment of
variables via external interaction (normally users).
• Race - tests multiple concurrent instances of the
application manipulating the same data.

78. We need support!
• OWASP-ZAP
• Burpsuite

79. #ZAP
Automatic Spidering

80. Fingerprint Web
Application Framework
(OTG-INFO-008)

81. OTG-INFO-008
Web framework or Content Management System
fingerprinting is an important subtask of the information
gathering process. Knowing the type of framework can
automatically give a great advantage if such a framework
has already been tested by the penetration tester. It is
not only the known vulnerabilities in unpatched versions
but specific misconfigurations in the framework and
known file structure that makes the fingerprinting
process so important.

82. Cut to the chase
To define type of used web framework so as to
have a better understanding of the security testing
methodology.

83. How? and Where?
• HTTP Headers
• Cookies
• HTML source code
• Specific files and folders

84. HTTP HEADER
HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT Content-
Type: text/html;charset=ISO-8859-1 Connection:
close
Vary: Accept-Encoding
X-Powered-By: Mono

85. Cookies
GET /cake HTTP /1.1
Host: defcon-moscow.org
User-Agent: Mozilla75.0 |Macintosh; Intel Mac OS X 10.7; rv: 22.0)
Gecko/20100101Firefox/22.0
Accept: text/html, application/xhtml + xml, application/xml; q=0.9, */*;
q=0 , 8
Accept-Language: ru-ru,ru;q=0.8,en-us;q=0.5,en;q=0.3 Accept -
Encoding: gzip, deflate
DNT: 1
Cookie: CAKEPHP=rm72kprivgmau5fmjdesbuqi71; Connection: Keep-
alive
Cache-Control: max-age=0

86. HTML source code
Find certain pattern in the

87. Specific Files and
Folders
For Example
• /wp-admin/
• modules/Drupal5-urdu-po/
• templates/joomlaport_metro/

88. Any assistance?
• Whatweb
• Blindelephant (Version 2 is coming soon)
• AppSpector
• Wappalyzer

89. #Wappalyzer

90. $(Remediation)
• obfuscate all HTTP-head- ers that disclose information the
technologies used
• change cookie names by making changes in the
corresponding configuration files.
• Manually check the contents of the HTML code and
remove every- thing that explicitly points to the framework.
• Remove Meta Generator Tags
• Do not use default scripts on the page or obfuscate them
if they must be used.

91. Fingerprint Web
Application
(OTG-INFO-009)

92. OTG-INFO-009
Apart from the Web Application Framework, Identify
the web application and version to determine known
vulnerabilities and the appropriate exploits to use
during testing.

93. Is it like 008?
• IMHO, similar to 008 but different objective
• 008 aims to draw what kind of framework they are
using (Name of the CMS)
• 009 aims to find which version of the web
application they are using (version of the CMS,
DB, Programming languge)

94. >show me the example
$(whatweb amazon.com && whatweb blognone.com

95. $(Remediation)
• Goto:OTG-INFO-008/remediation

96. Map Application
Architecture (OTG-INFO-
010)

97. OTG-INFO-010
The complexity of interconnected and heterogeneous web server infrastructure
can include hundreds of web applications and makes con- figuration
management and review a fundamental step in testing and deploying every
single application. In fact it takes only a single vulnerability to undermine the
security of the entire infrastructure, and even small and seemingly unimportant
problems may evolve into severe risks for another application on the same
server.
To address these problems, it is of utmost importance to perform an in-depth
review of configuration and known security issues. Before performing an in-
depth review it is necessary to map the network and application architecture.
The different elements that make up the infrastructure need to be determined to
understand how they interact with a web application and how they affect
security…….ZZzzzzzZZzz

98. TL;DR
• determine the architecture of the application to
find any impact against the application security

99. HOW?
• Ask the developers or system architecture
• navigating an application.
• Keep guessing by using the technique in 001-009
to determine the architectures.
• some strange input may returned the protection
systems e.g., %00

100. Why?
For example:
• On more complex setups, such as an online bank
system, multiple servers might be involved.
These may include a reverse proxy, a front-end
web server, an application server and a database
server or LDAP server.

101. Q&A