WHOIS the Master
•
0 likes•414 views
Presents a WHOIS database search engine tool I wrote to allow pentesters to access network information for specified targets. First presented at BSidesDE 2010
More Related Content
Similar to WHOIS the Master
Similar to WHOIS the Master (20)
More from Jason Ross
Recently uploaded
Recently uploaded (20)
WHOIS the Master
- 1. WHOIS the master an introduction to Sho'Nuff jason ross
- 2. about me • job: break stuff for the intrepidus group • play: with malware • poorly manage defcon group 585 • refuse to use caps in slide decks (acronyms excluded)
- 3. agenda • 2^32 addresses ought to be enough for anybody • alphabet soup, iron fists, and ipv6 • whois: awesomely full of crap • shonuff – the whois master
- 4. a (very) brief history of 'the internet' • lots of separate networks hooked up, some confusion ensued • InterNIC stepped out, ICANN stepped in • ICANN manages global addressing under contract to US Dept. of Commerce as IANA • (not for) profit!
- 5. ipv4 network allocation • large blocks of addresses are allocated to global geographic regions • large blocks may be allocated to national geographic regions • blocks are divided up and allocated to local ISPs • individual addresses or small blocks are assigned to ISP customers
- 6. early allocation methods • there's so much space! • large chunks of network space allocated to single organizations • justification requirements fairly lax
- 7. zomg! this thing works! • demand increased • address assignments got smaller • requirements to prove need of requested space got tighter
- 8. what's a RIR? • Regional Internet Registry • in charge of large geographic regions – AfriNIC : Africa – APNIC : Asia / Pacific – ARIN : North America – LACNIC : Latin America & some Caribbean – RIPE NCC : Europe, Middle East, Central Asia
- 9. what's a NIR? • National Internet Registry • in charge of small geographic regions • act as an agent of the RIR • not commonly used, but there's a few
- 10. what's a LIR? • Local Internet Registry • usually an ISP
- 11. why the push for ipv6? • ipv4 was not designed for security • "available address space is running low"
- 12. security • many con talks and whitepapers by folks lots smarter that i have already covered this • so i won't
- 13. scarcity • there have been comments and discussion around the fact that IPv4 space is 'running out' for years. • IEEE-USA published a report on this in 8/1999
- 14. the sky is falling! (aka: how low can you go?) image taken from arstechnica: http://is.gd/dCnMM
- 15. if ipv4 is running out, where did it go? • nobody that knows is telling ('freely') • nobody else knows • leading to much debate
- 16. how to find out • ask IANA! • when that fails, ask the RIRs • then ask the LIRs
- 17. overview of whois tools • *nix: whois • web: http://lmgtfy.com/?q=web+whois • www.robtex.com/whois
- 18. what's missing? • no standardized output • can't perform true wildcard queries – whois -h whois.arin.net " o . bank*" • query options vary by RIR • information is not centralized – chasing referrals sucks
- 19. how accurate is whois data? • contact data is required by law in most countries to be legit • ARIN is working on a policy to validate WHOIS POC info
- 20. theoretical challenges • most efficient way to scan • how to handle referrals • should i throttle queries • parsing the results
- 21. shonuff – the WHOIS master! • started as PHP/MySQL • then i got mocked (gently) • so i ported it to JSP/Postgres – to prove it can always get worse • is now written in ruby!
- 22. what’s new? • better integration with shodan • privacy policy • more query types supported
- 23. linking results to shodan • shodan has an API! • so i just make calls to it for you – many thanks to achillean, for letting this work!
- 24. interesting reports • organizational breakdown – who has the most allocations – who has the most network space • geographic breakdown – what countries have ip space – which countries have the most space
- 25. Demo!
- 26. future plans • add in WHOIS contact data • malware IP to WHOIS correlation – allows easy tie-back of malicious content to "real world" network & hosting businesses • integrate DNS records for netblocks • Maltego transform? • Tie-in for Fierce? • Metasploit fun?
Editor's Notes
- Because IP is 32 bit, there are about 4.2 billion potential IP addresses (2^32 = 4,294,967,296). Early thought was that we'd never run out. Now there are claims that we are running out (and there have been since late 1999 / early 2000) The address space is managed by a conglomeration of organizations, each with fun acronyms for names. Information about who the space has been assigned to is viewed as propietary, and confidential, information – despite the fact that it is publicly available in the form of WHOIS. WHOIS is useful, but has some shortcomings. Most specifically, it's difficult to query based on descriptive text, or obtain a list of "all networks that are associated with entity X". So… I wrote a tool to allow those things.
- The history of "the internet" as we know it today is a very convoluted (and in many cases disputed) tale. Accordingly, i'm ignoring it entirely, and simply summarizing it as "lots of private networks came together using a suite of protocols tested by the US DOD research arm known as ARPA. Things got crazy for a bit, but everyone saw it was useful, so eventually a non profit organization was created to help things out and manage the address and name space of the network going forward." That organization is ICANN (Internet Corporation for Assigned Names and Numbers), and they oversee the global address and namespace by managing a subsidiary organization called the Internet Assigned Numbers Authority (IANA) under contract to the US Dept. of Commerce. In addition to global IP space assignment, IANA controls the Autonomous System (AS) Number assignments, and documents IP port number assignments made by the IETF.
- In general, IANA doesn't provide IP addresses to the public. It provides large blocks of space to various NICs (Network Information Centers) so that they can then be parceled out and given to ISPs in smaller chunks in a hierarchical manner. The typical process is: * IANA assigns large blocks of addresses to Regional Internet Registries (RIR) * The RIR assigns blocks of addresses from their IANA assigned pool to National Internet Registries (NIR) or Local Internet Registries (LIR). * The RIR then assigns addresses to specific ISPs, which are then further divided into small pools for specific end user needs.
- As the use of the internet increased, address assignments got more specific to address concerns about scarcity. Additionally, requirements for obtaining blocks of network space from RIRs grew more stringent. The current ARIN guidelines for IP address allocation for private organizations (end-users…eg. not an ISP) is: minimum allocation: /20 (4,096 addresses) A 25% immediate utilization rate, and A 50% utilization rate within one year. https://www.arin.net/policy/nrpm.html#four
- A RIR assigns network space to LIRs or NIRs, not to end users directly.
- NIRs operate as an agent to the RIR which oversees them. they allocate the network space assigned to the the RIR and assign it to LIRs within their geographic region of responsibility. the NIR is not directly assigned network addresses as a RIR is, it simply manages the assignment of addresses for a specific area.
- LIRs are assigned small blocks of network space, which they further divide as needed to meet the operational needs of their end users.
- IEEE paper on this subject can be found at: http://www.ieeeusa.org/volunteers/committees/ccp/documents/IPv6FinalwhitepaperFinalAugust2009.pdf
- Image taken from an article on ars technica about the scarcity of ipv4 space.
- ARIN does provide a bulk database download option. "to support the work of bona fide academic researchers, and to operators and researchers who are using the data to provide a clear benefit to the broader networking community. ARIN does not provide bulk copies of Whois data to operators who wish to incorporate this data into products, services, or internal systems with no clear benefit to the broader community." To qualify: Step 1: Complete Request Form Step 2: Review Process "Upon receipt of your signed AUP, ARIN staff will begin a thorough review of your request by contacting you via e-mail using the e-mail address specified in the signature section of the Bulk Whois Request Form. ARIN will ask you a set of questions aimed at understanding why you require access to a bulk copy of the data." More information on that can be found at https://www.arin.net/resources/request/bulkwhois.html
- IANA maintains a list of top level block assignments at http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xml This file is useful for gaining a very general overview of where IP space has been assigned. Most of the entries on the list are RIRs however, making it useless for telling where the address space is actually being used. So, to get more meaningful information, there's a need to go down at least one level, and ask the RIRs
- On most unix based systems, there's a utility called whois which can be used to perform queries of domain names and/or IP addresses. In addition, robtex.com maintains a web based interface to whois, and a lot of other useful networking tools.
- The whois syntax above comes close to performing a wild card query, but ARIN only supports wildcard at the end of the string.
- Because of how network space is allocated and assigned, often the WHOIS data obtained from the RIR points to an NSP, which places a referral to the WHOIS server run by the LIR that space has been allocated to. If you then query the server in the referral, you can obtain more specific data about the customer the NSP has assigned the particular block to I decided to break parsing down into multiple stages: obtain all top level WHOIS information parse for referrals, then process them in a second stage continue until all referrals have been processed