47

One of the things I need to do from time to time is to find subdomains of a site for example.

Starting with example.com

  • sub1.example.com
  • other.example.com
  • another.example.com

I'm looking for any additional ways to perform recon on these targets and I want to get a list of all the subdomains of a domain.

I'm currently doing a number of things inlcuding

  • using maltego to crawl for info
  • Using search engines to search for subdomains
  • crawling site links
  • Examining DNS records
  • Examining incorrectly configured SSL certificates
  • Guessing things like 'vpn.example.com'

I reckon there are more than the ones i've found so far, but now I'm out of ideas.

Luc
  • 31,973
  • 8
  • 71
  • 135
NULLZ
  • 11,426
  • 17
  • 77
  • 111
  • 1
    There is another post on stackoverflow that's quite good: [List of Subdomains][1] [1]: http://stackoverflow.com/questions/131989/how-do-i-get-a-list-of-all-subdomains-of-a-domain – Dr.Ü Apr 30 '13 at 06:58
  • Then there is only one way - do it like maltego: make educated guesses... – Dr.Ü Apr 30 '13 at 07:12
  • 2
    I would try it with knock (http://code.google.com/p/knock/) but watch out: there is a risk of being blacklisted. – Dr.Ü Apr 30 '13 at 07:19
  • 1
    There's a python script called subdomainer.py that should be able to help you out... Have a search on google – AndyMac Apr 30 '13 at 07:38
  • 1
    FYI it can be found on the [edge-security.com website](http://www.edge-security.com/soft/subdomainer.py), but the old link posted on [SecurityTube](http://www.securitytube-tools.net/index.php@title=Subdomainer.html) wiki page is dead (albeit it does have usage example which is of course defined in `usage()` anyway). Judging by source code, what it does is it collates data from three major search engines (yahoo, msn, google) and to me obscure website `pgp.rediris.es` that seems to be an email scrapper. – TildalWave Apr 30 '13 at 07:57
  • You could even ask [google](http://google.com/?q=site:stackexchange.com)! But this won't be a complete list! – F. Hauri - Give Up GitHub Oct 13 '14 at 08:11

8 Answers8

34

As a pentester being able to find the subdomains for a site comes up often. So I wrote a tool, SubBrute that does this quite well if I do say so my self. In short, this is better than other tools (fierce2) in that its a lot faster, more accurate and easier to work with. This tool comes with a list of real subdomains obtained from spidering the web. This subdomain list is more than 16 times the size of fierce2 and subbrute will take about 15 minutes to exhaust this list on a home connection. The output is a clean newline separated list, that is easy to use as the input for other tools like nmap or a web application vulnerability scanner.

rook
  • 46,916
  • 10
  • 92
  • 181
  • Awesome, i'll check it out. Any idea how well it compares to 'knock'? – NULLZ May 05 '13 at 06:00
  • @D3C4FF idk i haven't used knock, i'll have to check that out. I expect that the subdomain brute force feature is better than knock. – rook May 05 '13 at 15:22
  • 1
    @D3C4FF knock is crap. – rook May 05 '13 at 19:32
  • @Rook but knock have the ability to try domaintransfers (even you need some luck to get one) – Dr.Ü May 06 '13 at 07:37
  • @Dr.Ü yes I will add that simple feature. But as you said, it doesn't work all of the time. As a note subbrute has more code, and is more complex without the addition of domain transfers. – rook May 06 '13 at 23:16
  • 2
    All glory be to Rook. I used the tool in a live test today and it worked like a charm. It only missed one out of two dozen sub domains which was named *mywebreading* – NULLZ May 09 '13 at 06:46
  • 1
    @D3C4FF hell yeah, I'm glad it did the trick ;) – rook May 09 '13 at 06:52
  • 2
    @Rook just a quick idea regarding subbrute, add the option to resolve (and print out) the IP address from the found hostnames as well. I made that change and it helped with a few tests where certain ranges were out of scope even though they were subdomains. Thanks again! – NULLZ Nov 26 '13 at 22:26
9

1. Zone transfer

Some nameservers allow for DNS zone transfers to anyone on the internet, usually unintentionally. In this question, it is explained further: DNS zone transfer attack.

Tools for zone transfers

The second answer on that question mentions how to test for it for both Windows and Linux:

Windows:

nslookup
> server <DNS you are querying>
> set type=any
> ls -d <target>

Unix (nslookup is deprecated on Unix):

dig -t axfr @<DNS you are querying> <target>

(I edited the Unix one, because -axfr does not appear to work. I needed to specify -t axfr.)

2. DNSSEC zone walk

DNSSEC signs DNS records, so you can be sure you receive the correct answer (well, given some trust roots and intermediaries). But how do you prove that something does not exist, e.g. when looking for nonexistentsub.example.com, how does the nameserver of example.com prove nonexistence of a subdomain? It doesn't have the signing key on it (signing is done upon updating the zone by administrators).

In their answer to How does DNSSec work? Are there known limitations or issues?, /u/tylerl explains:

Obviously that response needs to be signed, but generally the DNS server itself doesn't have access to your signing key and can't sign the response on-the-fly; the signatures are all created "offline" ahead of time. This keeps your key from being exposed if your DNS server gets compromised.

So instead, you alphabetize your subdomains and say "for every name between mail.example.com and pop.example.com, no other subdomains exist" and sign that assertion. Then when someone asks for nachos.example.com you can just give them that response (which has already been signed) and the client knows that because nachos.example.com falls alphabetically between mail.example.com and pop.example.com, then the "this domain doesn't exist" response is considered to be correctly signed and actually came from you.

The place where this becomes problematic is that by having a set of these negative responses which explicitly state that "no responses exist between X and Y, you can easily map out exactly which domains exist for the entire zone. You know that "X" exists, and you know that "Y" exists, and you know there is nothing else between them. Just do a little more poking at random and you'll quickly be able to compile a list of all the records that do exist.

The record that specifies "until pop.example.com there is nothing" is called NSEC (Next SECure record).

A workaround was designed for this: NSEC3. It hashes names, so mail turns into b83a88... and pop turns into b21afc.... Imagine those are the two only subdomains, then the signed response will say "no record exists between b21afc... and b83a88...". Again it works alphabetically and you can obtain them all, but this time you will need to crack each hash before you learn what the subdomains are.

In my experience, most have the NSEC3 extension enabled.

Tools for zone walking

NSEC3Walker does both the enumeration and the cracking. I cannot vouch for how efficient the cracking is, but it's definitely only CPU-based. Since NSEC3 uses SHA1 (at least originally), there are probably better cracking programs.

dnsrecon also appears to be able to do it: dnsrecon -z -d example.com. I don't know if there is an official website with information, but in Debian Stretch, Buster, and Bullseye I can apt install dnsrecon.

3. Reverse lookups in a subnet

By guessing a few, you will often find responses in a similar range. If you know www. exists and mail. exists, and they both resolve to 192.168.3.x, there might be more. Try to do a reverse lookup for all addresses in the 192.168.3.0-255 range (the /24), and you will probably find more subdomains. You may also want to try a WHOIS query on the IP address to find the range's boundaries (if they have their own block).

Tools for reverse lookups

dnsrecon can do this:

dnsrecon -t rvl -r 192.168.1.0/24

Where -t rvl means "type reverse-lookup" and -r passes an IP range in CIDR notation. I don't know if there is an official website with information, but in Debian Stretch, Buster, and Bullseye I can apt install dnsrecon.

4. DNS service records

One can set SRV (service) records for service discovery, for example _sip._tcp.example.com could point to sipserver.example.com on port 5060. Since the service names ("sip" in the example) are typically the standard ones registered with IANA, we can iterate through them.

Tools for querying srv records

dnsrecon can do this:

dnsrecon -t srv -d example.com

It will take a subset of the existing service names, selected by an unknown method, as mentioned in its man page (man dnsrecon).

5. Other methods

You already mentioned some of those. I won't go into detail, because they're quite self-explanatory, and either depend on an application running on the target (such as FTP), depend on a third party, or there is really just not much to say about them.

  • Certificate transparency logs may show for which subdomains certificates were obtained, e.g. see https://crt.sh.

  • Search results might reveal subdomains. Again, dnsrecon can do this with the -t goo option (uses Google specifically).

  • Checking other TLDs for the same name might reveal some other variants or IP addresses. E.g. if example.com exists, example.org might exist as well. dnsrecon can also do this with dnsrecon -t tld -d example.com.

  • Crawling a website or finding references elsewhere might give hints. (Help wanted: which tool to use?)

  • Looking at TLS certificates often yields results. Be sure to check the ports for HTTPS, SMTP(S), FTP(S), etc. and use STARTTLS.

  • There are third party tools which can list subdomains in a domain. Their methods are less clear, but crawling the internet and historical records (maybe a domain transfer was once possible?) are often part of it. (Help wanted: any recommendations? I only remember seeing that it exists.)

6. Guessing.

The last option is just guessing, either by a dictionary (I'd recommend that) or brute force. This is made harder by wildcards, though many tools will try to detect and solve this.

Tools for guessing/brute-forcing

Fierce was built to do this: https://github.com/mschwager/fierce
It is installed by default in Kali Linux.

As /u/rook mentioned in another answer in this thread, they wrote Subbrute for this purpose: https://github.com/TheRook/subbrute

dnsrecon can do this with dnsrecon -t brt -d example.com. Use -f to "Filter out of Brute Force Domain lookup records that resolve to the wildcard defined IP Address when saving records" (citing its man page). You can pass -D for a dictionary file.

Appendix: dictionaries

I am still looking for good dictionaries (for guessing/brute forcing), but here are some that I'm aware of. Please help me complete this list! The bigger the better, as long as they are sorted by likelihood.

Luc
  • 31,973
  • 8
  • 71
  • 135
3

I would try it with knock but watch out: there is a risk of being blacklisted.

Unfortunately is there no way around bruteforcing if a zone transfer doesn't work.

Hartley Brody
  • 103
  • 1
  • 1
  • 5
Dr.Ü
  • 1,029
  • 8
  • 16
3

Jason Haddix wrote my favorite subdomain/hostname discovery tool that depends on a very-recent version of recon-ng -- available here -- https://github.com/jhaddix/domain

subbrute is decent, fierce -dns <domain> works great, dnsmap <domain> -r file.txt is also valid, and I don't see any reason to dislike knock -wc <domain> (although the other features of knock may be suspect). All of these tools use techniques that are showing their age, however. The trick for some of these attack improvements is to come up with a customized file with hostnames that are geared specifically for the target.

However, the chainsaw for DNS discovery is dnsrecon. It does everything.

You might also consider a commercial offering, such as RiskIQ, which can do quite a lot more than all of these tools. Their techniques include a lot of surveying that most of you would not think of.

[UPDATE] Another favorite (for hostnames, not primarily subdomains -- is the OP interested in both?) is -- https://github.com/tomsteele/blacksheepwall

atdre
  • 18,885
  • 6
  • 58
  • 107
1

Over on Stack Overflow, Paul Melici suggested using WolframAlpha. (Screenshots by myself)

  1. Enter the domain into the search box and run the search. (E.g. stackexchange.com)

    Wolfram - Homepage

  2. In the 3rd section from the top (named "Web statistics for all of stackexchange.com") click Subdomains

    Wolfram - Subdomains button

  3. In the Subdomains section click More

    Wolfram - More subdomains button

You will be able to see a list of sub-domains there. Although I suspect it does not show ALL sub-domains.

Stevoisiak
  • 1,515
  • 1
  • 11
  • 27
0

Initially, I often use passive dns database to find subdomains for a site. The drawback with this method, is that you can only find sites that are listed in the database. But the benefit is that you can find sites that are not listed in any wordlists.

Some databases for reference:

https://www.virustotal.com/en/domain/stackexchange.com/information/

http://www.nonexiste.net/?q=stackexchange.com

Dog eat cat world
  • 5,759
  • 1
  • 27
  • 46
0

Take a look at http://ha.ckers.org/fierce/. It can take a dictionary file as well as brute force. It is included in Kali as well.

void_in
  • 5,541
  • 1
  • 20
  • 28
-1

Easy. Write site address to wolframalpha. And click "subdomins" button.

An Example; https://www.wolframalpha.com/input/?i=www.imdb.com

Click "Subdomains" for view subdomains of the site.

Also, wolfram alpha has api if you want to reach and use it. Hope it helps...