How to check if a domain is "alive" or "parked" from command line?

If some domain name becomes stale, it usually still replies to HTTP requests with some bunches of useless links, sometimes even without an explicit "This domain is expired/for sale" message instead of just failing to resolve at all.

How to automatically determine the "dead" (expired/not prolonged) domain without a browser? Can it be done with whois tool?

Expecting something like this:

while true; do
    if ! checkdomain something-on-verge-of-abandoning-business.com; then
        echo "Good night, sweet prince"
        break
    fi
    sleep 1d
done

For example, let's look at domain allmydata.com. As far as I see about Allmydata, there should be some Tahoe-LAFS-based backup service there. But actually it is parked domain.

$ whois allmydata.com
...
   Domain Name: ALLMYDATA.COM
   Registrar: DOMAIN MONKEYS, LLC
   Whois Server: whois.domainmonkeys.com
   Referral URL: http://www.domainmonkeys.com
   Name Server: NS1.DSREDIRECTION.COM
   Name Server: NS2.DSREDIRECTION.COM
   Status: clientTransferProhibited
   Updated Date: 04-aug-2013
   Creation Date: 03-aug-2004
   Expiration Date: 03-aug-2014
...

$ dig +short -t A allmydata.com @8.8.8.8
208.73.211.247

This shallow check shows like it is were a good domain. But how to reliably (i.e. not heuristically parsing the page and measuring "spammy-ness" or "parked-ness" from content) detect such thing?

Vi.

Posted 2013-12-04T14:12:57.573

Reputation: 13 705

linux I assume? – Journeyman Geek – 2013-12-05T11:00:05.557

For example, GNU/Linux. But I expect the approach to be portable. – Vi. – 2013-12-05T16:45:38.460

Answers

You could just use python to read in the html content and then search for "domain available"/"parked"/"renewal" etc and other keywords. You can feed it a CSV file of domains and then output the results as CSV and there you have a list of domains.

The other idea would be to parse whois records using something like this and parse the results for the renewal date. That's how I'd do it.

Mud

Posted 2013-12-04T14:12:57.573

Reputation: 65

This looks like a workaroundish "heuristic" method... What is the essential technical difference between normal and "dead" domain? – Vi. – 2013-12-05T12:13:46.630

Well a dead domain will have no valid nameserver or A records - you could just dig for those and use that as a test/fail thing, otherwise I'm out of ideas. Personally I would just query whois records and parse the results for the renewal date - any date that <= today would be a dead domain – Mud – 2013-12-05T13:46:18.600

Appended a domain example that satisfies "expire date > today", but yet still a parked domain. – Vi. – 2013-12-05T16:46:28.307

Well this wouldn't show parked domains because a parked domain is just a domain with a holding page on it. To check for a holding page you would have to read in the source code of the site and search for obvious strings like "holding page" "parking" "domain" and maybe grab a list of the top 200 hosts and iterate through that.

My solution above is only for domains that have expired or are close to expiring – Mud – 2013-12-06T15:10:40.043