Categories
Admin DNS Network Technologies Security

The IT Detective agency: Live hack caught, partially stopped

Intro
In my years at cybersecurity I’ve been sufficiently removed from the action that I’ve rarely been involved in an actual case. Until last night. A friend, whom I’ll call Jute, got a formal complaint about one of his hosted Windows servers.

We have detected multiple hacking attempts from your ip address 47.5.105.236 (Hilfer Online) to access our systems.
>
> Log of attempts:
> – Hack attempt failed at 2019-01-17T14:22:41.6539784Z. Attempted user name: Not specified (typical for port scanners or denial of service attacks), system accessed: RDP, ip address accessed: 158.69.241.92
> – Hack attempt failed at 2019-01-17T14:22:26.2213808Z. Attempted user name: Not specified (typical for port scanners or denial of service attacks), system accessed: RDP, ip address accessed: 158.69.241.92
> – Hack attempt failed at 2019-01-17T14:22:10.6304194Z. Attempted user name: Not specified (typical for port scanners or denial of service attacks), system accessed: RDP, ip address accessed: 158.69.241.92
>
>
> Please investigate this problem.
>
> Sent using IP Ban Pro
> http://ipban.com

Hack, cont.
I’ve changed the IPs to protect the guilty! But I’ve conveyed the specificity of the error reporting. Nice and detailed.

Jute has a Windows Server 2012 at that IP. He is not running a web server, so that conveniently and dramatically narrows the hackable footprint of his server. I ran a port scan and found ports 135, 139 and 3389 open. His provider (which is not AWS) offers a simple firewall which I suggested we use to block ports 135 and 139 which are for Microsoft stuff. He was running it as a local sever so I don’t tihnk he needed it.

Bright idea: use good ole netstat
The breakthrough came when I showed him the poor man’s packet trace:

netstat -an

from a CMD prompt. He ran that and I saw not one but two RDP connections. One we easily identified as his, but the other? It was coming form another IP belonging to the same provider! RDP is easily identified by just looking for the port 3389 connections. Clearly we had caught first-hand an unauthorized user.

I suggested a firewall rule to allow only his Verizon range to connect to server on port 3389. But, I am an enterprise guy, used to stateful firewalls. When we set it up we cut off his RDP session to his own server! Why? I quickly concluded this was amateur hour and a primitive, ip-chains-like stateless firewall. So we have to think about rules for each packet, not for each tcp connection.

Once we put in a rule to block access to ports 135 and 139, we also blocked Jute’s own RP session. So the instructions said once you use the firewall, an implicit DENY ALL is added to the bottom of the rules.

So we needed to add a rule like:

SRC: ANY DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: 3389 ACTION: allow

But his server needs to access web sites. That’s a touch difficult with a stateless firewall. You have to enter the “backwards” rule (outbound traffic is not restricted by firewall):

SRC: ANY DST: 47.5.105.236 SRC_PORT: 443 DST_PORT: ANY ACTION: allow

But he also needs to send smtp email, and look up DNS! This is getting messy, but we can do it:

SRC: ANY DST: 47.5.105.236 SRC_PORT: 25 DST_PORT: ANY ACTION: allow
SRC: ANY DST: 47.5.105.236 SRC_PORT: 53 DST_PORT: ANY ACTION: allow

We looked up the users and saw Administrator and another user Update. We did not recognize Update so he deleted it! And changed the password to Administrator.

Finally we decided we had to bump this hacker.

So we made two rules to allow him but deny the zombie computer:

SRC: 158.69.240.92 DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: 3389 ACTION: reject
SRC: ANY DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: 3389 ACTION: allow

Pyrrhic victory
Success. We bumped that user right out while permitting Jute’s access to continue. The bad news? A new interloper replaced it! 95.216.86.217.

OK. So with another rule we can bump that one too.

Yup. Another success. another interloper jumps on in its place. 124.153.74.29. So we bump that one. But I begin to suspect we are bailing the Titanic with a thimble. It’s amazing. Within seconds a blocked IP is being replaced with a new one.

We need a more sweeping restriction. So we reasoned that Jute will RP from his provider where his IP does not really change.

So we replace

SRC: ANY DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: 3389 ACTION: allow

with

SRC: 175.198.0.0/16 DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: 3389 ACTION: allow

and we also delete the specific reject rules.

But now at this point for some reason the implicit DENY ALL rule stops working. From my server I could do an nc -v 47.5.105.236 3389 and see that that port was open, though it should ont have been. So we have to add a cleanup rule at the bottom:

SRC: ANY DST: 47.5.105.236 SRC_PORT: ANY DST_PORT: ANY ACTION: reject

That did the trick. Port no longer opened.

There still appears in netstat -an listing the last interloper, but I think it just hasn’t been timed out yet. netstat -an also clearly shows (to me anyway) what they were doing: scanning large swaths of the Internet for other vulnerable servers! The tables were filled with SYN-SENT to port 3389 of consecutive IPs! Amazing.

So I think Jute’s server was turned into a zombie which was tasked with recruiting new zombies.

We had finally frozen them out.

Later that night
Late that same night he calls me in a panic. He uses a bunch of downstream servers and that wasn’t working! The downstream servers run on a range of ports 14800 – 15200.

Now bear in mind the provider only permits us 10 firewall rules, so it’s getting kind of tight. But we manage to squeeze in another rule:

SRC: ANY DST: 47.5.105.236 SRC_PORT: 14800-15200 DST_PORT: ANY ACTION: allow

He breathes a sigh of relief because this works! But I want him we are opening a slight hole now. Short term there’s nothing we can do. It’s a small exposure: 400 open ports out of 65000 possible. It should hold him for awile with any luck.

He also tried to apply all updates at my suggestion. I’m still not sure what vulnerability was (is) exploited.

Case: tentaively closed

Our first attempt to use the Windows firewall itself was not initially successful. We may return to it.

Conclusion
We catch a zombie computer totally exploiting RDP on a Windows 2012 server. We knocked it off and it was immediately repaced with another zombie doing the same thing. Their task was to find more zombies to join to the botnet. Inbound firewall rules defined on a stateless firewall were identified which stopped this exploitation while permitting desired traffic. Not so easy when you are limited to 10 firewall rules!

This is a case where IPBAN did us a favor. The system worked as it was supposed to. We got the alert, and acted on it immediately.

I’m not 100% sure which RP vulnerability was exploited. It may have been an RCE – remote code execution not even requiring a valid logon.

References and related
The rest of the security world finally caught up with this, with Microsoft releasing a critical patch in May. I believe I was one of the first to publicly document this exploit. https://nakedsecurity.sophos.com/2019/06/10/the-goldbrute-botnet-is-trying-to-crack-open-1-5-million-rdp-servers/?utm_source=Naked+Security+-+Sophos+List&utm_campaign=0fd82f7fce-Naked+Security+-+June+test+-+groups+1+and+3&utm_medium=email&utm_term=0_31623bb782-0fd82f7fce-418487137

Categories
DNS

The IT detective agency: rogue IPv6 device messes up DHCP for entire subnet

Intro
This was a fascinating case insofar as it was my first encounter with a real life IPv6 application. So it was trial by fire.

The details
I think the title of the post makes clear what happened. The site people were saying they can ping hosts by IP but not by DNS name. So basically nothing was working. I asked them to do an ipconfig /all and send me the output. At the top of the list of DNS servers was this funny entry:

IPv6 DNS server shows up first

I asked them to run nslookup, and sure enough, it timed out trying to talk to that same IPv6 server. Yet they could PING it.

The DNS servers listed below the IPv6 one were the expected IPv4 our enterprise system hands out.

My quick conclusion: there is a rogue host on their subnet acting as an IPv6 DHCP server! It took some convincing on my part before they got on board with that idea.

But I goofed too. In my haste to move on, I confused an IPv6 address with a MAC address. Rookie IPv6 mistake I suppose. It looked strange, had letters and even colons, so it kind of looks like a MAC address, right? So I gave some quick advice to get rid of the problem: identify this address on the switch, find its port and disable it. So the guy looked for this funny MAC address and of course didn’t find it or anything that looked like it.

My general idea was right – there was a rogue IPv6 DHCP server.

My hypothesis as to what happened
The PCs have both an IPv4 as well as an IPv6 stack, as does just about everyone’s PC. These stacks run independently of each other. Everyone blissfully ignores the IPv6 communication, but that doesn’t mean it’s not occurring. I think these PCs got an IPv4 IP and DNS servers assigned to them in the usual way. All good. Then along came a DHCPv6 server and the PC’s IPv6 stack sent out a DHCPv6 request to the entire subnet (which it probably is doing periodically all along, there just was never a DHCPv6 server answering before this). This time the DHCPv6 server answered and gave out some IPv6-relevant information, including a IPv6 DNS server.

I further hypothesize that what I said above about the IPv4 and IPv6 stacks being independent is not entirely true. These stacks are joined in one place: the resolving nameservers. You only get one set of resolving namesevrers for your combined IPv4/IPv6 stacks, which sort of makes sense because DNS servers can answer queries about IPv6 objects if they are so configured. So, anyway, the DHCPv6 client decides to put the DNS server it has learned about from its DHCPv6 server at the front of the existing nameserver list. This nameserver is totally busted, however and sits on the request and the client’s error handling isn’t good enough to detect the problem and move on to the next nameserver in the list – an IPv4 nameserver which would have worked just great – despite the fact that it is designed to do just that. And all resolution breaks and breaks badly.

What was the offending device? They’re not saying, except we heard it was a router, hence, a host introduced by the LAN vendor who can’t or won’t admit to having made such an error, instead making a quiet correction. Quiet because of course they initially refused the incident and had us look elsewhere for the source of the “DHCP problem.”

Alternate theory
I see that IPv6 devices do not need to get DNS servers via DHCPv6. They can use a new protocol, NDP, neighbor discovery protocol. Maybe the IPv6 stack is periodically trying NDP and finally got a response from the rogue device and put that first on the list of nameservers. No DHCPv6 really used in that scenario, just NDP.

Useful tips for layer 2 stuff
Here’s how you can find the MAC of an IPv6 device which you have just PINGed:

netsh interface ipv6 show neighbors

from a CMD prompt on a Windows machine.

In Linux it’s

ip ‐6 neigh show

Conclusion
Another tough case resolved! We learn some valuable things about IPv6 in the process.

References and related
I found the relevant commands in this article: https://www.midnightfreddie.com/how-to-arp-a-in-ipv6.html

Categories
DNS Linux Network Technologies Raspberry Pi Security

Whois information without the pushy hard sell tactics

Intro
Did you ever want to learn about a domain registration but were put off by the hard sell tactics that basically all web-based whois searches subject you to? Me, too. Here’s what you can do.

The details
Linux – so that includes you, Raspberry Pi owners – has a little utility called whois which you can use to get the registrant information of a domain, e.g.,

$ whois johnstechtalk.com

   Domain Name: JOHNSTECHTALK.COM
   Registry Domain ID: 1795918838_DOMAIN_COM-VRSN
   Registrar WHOIS Server: whois.godaddy.com
   Registrar URL: http://www.godaddy.com
   Updated Date: 2017-03-27T00:52:51Z
   Creation Date: 2013-04-23T00:54:17Z
   Registry Expiry Date: 2019-04-23T00:54:17Z
   Registrar: GoDaddy.com, LLC
   Registrar IANA ID: 146
   Registrar Abuse Contact Email: abuse@godaddy.com
   Registrar Abuse Contact Phone: 480-624-2505
   Domain Status: clientDeleteProhibited https://icann.org/epp#clientDeleteProhibited
   Domain Status: clientRenewProhibited https://icann.org/epp#clientRenewProhibited
   Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
   Domain Status: clientUpdateProhibited https://icann.org/epp#clientUpdateProhibited
   Name Server: NS45.DOMAINCONTROL.COM
   Name Server: NS46.DOMAINCONTROL.COM
   DNSSEC: unsigned
   URL of the ICANN Whois Inaccuracy Complaint Form: https://www.icann.org/wicf/
>>> Last update of whois database: 2018-04-19T19:59:35Z <<<
...

Admittedly that did not tell us much, but it points us to another whois server we can try, whois.godaddy.com. So try that:

$ whois ‐h whois.godaddy.com johnstechtalk.com

Domain Name: JOHNSTECHTALK.COM
Registry Domain ID: 1795918838_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.godaddy.com
Registrar URL: http://www.godaddy.com
Updated Date: 2017-03-27T00:52:50Z
Creation Date: 2013-04-23T00:54:17Z
Registrar Registration Expiration Date: 2019-04-23T00:54:17Z
Registrar: GoDaddy.com, LLC
Registrar IANA ID: 146
Registrar Abuse Contact Email: abuse@godaddy.com
Registrar Abuse Contact Phone: +1.4806242505
Domain Status: clientTransferProhibited http://www.icann.org/epp#clientTransferProhibited
Domain Status: clientUpdateProhibited http://www.icann.org/epp#clientUpdateProhibited
Domain Status: clientRenewProhibited http://www.icann.org/epp#clientRenewProhibited
Domain Status: clientDeleteProhibited http://www.icann.org/epp#clientDeleteProhibited
Registry Registrant ID: Not Available From Registry
Registrant Name: ******** ******** (see Notes section below on how to view unmasked data)
Registrant Organization:
Registrant Street: ***** ****
Registrant City: Newton
Registrant State/Province: New Jersey
Registrant Postal Code: 078**
Registrant Country: US
Registrant Phone: +*.**********
Registrant Phone Ext:
Registrant Fax:
Registrant Fax Ext:
Registrant Email: ********@*****.***
Registry Admin ID: Not Available From Registry
Admin Name: ******** ******** (see Notes section below on how to view unmasked data)
...

So now we’re getting somewhere. So GoDaddy tries to force you to their web page an sell you stuff in any case. Not at all surprising for anyone who’s ever been a GoDaddy customer (includes yours truly). Because that’s what they do. But not all registrars do that.

Here’s a real-life example which made me decide this technique should be more broadly disseminated. I searched for information on a domain in Argentina:

$ whois buenosaires.com.ar

This TLD has no whois server, but you can access the whois database at
http://www.nic.ar/

Now if you actually try their suggested whois server, it doesn’t even work:

$ whois ‐h www.nic.ar buenosaires.com.ar

Timeout.

What you can do to find the correct whois server is use iana – Internet Assigned Numbers Authority – namely, this page:

https://www.iana.org/domains/root/db

So for Argentina I clicked on .ar (I expected to find a separate listing for .com.ar but that was not the case), leading to the page:

See it? At the bottom it shows Whois server: whois.nic.ar. So I try that and voila, meaningful information is returned, no ads accompanying:

$ whois ‐h whois.nic.ar buenosaires.com.ar

% La información a la que estás accediendo se provee exclusivamente para
% fines relacionados con operaciones sobre nombres de dominios y DNS,
% quedando absolutamente prohibido su uso para otros fines.
%
% La DIRECCIÓN NACIONAL DEL REGISTRO DE DOMINIOS DE INTERNET es depositaria
% de la información que los usuarios declaran con la sola finalidad de
% registrar nombres de dominio en ‘.ar’, para ser publicada en el sitio web
% de NIC Argentina.
%
% La información personal que consta en la base de datos generada a partir
% del sistema de registro de nombres de dominios se encuentra amparada por
% la Ley N° 25326 “Protección de Datos Personales” y el Decreto
% Reglamentario 1558/01.
 
domain:         buenosaires.com.ar
registrant:     50030338720
registrar:      nicar
registered:     2012-07-05 00:00:00
changed:        2017-06-27 17:42:45.944889
expire:         2018-07-05 00:00:00
 
contact:        50030338720
name:           TRAVEL RESERVATIONS SRL
registrar:      nicar
created:        2013-09-05 00:00:00
changed:        2018-04-17 13:14:55.331068
 
nserver:        ns-1588.awsdns-06.co.uk ()
nserver:        ns-925.awsdns-51.net ()
nserver:        ns-1385.awsdns-45.org ()
nserver:        ns-239.awsdns-29.com ()
registrar:      nicar
created:        2016-07-01 00:02:28.608837

2nd example: goto.jobs
I actually needed this one! So I learned of a domain goto.jobs and I wanted to get some background. So here goes…
$ whois goto.jobs

getaddrinfo(jobswhois.verisign-grs.com): Name or service not known

So off to a bad start, right? So we hit up the .jobs link on iana, https://www.iana.org/domains/root/db/jobs.html, and we spy a reference to their whois server:

Registry Information
This domain is managed under ICANN's registrar system. You may register domains in .JOBS through an ICANN accredited registrar. The official list of ICANN accredited registrars is available on ICANN's website.
URL for registration services: http://www.goto.jobs
WHOIS Server: whois.nic.jobs

So we try that:
$ whois ‐h whois.nic.jobs goto.jobs

   Domain Name: GOTO.JOBS
   Registry Domain ID: 91478530_DOMAIN_JOBS-VRSN
   Registrar WHOIS Server: whois-all.nameshare.com
   Registrar URL: http://www.nameshare.com
   Updated Date: 2018-03-29T20:08:46Z
   Creation Date: 2010-02-04T23:54:33Z
   Registry Expiry Date: 2019-02-04T23:54:33Z
   Registrar: Name Share, Inc
   Registrar IANA ID: 667
   Registrar Abuse Contact Email:
   Registrar Abuse Contact Phone:
   Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
   Name Server: KATE.NS.CLOUDFLARE.COM
   Name Server: MARK.NS.CLOUDFLARE.COM
   Name Server: NS1.REGISTRY.JOBS
   Name Server: NS2.REGISTRY.JOBS
   DNSSEC: unsigned
   URL of the ICANN Whois Inaccuracy Complaint Form: https://www.icann.org/wicf/
>>> Last update of WHOIS database: 2018-04-23T18:54:31Z <<<

Better, but it seems to merely point to a registrar and its whois server:

Registrar WHOIS Server: whois-all.nameshare.com

So let’s try that:

$ whois ‐h whois-all.nameshare.com goto.jobs

Domain Name: GOTO.JOBS
Registry Domain ID: 91478530_DOMAIN_JOBS-VRSN
Registrar WHOIS Server: whois-jobs.nameshare.com
Registrar URL: http://www.nameshare.com
Updated Date: 2018-03-29T20:08:46Z
Creation Date: 2010-02-04T23:54:33Z
Registrar Registration Expiration Date: 2017-02-04T23:54:33Z
Registrar: NameShare, Inc.
Registrar IANA ID: 667
Registrar Abuse Contact Email: abuse-2014-2@encirca.com
Registrar Abuse Contact Phone: +1.7809429975
Domain Status: clientTransferProhibited http://www.icann.org/epp#clientTransferProhibited
Registry Registrant ID:
Registrant Name: DNS Administrator
Registrant Organization: Employ Media LLC
Registrant Street: 3029 Prospect Avenue
Registrant City: Cleveland
Registrant State/Province: OH
Registrant Postal Code: 44115
Registrant Country: United States
Registrant Phone: +1.2064261500
Registrant Phone Ext:
Registrant Fax: +1.1111111111
Registrant Fax Ext:
Registrant Email: supportgoto@goto.jobs
Registry Admin ID:
Admin Name: DNS Administrator
Admin Organization: Employ Media LLC
Admin Street: 3029 Prospect Avenue
...

Bingo! We have hit pay dirt. We have meaningful information about the registrant – an address, phone number and email address – and received no obnoxious ads in return. For me it’s worth the extra steps.

ICANN: another alternative
Most registrar’s whois sites are rate-limited. ICANN’s is not. And they also do not sic ads on you. It is

https://whois.icann.org/en/lookup?name=

ICANN, for the record, it the body that decides what goes on in DNS namespace, for instance, what new gTLDS should be added. You can use its whois tool for all gTLDs, but not in general for ccTLDs.

whois is undergoing changes due to GDPR. Especially the “social” information of the contacts: registrant, admin and technical contacts will be masked, except for perhaps state and country, in the future. But whois is slowly dying and a new standard called RDAP will take its place.

References and related
This page has some great tips. Wish I had seen it first! https://superuser.com/questions/758647/how-to-whois-new-tlds

Here’s that iana root zone database link again: https://www.iana.org/domains/root/db

ICANN’s whois: https://whois.icann.org/en/lookup?name=

Categories
Admin DNS Network Technologies

Bluecoat ProxySG and DNS using edns seem incompatible

Intro
Imagine your DNS server had this behaviour when queried using dig:

$ dig drjohnstechtalk.com @146.201.145.30

; <<>> DiG 9.9.2-P2 <<>> drjohnstechtalk.com @10.201.145.30
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: FORMERR, id: 48905
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
 
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;drjohnstechtalk.com.           IN      A
 
;; Query time: 1 msec
;; SERVER: 10.201.145.30#53(146.201.145.30)
;; WHEN: Fri Feb 24 12:16:42 2017
;; MSG SIZE  rcvd: 48

That would be pretty disturbing, right? The only way to get dig to behave is to turn off edns like this:

$ dig +noedns drjohnstechtalk.com @146.201.145.30

; <<>> DiG 9.9.2-P2 <<>> +noedns drjohnstechtalk.com @10.201.145.30
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31299
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
 
;; QUESTION SECTION:
;drjohnstechtalk.com.           IN      A
 
;; ANSWER SECTION:
drjohnstechtalk.com.    3277    IN      A       50.17.188.196
 
;; Query time: 3 msec
;; SERVER: 10.201.145.30#53(146.201.145.30)
;; WHEN: Fri Feb 24 12:17:00 2017
;; MSG SIZE  rcvd: 53

Nslookup works. But who uses nslookup anyway?

Furthermore, imagine that DNS client and server are on the same subnet, so there is no firewall intermediating their traffic. so we know we can’t blame firewall cutting off large DNS packets, unlike the suggestions made in the references section.

Well, this is exactly the situation in a large company where I consult. The DNS server is unusual: a Bluecoat ProxySG, which can conveniently combine replies from nameservers from two different namespaces.

There does not seem to be an option to handle edns queries correctly on a Bluecoat device.

The client is running SLES version 11. The real question is how will applications behave? Which type of query will they make?

Bluecoat Response
Bluecoat does not support eDNS and gives a response permitted by RFC2671. RFC2671 also encourages clients to account for error responses and drop the use of eDNS in a retry.

References and related
EDNS: What is it all about? is a really good explanation of edns and how it came about, how it’s supposed to work, etc.
This post suggests some scenarios where edns may not work, though it does not address the Bluecoat issue: http://blog.fpweb.net/strange-dns-issues-better-check-out-edns/#.WLBmw3dvDkk
RFC 2671

Categories
DNS Hosting Service

Free DNS services

Intro
I stumbled upon freenom.com, which offers free DNS domain names. So I tested it and successfully registered drjohnstechtalk.ml. .ml is Mali’s top level domain. I’ve heard it’s the only African country to open up its domains for free registration. There are a few other choices like .tk and .gq which are even more obscure.

As far as I can see this is a no strings service. It’s possible that they will share your registration data with third parties, but I don’t think this is the case since freenom is from the Netherlands where privacy tends to be stricter than in the US.

Limitations
Creating an address record is easy enough, but no other kind of resource record seems available to free users. That can be pretty limiting.

References and related
www.freenom.com. In the search field just put the domain name without the extension, e.g., drjohnstechalk. It will tell you which extensions are available for free.
How about putting a free certificate on your free domain? Let’s Encrypt makes it possible.

Categories
CentOS DNS Linux Network Technologies Raspberry Pi Security Web Site Technologies

Roll your own dynamic DNS update service

Intro
I know my old Cisco router only has built-in support for two dynamic DNS services, dyndns.org and TZO.com. Nowadays you have to pay for those, if even they work (the web site domain names seem to have changed, but perhaps they still support the old domain names. Or perhaps not!). Maybe this could be fixed by firmware upgrades (to hopefully get more choices and hopefully a free one, or a newer router, or running DD-WRT. I didn’t do any of those things. Being a network person at heart, I wrote my own. I found the samples out there on the Internet needed some updating, so I am sharing my recipe. I didn’t think it was too hard to pull off.

What I used
– GoDaddy DNS hosting (basically any will do)
– my Amazon AWS virtual server running CentOS, where I have sudo access
– my home Raspberry Pi
– a tiny bit of php programming
– my networking skills for debugging

As I have prior experience with all these items this project was right up my alley.

Delegating our DDNS domain from GoDaddy
Just create a nameserver record from the domain, say drj.com, called, say, raspi, which you delegate to your AWS server. Following the example, the subdomain would be raspi.drj.com whose nameserver is drj.com.


DNS Setup on an Amazon AWS server

/etc/named.conf

//
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS
// server as a caching only nameserver (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//
 
options {
//      listen-on port 53 { 127.0.0.1; };
//      listen-on port 53;
        listen-on-v6 port 53 { ::1; };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
        allow-query     { any; };
        recursion no;
 
        dnssec-enable yes;
        dnssec-validation yes;
        dnssec-lookaside auto;
 
        /* Path to ISC DLV key */
        bindkeys-file "/etc/named.iscdlv.key";
 
        managed-keys-directory "/var/named/dynamic";
};
 
logging {
        channel default_debug {
                file "data/named.run";
                severity dynamic;
        };
};
 
zone "." IN {
        type hint;
        file "named.ca";
};
 
include "/etc/named.rfc1912.zones";
include "/var/named/dynamic.conf";
include "/etc/named.root.key";

/var/named/dynamic.conf

zone "raspi.drj.com" {
  type master;
  file "/var/named/db.raspi.drj.com";
// designed to work with nsupdate -l used on same system - DrJ 10/2016
// /var/run/named/session.key
  update-policy local;
};

/var/named/db.raspi.drj.com

$ORIGIN .
$TTL 1800       ; 30 minutes
raspi.drj.com      IN SOA  drj.com. postmaster.drj.com. (
                                2016092812 ; serial
                                1700       ; refresh (28 minutes 20 seconds)
                                1700       ; retry (28 minutes 20 seconds)
                                1209600    ; expire (2 weeks)
                                600        ; minimum (10 minutes)
                                )
                        NS      drj.com.
$TTL 3600       ; 1 hour
                        A       125.125.73.145

Named re-starting program
Want to make sure your named restarts if it happens to die? nanny.pl is a good, simple monitor to do that. Here is the version I use on my server. Note the customized variables towards the top.

#!/usr/bin/perl
#
# Copyright (C) 2004, 2007, 2012  Internet Systems Consortium, Inc. ("ISC")
# Copyright (C) 2000, 2001  Internet Software Consortium.
#
# Permission to use, copy, modify, and/or distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
# AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
# OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
# PERFORMANCE OF THIS SOFTWARE.
 
# $Id: nanny.pl,v 1.11 2007/06/19 23:47:07 tbox Exp $
 
# A simple nanny to make sure named stays running.
 
$pid_file_location = '/var/run/named/named.pid';
$nameserver_location = 'localhost';
$dig_program = 'dig';
$named_program =  '/usr/sbin/named -u named';
 
fork() && exit();
 
for (;;) {
        $pid = 0;
        open(FILE, $pid_file_location) || goto restart;
        $pid = <FILE>;
        close(FILE);
        chomp($pid);
 
        $res = kill 0, $pid;
 
        goto restart if ($res == 0);
 
        $dig_command =
               "$dig_program +short . \@$nameserver_location > /dev/null";
        $return = system($dig_command);
        goto restart if ($return == 9);
 
        sleep 30;
        next;
 
 restart:
        if ($pid != 0) {
                kill 15, $pid;
                sleep 30;
        }
        system ($named_program);
        sleep 120;
}

The PHP updating program myip-update.php

<?php
# DrJ: lifted from http://pablohoffman.com/dynamic-dns-updates-with-a-simple-php-script
# but with some security improvements
# 10/2016
# PHP script for very simple dynamic DNS updates
#
# this script was published in http://pablohoffman.com/articles and
# released to the public domain by Pablo Hoffman on 27 Aug 2006
 
# CONFIGURATION BEGINS -------------------------------------------------------
# define password here
$mysecret = 'myBigFatsEcreT';
# CONFIGURATION ENDS ---------------------------------------------------------
 
 
$ip = $_SERVER['REMOTE_ADDR'];
$host = $_GET['host'];
$secret = $_POST['secret'];
$zone = $_GET['zone'];
$tmpfile = trim(`mktemp /tmp/nsupdate.XXXXXX`);
 
if ((!$host) or (!$zone) or (!($mysecret == $secret))) {
    echo "FAILED";
    unlink($tmpfile);
    exit;
}
 
$oldip = trim(`dig +short $host.$zone @localhost`);
if ($ip == $oldip) {
    echo "UNCHANGED. ip: $ip\n";
    unlink($tmpfile);
    exit;
}
 
echo "$ip - $oldip";
 
$nsucmd = "update delete $host.$zone A
update add $host.$zone 3600 A $ip
send
";
 
$fp = fopen($tmpfile, 'w');
fwrite($fp, $nsucmd);
fclose($fp);
`sudo nsupdate -l $tmpfile`;
unlink($tmpfile);
echo "OK ";
echo `date`;
?>

In the above file I added the “sudo” after awhile. See explanation further down below.

Raspberry Pi requirements
I’ve assumed you can run your Pi 24 x 7 and constantly and consistently on your network.

Crontab entry on the Raspberry Pi
Edit the crontab file for periodically checking your IP on the Pi and updating external DNS if it has changed by doing this:

$ crontab ‐e
and adding the line below:

# my own method of dynamic update - DrJ 10/2016
0,10,20,30,40,50 * * * * /usr/bin/curl -s -k -d 'secret=myBigFatsEcreT' 'https://drj.com/myip-update.php?host=raspi&zone=drj.com' >> /tmp/ddns 2>&1

A few highlights
Note that I’ve switched to use of nsupdate -l on the local server. This will be more secure than the previous solution which suggested to have updates from localhost. As far as I can tell localhost updates can be spoofed and so should be considered insecure in a modern infrastructure. I learned a lot by running nsupdate -D -l on my AWS server and observing what happens.
And note that I changed the locations of the secret. The old solution had the secret embedded in the URL in a GET statement, which means it would also be embedded in every single request in the web server’s access file. That’s not a good idea. I switched it to a POSTed variable so that it doesn’t show up in the web server’s access file. This is done with the -d switch of curl.

Contents of temporary file
Here are example contents. This is useful when you’re trying to run nsupdate from the command line.

update delete raspi.drj.com A
update add raspi.drj.com 3600 A 51.32.108.37
send


Permissions problems

If you see something like this on your DNS server:

$ ll /var/run/named

total 8
-rw-r--r-- 1 named www-data   6 Nov  6 03:15 named.pid
-rw------- 1 named www-data 102 Oct 24 09:42 session.key

your attempt to run nsupdate by your web server will be foiled and produce something like this:

$ /usr/bin/nsupdate ‐l /tmp/nsupdate.LInUmo

06-Nov-2016 17:14:14.780 none:0: open: /var/run/named/session.key: permission denied
can't read key from /var/run/named/session.key: permission denied

The solution may be to permit group read permission:

$ cd /var/run/named; sudo chmod g+r session.key

and make the group owner of the file your webserver user ID (which I’ve already done here). I’m still working this part out…

That approach doesn’t seem to “stick,” so I came up with this other approach. Put your web server user in sudoers to allow it to run nsupdate (my web server user is www-data for these examples):

Cmnd_Alias     NSUPDATE = /usr/bin/nsupdate
# allow web server to run nsupdate
www-data ALL=(root) NOPASSWD: NSUPDATE

But you may get the dreaded

sudo: sorry, you must have a tty to run sudo

if you manage to figure out how to turn on debugging.

So if your sudoers has a line like this:

Defaults    requiretty

you will need lines like this:

# turn of tty requirements only for www-data user
Defaults:www-data !requiretty

Debugging
Of course for debugging I commented out the unlink line in the PHP update file and ran the
nsupdate -l /tmp/nsupdate.xxxxx
by hand as user www-data.

During some of the errors I worked through that wasn’t verbose enough so I added debugging arguments:

$ nsupdate ‐D ‐d ‐l /tmp/nsupdate.xxxxx

When that began to work, yet when called via the webserver it wasn’t working, I ran the above command from within PHP, recording the output to a file:

...
`sudo nsupdate -d -D -l $tmpfile > /tmp/nsupdate-debug 2>&1`

That turned out to be extremely informative.

Conclusion
We have shown how to combine a bunch of simple networking tools to create your own DDNS service. The key elements are a Raspberry Pi and your own virtual server at Amazon AWS. We have built upon previous published solutions to this problem and made them more secure in light of the growing sophistication of the bad guys. Let me know if there is interest in an inexpensive commercial service.

References and related

nanny.pl write-up: https://www.safaribooksonline.com/library/view/dns-bind/0596004109/ch05s09.html

Categories
DNS Linux Perl Raspberry Pi Web Site Technologies

Roll your own domain drop catching service using GoDaddy

Intro
I’m after a particular domain and have been for years. But as a matter of pride I don’t want to overpay for it, so I don’t want to go through an auction. There are services that can help grab a DNS domain immediately after it expires, but they all want $$. That may make sense for high-demand domains. Mine is pretty obscure. I want to grab it quickly – perhaps within a few seconds after it becomes available, but I don’t expect any competition for it. That is a description of domain drop catching.

Since I am already using GoDaddy as my registrar I thought I’d see if they have a domain catching service. They don’t which is strange because they have other specialized domain services such as domain broker. They have a service which is designed for much the same purpose, however, called backorder. That creates an auction bid for the domain before it has expired. The cost isn’t too bad, but since I started down a different path I will roll my own. Perhaps they have an API which can be used to create my own domain catcher? It turns out they do!

It involves understanding how to read a JSON data file, which is new to me, but otherwise it’s not too bad.

The domain lifecycle
This graphic from ICANN illustrates it perfectly for your typical global top-level domain such as .com, .net, etc:
gtld-lifecycle

To put it into words, there is the
initial registration,
optional renewals,
expiration date,
auto-renew grace period of 0 – 45 days,
redemption grace period of 30 days,
pending delete of 5 days, and then
it’s released and available.

So in domain drop catching we are keenly interested in being fully prepared for the pending delete five day window. From an old discussion I’ve read that the precise time .com domains are released is usually between 2 -3 PM EST.

A word about the GoDaddy developer site
It’s developer.godaddy.com. It looks like one day it will be a great site, but for now it is wanting in some areas. Most of the menu items are duds and are placeholders. Really there are only three (mostly) working sections: get started, documentation and demo. Get started is only a few words and one slender snippet of Ajax code, and the demo itself also extremely limited, so the only real resource they provide is Documentation. Documentation is designed as an active documentation that you can try out functions with your data. You run it and it shows you all the needed request headers and data as well as the received response. The thing is that it’s very finicky. It’s supposed to show all the available functions but I couldn’t get it to work under Firefox. And with Internet Explorer/Edge it only worked about half the time. It seems to help to access it with a newly launched browser. The documentation, as good as it is, leaves some things unsaid. I have found:

https://api.ote-godaddy.com/ – use for TEST. Maybe ote stands for optional test environment?
https://api.godaddy.com/ – for production (what I am calling PROD)

The TEST environment does not require authentication for some things that PROD does. This shell script for checking available domains, which I call available-test.sh, works in TEST but not in PROD:

#!/bin/sh
# pass domain as argument
# apparently no AUTH is rquired for this one
curl -k 'https://api.ote-godaddy.com/v1/domains/available?domain='$1'&checkType=FAST&forTransfer=false'

In PROD I had to insert the authorization information – the key and secret they showed me on the screen. I call this script available.sh.

#!/bin/sh
# pass domain as argument
curl -s -k -H 'Authorization: sso-key *******8m_PwFAffjiNmiCUrKe******:**FF73L********' 'https://api.godaddy.com/v1/domains/available?domain='$1'&checkType=FULL&forTransfer=false'

I found that my expiring domain produced different results about five days after expiring if I used checkType of FAST versus checkType of FULL – and FAST was wrong. So I learned you have to use FULL to get an answer you can trust!

Example usage of an available domain

$ ./available.sh dr-johnstechtalk.com

{"available":true,"domain":"dr-johnstechtalk.com","definitive":false,"price":11990000,"currency":"USD","period":1}

2nd example – a non-available domain
$ ./available.sh drjohnstechtalk.com

{"available":false,"domain":"drjohnstechtalk.com","definitive":true,"price":11990000,"currency":"USD","period":1}

Example JSON file
I had to do a lot of search and replace to preserve my anonymity, but I feel this post wouldn’t be complete without showing the real contents of my JSON file I am using for both validate, and, hopefully, as the basis for my API-driven domain purchase:

{
  "domain": "dr-johnstechtalk.com",
  "renewAuto": true,
  "privacy": false,
  "nameServers": [
  ],
  "consent": {
    "agreementKeys": ["DNRA"],
    "agreedBy": "50.17.188.196",
    "agreedAt": "2016-09-29T16:00:00Z"
  },
  "period": 1,
  "contactAdmin": {
    "nameFirst": "Dr","nameLast": "John",
    "email": "dr-john@gmail.com",
    "addressMailing": {
      "address1": "555 Piney Drive",
      "city": "Smallville","state": "New Jersey","postalCode": "55555",
      "country": "US"
    },
    "phone": "+1.5555551212"
  },
  "contactBilling": {
    "nameFirst": "Dr","nameLast": "John",
    "email": "dr-john@gmail.com",
    "addressMailing": {
      "address1": "555 Piney Drive",
      "city": "Smallville","state": "New Jersey","postalCode": "55555",
      "country": "US"
    },
    "phone": "+1.5555551212"
  },
  "contactRegistrant": {
    "nameFirst": "Dr","nameLast": "John",
    "email": "dr-john@gmail.com",
    "phone": "+1.5555551212",
    "addressMailing": {
      "address1": "555 Piney Drive",
      "city": "Smallville","state": "New Jersey","postalCode": "55555",
      "country": "US"
    }
  },
  "contactTech": {
    "nameFirst": "Dr","nameLast": "John",
    "email": "dr-john@gmail.com",
    "phone": "+1.5555551212",
    "addressMailing": {
      "address1": "555 Piney Drive",
      "city": "Smallville","state": "New Jersey","postalCode": "55555",
      "country": "US"
    }
  }
}

Note the agreementkeys value: DNRA. GoDaddy doesn’t document this very well, but that is what you need to put there! Note also that the nameservers are left empty. I asked GoDaddy and that is what they advised to do. The other values are pretty much what you’d expect. I used my own server’s IP address for agreedBy – use your own IP. I don’t know how important it is to get the agreedAt date close to the current time. I’m going to assume it should be within 24 hours of the current time.

How do we test this JSON input file? I wrote a validate script for that I call validate.sh.

#!/bin/sh
# DrJ 9/2016
# godaddy-json-register was built using GoDaddy's documentation at https://developer.godaddy.com/doc#!/_v1_domains/validate
jsondata=`tr -d '\n' < godaddy-json-register`
 
curl -i -k -H 'Authorization: sso-key *******8m_PwFAffjiNmiCUrKe******:**FF73L********' -H 'Content-Type: application/json' -H 'Accept: application/json' -d "$jsondata" https://api.godaddy.com/v1/domains/purchase/validate

Run the validate script
$ ./validate.sh

HTTP/1.1 100 Continue
 
HTTP/1.1 100 Continue
Via: 1.1 api.godaddy.com
 
HTTP/1.1 200 OK
Date: Thu, 29 Sep 2016 20:11:33 GMT
X-Powered-By: Express
Vary: Origin,Accept-Encoding
Access-Control-Allow-Credentials: true
Content-Type: application/json; charset=utf-8
ETag: W/"2-mZFLkyvTelC5g8XnyQrpOw"
Via: 1.1 api.godaddy.com
Transfer-Encoding: chunked

Revised versions of the above scripts
So we can pass the domain name as argument I revised all the scripts. Also, I provide an agreeddAt date which is current.

The data file: godaddy-json-register

{
  "domain": "DOMAIN",
  "renewAuto": true,
  "privacy": false,
  "nameServers": [
  ],
  "consent": {
    "agreementKeys": ["DNRA"],
    "agreedBy": "50.17.188.196",
    "agreedAt": "DATE"
  },
  "period": 1,
  "contactAdmin": {
    "nameFirst": "Dr","nameLast": "John",
    "email": "dr-john@gmail.com",
    "addressMailing": {
      "address1": "555 Piney Drive",
      "city": "Smallville","state": "New Jersey","postalCode": "55555",
      "country": "US"
    },
    "phone": "+1.5555551212"
  },
  "contactBilling": {
    "nameFirst": "Dr","nameLast": "John",
    "email": "dr-john@gmail.com",
    "addressMailing": {
      "address1": "555 Piney Drive",
      "city": "Smallville","state": "New Jersey","postalCode": "55555",
      "country": "US"
    },
    "phone": "+1.5555551212"
  },
  "contactRegistrant": {
    "nameFirst": "Dr","nameLast": "John",
    "email": "dr-john@gmail.com",
    "phone": "+1.5555551212",
    "addressMailing": {
      "address1": "555 Piney Drive",
      "city": "Smallville","state": "New Jersey","postalCode": "55555",
      "country": "US"
    }
  },
  "contactTech": {
    "nameFirst": "Dr","nameLast": "John",
    "email": "dr-john@gmail.com",
    "phone": "+1.5555551212",
    "addressMailing": {
      "address1": "555 Piney Drive",
      "city": "Smallville","state": "New Jersey","postalCode": "55555",
      "country": "US"
    }
  }
}

validate.sh

#!/bin/sh
# DrJ 10/2016
# godaddy-json-register was built using GoDaddy's documentation at https://developer.godaddy.com/doc#!/_v1_domains/validate
# pass domain as argument
# get date into accepted format
domain=$1
date=`date -u --rfc-3339=seconds|sed 's/ /T/'|sed 's/+.*/Z/'`
jsondata=`tr -d '\n' < godaddy-json-register`
jsondata=`echo $jsondata|sed 's/DATE/'$date'/'`
jsondata=`echo $jsondata|sed 's/DOMAIN/'$domain'/'`
#echo date is $date
#echo jsondata is $jsondata
curl -i -k -H 'Authorization: sso-key *******8m_PwFAffjiNmiCUrKe******:**FF73L********' -H 'Content-Type: application/json' -H 'Accept: application/json' -d "$jsondata" https://api.godaddy.com/v1/domains/purchase/validate

available.sh
No change. See listing above.

purchase.sh
Exact same as validate.sh, with just a slightly different URL. I need to test that it really works, but based on my reading I think it will.

#!/bin/sh
# DrJ 10/2016
# godaddy-json-register was built using GoDaddy's documentation at https://developer.godaddy.com/doc#!/_v1_domains/purchase
# pass domain as argument
# get date into accepted format
domain=$1
date=`date -u --rfc-3339=seconds|sed 's/ /T/'|sed 's/+.*/Z/'`
jsondata=`tr -d '\n' < godaddy-json-register`
jsondata=`echo $jsondata|sed 's/DATE/'$date'/'`
jsondata=`echo $jsondata|sed 's/DOMAIN/'$domain'/'`
#echo date is $date
#echo jsondata is $jsondata
curl -s -i -k -H 'Authorization: sso-key *******8m_PwFAffjiNmiCUrKe******:**FF73L********' -H 'Content-Type: application/json' -H 'Accept: application/json' -d "$jsondata" https://api.godaddy.com/v1/domains/purchase

Putting it all together
Here’s a looping script I call loop.pl. I switched to perl because it’s easier to do certain control operations.

#!/usr/bin/perl
#DrJ 10/2016
$DEBUG = 0;
$status = 0;
open STDOUT, '>', "loop.log" or die "Can't redirect STDOUT: $!";
                   open STDERR, ">&STDOUT"     or die "Can't dup STDOUT: $!";
 
                   select STDERR; $| = 1;      # make unbuffered
                   select STDOUT; $| = 1;      # make unbuffered
# edit this and change to your about-to-expire domain
$domain = "dr-johnstechtalk.com";
while ($status != 200) {
# show that we're alive and working...
  print "Now it's ".`date` if $i++ % 10 == 0;
  $hr = `date +%H`;
  chomp($hr);
# run loop more aggressively during times of day we think Network Solutions releases domains back to the pool, esp. around 2 - 3 PM EST
  $sleep = $hr > 11 && $hr < 16 ? 1 : 15;
  print "Hr,sleep: $hr,$sleep\n" if $DEBUG;
  $availRes = `./available.sh $domain`;
# {"available":true,"domain":"dr-johnstechtalk.com","definitive":false,"price":11990000,"currency":"USD","period":1}
  print "$availRes\n" if $DEBUG;
  ($available) = $availRes =~ /^\{"available":([^,]+),/;
  print "$available\n" if $DEBUG;
  if ($available eq "false") {
    print "test comparison OP for false result\n" if $DEBUG;
  } elsif ($available eq "true") {
# available value of true is extremely unreliable with many false positives. Confirm availability by making a 2nd call
    print "available.sh results: $availRes\n";
    $availRes = `./available.sh $domain`;
    print "available.sh re-test results: $availRes\n";
    ($available2) = $availRes =~ /^\{"available":([^,]+),/;
    next if $available2 eq "false";
# We got two available eq true results in a row so let's try to buy it!
    print "$domain is very likely available. Trying to buy it at ".`date`;
    open(BUY,"./purchase.sh $domain|") || die "Cannot run ./purchase.pl $domain!!\n";
    while(<BUY>) {
# print out each line so we can analyze what happened
      print ;
# we got it if we got back
# HTTP/1.1 200 OK
      if (/1.1 200 OK/) {
        print "We just bought $domain at ".`date`;
        $status = 200;
     }
    } # end of loop over results of purchase
    close(BUY);
    print "\n";
    exit if $status == 200;
  } else {
    print "available is neither false nor true: $available\n";
  }
  sleep($sleep);
}

Running the loop script
$ nohup ./loop.pl > loop.log 2>&1 &
Stopping the loop script
$ kill ‐9 %1

Description of loop.pl
I gotta say this loop script started out as a much simpler script. I fortunately started on it many days before my desired domain actually became available so I got to see and work out all the bugs. Contributing to the problem is that GoDaddy’s API results are quite unreliable. I was seeing a lot of false positives – almost 20%. So I decided to require two consecutive calls to available.sh to return true. I could have required available true and definitive true, but I’m afraid that will make me late to the party. The API is not documented to that level of detail so there’s no help there. But so far what I have seen is that when available incorrectly returns true, simultaneously definitive becomes false, whereas all other times definitive is true.

Results of running an earlier and simpler version of loop.pl

This shows all manner of false positives. But at least it never allowed me to buy the domain when it wasn’t available.

Now it's Wed Oct  5 15:20:01 EDT 2016
Now it's Wed Oct  5 15:20:19 EDT 2016
Now it's Wed Oct  5 15:20:38 EDT 2016
available.sh results: {"available":true,"domain":"dr-johnstechtalk.com","definitive":false,"price":11990000,"currency":"USD","period":1}
dr-johnstechtalk.com is available. Trying to buy it at Wed Oct  5 15:20:46 EDT 2016
HTTP/1.1 100 Continue
 
HTTP/1.1 100 Continue
Via: 1.1 api.godaddy.com
 
HTTP/1.1 422 Unprocessable Entity
Date: Wed, 05 Oct 2016 19:20:47 GMT
X-Powered-By: Express
Vary: Origin,Accept-Encoding
Access-Control-Allow-Credentials: true
Content-Type: application/json; charset=utf-8
ETag: W/"7d-O5Dw3WvJGo8h30TqR7j8zg"
Via: 1.1 api.godaddy.com
Transfer-Encoding: chunked
 
{"code":"UNAVAILABLE_DOMAIN","message":"The specified `domain` (dr-johnstechtalk.com) isn't available for purchase","name":"ApiError"}
Now it's Wed Oct  5 15:20:58 EDT 2016
Now it's Wed Oct  5 15:21:16 EDT 2016
Now it's Wed Oct  5 15:21:33 EDT 2016
available.sh results: {"available":true,"domain":"dr-johnstechtalk.com","definitive":false,"price":11990000,"currency":"USD","period":1}
dr-johnstechtalk.com is available. Trying to buy it at Wed Oct  5 15:21:34 EDT 2016
HTTP/1.1 100 Continue
 
HTTP/1.1 100 Continue
Via: 1.1 api.godaddy.com
 
HTTP/1.1 422 Unprocessable Entity
Date: Wed, 05 Oct 2016 19:21:36 GMT
X-Powered-By: Express
Vary: Origin,Accept-Encoding
Access-Control-Allow-Credentials: true
Content-Type: application/json; charset=utf-8
ETag: W/"7d-O5Dw3WvJGo8h30TqR7j8zg"
Via: 1.1 api.godaddy.com
Transfer-Encoding: chunked
 
{"code":"UNAVAILABLE_DOMAIN","message":"The specified `domain` (dr-johnstechtalk.com) isn't available for purchase","name":"ApiError"}
Now it's Wed Oct  5 15:21:55 EDT 2016
Now it's Wed Oct  5 15:22:12 EDT 2016
Now it's Wed Oct  5 15:22:30 EDT 2016
available.sh results: {"available":true,"domain":"dr-johnstechtalk.com","definitive":false,"price":11990000,"currency":"USD","period":1}
dr-johnstechtalk.com is available. Trying to buy it at Wed Oct  5 15:22:30 EDT 2016
...

These results show why i had to further refine the script to reduce the frequent false positives.

Review
What have we done? Our looping script, loop.pl, loops more aggressively during the time of day we think Network Solutions releases expired .com domains (around 2 PM EST). But just in case we’re wrong about that we’ll run it anyway at all hours of the day, but just not as quickly. So during the aggressive period we sleep just one second between calls to available.sh. When the domain finally does become available we call purchase.sh on it and exit and we write some timestamps and the domain we’ve just registered to our log file.

Performance
Miserable. This API seems tuned for relative ease-of-use, not speed. The validate call often takes, oh, say 40 seconds to return! I’m sure the purchase call will be no different. For a domainer that’s a lifetime. So any strategy that relies on speed had better turn to a registrar that’s tuned for it. GoDaddy I think is more aiming at resellers of their services.

Don’t have a linux environment handy?
Of course I’m using my own Amazon AWS server for this, but that needn’t be a barrier to entry. I could have used one of my Raspberry Pi’s. Probably even Cygwin on a Windows PC could be made to work.

Appendix A
How to remove all newline characters from your JSON file

Let’s say you have a nice JSON file which was created for you from the Documentation exercises called godaddy-json-register. It will contain lots of newline (“\n”) characters, assuming you’re using a Linux server. Remove them and put the output into a file called compact-json:

$ tr ‐d '\n'<godaddy‐json‐register>compact‐json

I like this because then I can still use curl rather than wget to make my API calls.

Appendix B
What an expiring domain looks like in whois

Run this from a linux server
$ whois <expiring‐domain.com>

Domain Name: expiring-domain.com
...
Creation Date: 2010-09-28T15:55:56Z
Registrar Registration Expiration Date: 2016-09-27T21:00:00Z
...
Domain Status: clientDeleteHold
Domain Status: clientDeleteProhibited
Domain Status: clientTransferProhibited
...

You see that Domain Status: clientDeleteHold? You don’t get that for regular domains whose registration is still good. They’ll usually have the two lines I show below that, but not that one. This is shown for my desired domain just a few days after its official expiration date.

Conclusion
We show that GoDaddy’s API works and we provide simple scripts which can automate what is known as domain dropcatching. This approach should attempt to register a domain within a cople seconds of its being released – if we’ve done everything right. The GoDaddy API results are a little unstable however.

References and related
If you don’t mind paying, Fabulous.com has a domain drop catching service.
ICANN’s web site with the domain lifecycle infographic.
GoDaddy’s API documentation: http://developer.godaddy.com/
More about Raspberry Pi: http://raspberrypi.org/
I really wouldn’t bother with Cygwin – just get your hands on real Linux environment.
Curious about some of the curl options I used? Just run curl ‐‐help. I left out the description of the switches I use because it didn’t fit into the narrative.
Something about my Amazon AWS experience from some years ago.
All the perl tricks I used would take another blog post to explain. It’s just stuff I learned over the years so it didn’t take much time at all.
People who buy and sell domains for a living are called domainers. They are professionals and my guide will not make you competitive with them.

Categories
DNS Linux Network Technologies Perl

Announcing a simple DNS web interface and code

Intro
For demonstration purposes I’ve written a WEB interface to do DNS queries. This can be used for light querying. Once it gets abused I will pull it from the web site.

Motivation
Some large enterprises are behind not only a corporate firewall, but also confined to a private namespace with no access to Internet name resolution. Users in such situations can use one of the many available tools to do DNS resolution through the web, but they all want to throw advertising at you and it’s not clear which can be trusted not to load you up with spyware. I am offering this ad-free DNS lookup using my position on the Internet as a trusted source.

And if you’re lucky and looking for code to do this yourself, you might find it. But nowhere will you find a site that’s running its own published code for DNS resolution. Except here.

The code
Admittedly very simple-minded, but hopefully not fatally flawed, here it is in Perl.

#!/usr/bin/perl
use CGI;
$query = new CGI;
%allowedArgs = (domainname => 'dum',type => 'dum',short => 'dum');
#
print "Content-type: text/html\n\n";
print "<pre>\n";
foreach $key ($query->param) {
  exit(1) unless defined $allowedArgs{$key};
  exit(1) if $query->param($key) !~ /^([a-zA-Z0-9\.-]){2,256}$/;
  print "$key " . $query->param($key) . "\n";
}
# possible keys: domainname, type
$domainname = $query->param(domainname);
$type     = $query->param(type);
$type = "any" unless $type;
# argument validation checks
exit(1) if $domainname !~ /^([a-zA-Z0-9\.-]){2,256}$/ || $domainname =~ /\.\./ ||  ! $domainname;
exit(1) if $type !~ /^([a-zA-Z]){1,8}$/;
 
# short answer?
$short = "+short" if defined $query->param(short);
 
# authoritative request?
if (defined $query->param(authoritative)) {
# this will be a lot more complicated and so is not implemented. Perhaps someday if there is a request...
}
 
open(DIG,"dig $short $type $domainname|") || die "Cannot run dig!!\n";
while(<DIG>) {
  print ;
}

Yes it’s very old-school. I do not even use a DNS package. Why bother? It’s not rocket science. There’s a lot more to argument validation than it looks like – you would not believe the evil things people send to your web server. So you have to vigilant about injection attacks or shelling out by use of unexpected characters.

Usage

example 1

https://drjohnstechtalk.com/cgi-bin/digiface.cgi?domainname=johnstechtalk.com&type=a

domainname johnstechtalk.com
type a
 
; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.4 <<>> a johnstechtalk.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8711
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
 
;; QUESTION SECTION:
;johnstechtalk.com.		IN	A
 
;; ANSWER SECTION:
johnstechtalk.com.	3600	IN	A	50.17.188.196
 
;; Query time: 10 msec
;; SERVER: 172.16.0.23#53(172.16.0.23)
;; WHEN: Mon May  4 14:59:05 2015
;; MSG SIZE  rcvd: 51

example 2
https://drjohnstechtalk.com/cgi-bin/digiface.cgi?domainname=drjohnstechtalk.com&short

domainname drjohnstechtalk.com
short 
50.17.188.196

Familiarity with dig will help you determine the best switches to use as you can see that at the end of the day it is merely calling dig and sending back that output with a minimum of html markup. This will make it easy to parse the output programatically.

Conclusion
A simple DNS web interface is being announced today. Both the service and the code are being made available. The service may be pulled once it becomes abused.

References
A nice, not too commercial web interface to dig and traceroute that is more user-friendly than mine is http://www.kloth.net/services/dig.php
The dig man pages can be helpful.

Although this link has ads, it’s quite interesting because it sends your query to open DNS servers around the world: https://dnschecker.org/.

You can explore some details behind Google’s public resolving server 8.8.8.8 by using the web site: https://dns.google.com/. It’s quite helpful.

I won’t paste the link to my service but you can see what it is from the examples above.

There’s a simple but effective DIG available for your Android smartphone from the Playstore. That’s DNS debugger from TurboBytes. No obnoxious ads and yet no cost.

Of course if you are on the Internet and have access to dig, Google’s DNS servers are available for you to use directly.

Categories
Admin DNS

Example of case-sensitive DNS usage

Intro
From RFC 1035, written in November, 1987:


Note that while upper and lower case letters are allowed in domain
names, no significance is attached to the case. That is, two names with
the same spelling but different case are to be treated as if identical.

The details
Now fast forward in time 27 years. I learned that Cisco IP Phones, when resolving the Call manager name, require that the DNS name for the Cisco Unified Call Manager be in the same exact upper or lower case as what is configured into the phone.

Suppose your Call Manager’s hostname was configured as CUCM.drjohnstechtalk.com and your DNS servers behaved like this:

> dig CUCM.drjohnstechtalk.com @208.109.255.46

; <<>> DiG 9.9.4-P2 <<>> CUCM.drjohnstechtalk.com @208.109.255.46
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15899
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 1
;; WARNING: recursion requested but not available
 
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;CUCM.drjohnstechtalk.com.      IN      A
 
;; ANSWER SECTION:
cucm.drjohnstechtalk.com. 3600  IN      A       50.17.188.196

Well, every application that is compliant with this 27-year-old DNS standard would work just fine. But Cisco phone’s will not. If they were configured to use CUCM.drjohnstechtalk.com and your DNS server spits back the answer to an A (address record) query, changing the FQDN to lower-case, it won’t “find” the call manager and won’t boot! So it’s a garbage implementation of DNS.

Shame on Cisco!

I happened to hear about this problem today, so it can occur under those very special circumstances outlined above. We can’t merely say it is only theoretical. However, mitigating circumstances abound that will make this a rarely observed problem.

Mitigation
Newer DNS servers actually spit back the FQDN in the exact same case as it received in the original query. I’m not sure at this point if this is an option or simply a change in behaviour that occurred at some point in the evolution of the ISC BIND resolver. It would be interesting to see when this behaviour changed.

The other mitigation, if you do have the older DNS servers that spit back the FQDN in lower-case is to configure the hostname in your zone file using upper case to agree with the upper-case version you’ve configured on the phone. With either of these mitigations the DNS server response will look like this:

; <<>> DiG 9.9.4-P2 <<>> CUCM.drjohnstechtalk.com @208.109.255.46
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15899
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 1
;; WARNING: recursion requested but not available
 
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;CUCM.drjohnstechtalk.com.      IN      A
 
;; ANSWER SECTION:
CUCM.drjohnstechtalk.com. 3600  IN      A       50.17.188.196

and the phone will be happy, seeing the case matched and will be able to contact the Call Manager so it can finish booting.

Conclusion
Cisco of all companies has built in to its IP Phones a bad DNS resolver that is case-sensitive. There are some mitigations which can be done while waiting for them to fix this embarrassing bug.

Second example from VMWare circa June 2020
The VMWare Horizon Client v 5.4 has a similar issue. If you use a proxy PAC file with contents like *.drjohnstechtalk.com DIRECT, that may not work for this client if the DNS entry for the hostname was entered in upper case! For instance HostName.DRJOHNSTECHTALK.COM. In that case it acts with case-sensitivyt and ignores the PAC file entry which it should have used to know to make DIRECT (without the aid of a proxy) HTTP connection. Very unfortunate.

References
RFC 1035 – things were so much simpler then!
ISC BIND web site.

Categories
DNS Network Technologies

The IT Detective Agency: internal DNS queries getting clobbered after bind upgrade

Intro
We’ve upgraded BIND innumerable times over the years. There’s never really been an issue. The new version just picks up and behaves exactly like the old version and all is good. But this time, in upgrading from ISC’s BIND v 9.8.5-P2 to BIND v 9.9.5-P1 something was dramatically different.

The details

Look at these queries:

> dig ns 10.in-addr.arpa

; <<>> DiG 9.9.2-P2 <<>> ns 10.in-addr.arpa
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60248
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
 
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;10.in-addr.arpa.               IN      NS
 
;; ANSWER SECTION:
10.in-addr.arpa.        0       IN      NS      10.IN-ADDR.ARPA.
 
;; Query time: 0 msec
;; SERVER: blah, blah
;; WHEN: Wed Jun 25 09:49:30 2014
;; MSG SIZE  rcvd: 73

> dig -x 10.100.208.10

; <<>> DiG 9.9.2-P2 <<>> -x 10.100.208.10
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 6088
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
 
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;10.208.100.10.in-addr.arpa.   IN      PTR
 
;; AUTHORITY SECTION:
10.IN-ADDR.ARPA.        86400   IN      SOA     10.IN-ADDR.ARPA. . 0 28800 7200 604800 86400
 
;; Query time: 0 msec
;; SERVER: blah, blah
;; WHEN: Wed Jun 25 09:49:56 2014
;; MSG SIZE  rcvd: 106

That is seriously bad and wrong – for us! This is a cache-only server and there are indeed RFC-1918 addresses defined on internal nameservers, such as that 10.100.200.10.

An email relay which relies on reverse lookups started to fail.

A DuckDuckGo search did not show anything relevant. Maybe Google would have.

I ultimately registered for an account at the knowledge base at isc.org, kb.isc.org, and quickly found my answer.

In fact they were crystal clear in explaining this very problem, so I hesitated to document it here, but I figure others might leap first and then read the documentation later like myself so it might do someone else some good.

They say:

...
Although this will be effective as a workaround, administrators are urged not to just specify empty-zones-enable no;
 
It is much better to use one or more disable-empty-zone option declarations to disable only the RFC 1918 empty zones that are in use internally.  
...

That empty-zones-enable no; by the way is a configuration option you can toss in your main configuration file in the options section.

Case closed.

Conclusion
Our reverse lookups on the Intranet began to fail after an innocuous upgrade of the ISC bind nameserver to version 9.9. A simple addition of an extra configuration statement resolved the matter. I guess it really is a good idea to RTFM.

References
ISC’s site is www.isc.org.
A different type of DNS clobbering was described in this case.
A word about Google’s awesome public DNS service.