dnssec implementation

[Dieselboy]

Does any of you have any experience with dnssec?

I am looking into this and seems to be two main parts (to try and put it simply)
1. As a DNS client, ie when you're browsing the web; dnssec ensures the requests from the WWW are legitimate for your PC
2. As a domain name, ie your domain as seen (eg your website)

I enabled dnssec for recursive internet dns today (point 1 above). This was as easy as connecting to one of the AD DNS servers in the domain and running a CMD line to get the dns server to connect via https and download the root trusts. There was information to do a refresh but it was taken care of automatically as was the replication of the trust to the other DCs. This process took <5 minutes while reading the documentation took a lot longer than that.

Now I am looking at number 2. I'll need to create some keys and apply them onto our domain dns on the WWW. Additionally, I might need to sign the same domain as issued from our internal DNS servers (company.com resolves internally in a split-brain fashion).

Any pointers or experiences seen by you guys?

[icecream-guy]

I'm about 2 months into being hostmaster here, been working the DNSSEC, unfortunately here's it's been fully automated previously, so all I do is run some script, and it's done.  I'll see if I can pull apart the script, and see what it is actually doing.  My task is pretty much, make a backup of the zone, make the changes to the zone, up the version, sign the zone, and wait for cron to restart bind.

Generally
Generate the Key Signing Key (KSK) and the Zone Signing Key (ZSK) for the specific zone file
Add the KSK and the ZSK to the bottom of the regular unsigned zone file.
Sign the zone with the keys KSK and ZSK.

[Dieselboy]

Thanks!

This morning I couldnt get to one of Cisco's Spark chat web pages which is usually a bit flaky anyway. It's a website that you log into that allows you to chat with customers on your website. The reason was because DNS lookup was failing to the domain. I'm not sure why yet but I had to initiate an active-refresh for the trust on the AD DNS server. As soon as I done that, the web page loaded fine.
DNSSEC has been enabled for around 18 hours and the refresh schedule is every 24 hours.

Ref: https://www.icann.org/dns-resolvers-updating-latest-trust-anchor

EDIT: this broke again so I checked the root hints and found that there was no A.root-servers.net so I added that. Secondly the resolved IP for B.root-servers.net was not as per IANA root hints file, so in AD DNS I clicked the edit button on the b.root-servers.net entry and then clicked the 'resolve' button which updated the IP address. I had a scan through the rest of the list and servers from J onwards were also not included so I added those. Issue is resolved once more.

[Dieselboy]

This is what keeps failing, looks like dnssec hasn't been set up for this domain so I am confused why it has been failing; unless the issue was with the root hint server.

[icecream-guy]

try some of them buttons at https://mxtoolbox.com/NetworkTools.aspx
it will help troubleshoot.

[Otanx]

mxtoolbox is awesome. I don't know how many times I have used it to show the problem isn't the network, and DNS, or DMARC/DKIM, or SPF, etc, etc is broke.

-Otanx

[Dieselboy]

This one is confusing me. The domain 'discovery.produs1.ciscoccservice.com' is not using dnssec at all (from what I can see using he online tools). So, I understood that being backwards compatible, the result should be returned as valid but is instead getting SERVFAIL:

Code Select Expand

$ dig @192.168.7.234 discovery.produs1.ciscoccservice.com A +dnssec

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @192.168.7.234 discovery.produs1.ciscoccservice.com A +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 23681
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4000
;; QUESTION SECTION:
;discovery.produs1.ciscoccservice.com. IN A

;; Query time: 4530 msec
;; SERVER: 192.168.7.234#53(192.168.7.234)
;; WHEN: Fri Jun 08 10:45:24 STD 2018
;; MSG SIZE  rcvd: 65

However if I query another dns directly:

Code Select Expand

$ dig @1.1.1.1 discovery.produs1.ciscoccservice.com A +dnssec

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @1.1.1.1 discovery.produs1.ciscoccservice.com A +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53162
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1452
;; QUESTION SECTION:
;discovery.produs1.ciscoccservice.com. IN A

;; ANSWER SECTION:
discovery.produs1.ciscoccservice.com. 5 IN CNAME produs1-haproxy-elb-1367689809.us-east-1.elb.amazonaws.com.
produs1-haproxy-elb-1367689809.us-east-1.elb.amazonaws.com. 60 IN A 34.237.28.144
produs1-haproxy-elb-1367689809.us-east-1.elb.amazonaws.com. 60 IN A 50.16.166.143

;; Query time: 847 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Fri Jun 08 10:59:30 STD 2018
;; MSG SIZE  rcvd: 166

So would look like a problem with my local dns server. However if I query a known dnssec working domain like verisign.com:

Code Select Expand

$ dig @192.168.7.234 verisign.com A +dnssec

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @192.168.7.234 verisign.com A +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64879
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4000
;; QUESTION SECTION:
;verisign.com.                  IN      A

;; ANSWER SECTION:
verisign.com.           52      IN      A       72.13.63.55
verisign.com.           52      IN      RRSIG   A 8 2 60 20180621220236 20180607220236 44245 verisign.com. YxopfEcqcFpBGIgPOpGglYwp3Fhvu+NZjCUWg+a0LB+hlnhJ4FSH7k12 +lAMFWKBp6ngjovxdGNuNYhVrYVZyto3gh2ZiLYqd2I0MxgdKo905LDk StYUDXFNz46qA8oPuh8JjapL25jfTzBhYzEC7+4w5E8gNZ/cgUA+JrTn 2rc=

;; Query time: 16 msec
;; SERVER: 192.168.7.234#53(192.168.7.234)
;; WHEN: Fri Jun 08 10:56:13 STD 2018
;; MSG SIZE  rcvd: 229

Event viewer logs don't really say much.

[Dieselboy]

Checking firewall logs and capturing traffic showed that the packet capture didn't contain the lookup to the problem domain. So I cleared the cache and re-ran the query and now the lookup is successful:

Code Select Expand

$ dig @192.168.7.234 discovery.produs1.ciscoccservice.com A +dnssec

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @192.168.7.234 discovery.produs1.ciscoccservice.com A +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33965
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4000
;; QUESTION SECTION:
;discovery.produs1.ciscoccservice.com. IN A

;; ANSWER SECTION:
discovery.produs1.ciscoccservice.com. 0 IN CNAME produs1-haproxy-elb-1367689809.us-east-1.elb.amazonaws.com.
produs1-haproxy-elb-1367689809.us-east-1.elb.amazonaws.com. 55 IN A 50.16.166.143
produs1-haproxy-elb-1367689809.us-east-1.elb.amazonaws.com. 55 IN A 34.237.28.144

;; Query time: 4851 msec
;; SERVER: 192.168.7.234#53(192.168.7.234)
;; WHEN: Fri Jun 08 11:40:20 STD 2018
;; MSG SIZE  rcvd: 166

But after less than a minute, retrying the query results in SERVFAIL again.

Found a bunch of people online having the same issue from back in 2015 but also found this forum https://forums.adobe.com/thread/1711111
That post is about a similar issue with Adobe's servers because a dns name is a cname for another domain name that's not authoritative.

[deanwebb]

So how are things going with this issue now? Still having failures, or is it all resolved?

[Dieselboy]

Had to turn off dnssec because Windows DNS was blocking a critical site we need to access daily. I was unable figure out why with certainty that the Windows 2012 server DNS server was blocking the dns. I'll need to come back to this at some point.

[deanwebb]

Are you able to test with a Linux box running a DNS daemon? You might get better results with a different implementation of DNS.

[icecream-guy]

Are you able to test with a Linux box running a DNS daemon? You might get better results with a different implementation of DNS.

Setting up bind can be a challenge in itself. 

[Dieselboy]

Are you able to test with a Linux box running a DNS daemon? You might get better results with a different implementation of DNS.

I think you might be correct. I do want to rule out any misconfig on my side. I don't want to run other dns servers though as I'm using Active Directory DNS

[icecream-guy]

Are you able to test with a Linux box running a DNS daemon? You might get better results with a different implementation of DNS.

I think you might be correct. I do want to rule out any misconfig on my side. I don't want to run other dns servers though as I'm using Active Directory DNS

you can run more than one DNS implementation.... you can do a master/slave kind of thing and setup zone transfers between to populate zones. But don't give out your DNS testing IP's or they will end up production.

[deanwebb]

But don't give out your DNS testing IP's or they will end up production.

TRUTH