<html><head></head><body>Without taking the time to examine these carefully, I'd guess that those domains are being served off less-than-stellar DNS servers, and the fault is likely not at your end.<br>
There's a disgustingly-large % of DNS service in the wild that's outright held together with chewing gum and bailing twine... bad glue records in particular are a problem.<br>
Examine the chain of authoritative servers for each and I'll bet you find some commonalities.<br>
Also there are dozens of DNS "lint" tools that will help you track down other people's errors as well as your own.<br>
Best guess without testing: domain has 3-4 servers listed at gTLD, only 2-3 of those are authoritative for the domain, and something along the line has an illegally-short TTL.<br>
-Adam<br><br><div class="gmail_quote">On April 20, 2016 2:03:59 AM CDT, Trevor Cordes <trevor@tecnopolis.ca> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre class="k9mail">Some recurring email bounces have tipped me off to a really strange DNS <br />issue on the boxes I admin. I really need help on this as it's impacting <br />a real customer in a real way: delayed emails.<br /><br />Often, and somewhat randomly, DNS will fail to resolve a domain. <br />Sometimes it will fail to resolve it for >4 hours and trigger a diagnostic <br />bounce from my sendmail.<br /><br />When I run tests at the command line (dig) on the domains in question (the <br />ones I've seen in email bounces) I will often very quickly get a resolve <br />failure, but usually 5s later another dig to the same domain will resolve <br />100% ok!<br /><br />Every box I am testing on has a similar config with BIND named 9.10 (9.8 <br />on one box) running as the local recursive resolver. /etc/resolv.conf on <br />all is <a href="http://127.0.0.1">127.0.0.1</a>. So that means every lookup that isn't cached is going <br />to the root NS's.<br /><br />When it
fails to resolve, named.log logs an entry like:<br />20-Apr-2016 00:37:28.276 query-errors: debug 1: client <a href="http://127.0.0.1">127.0.0.1</a>#33971 (<a href="http://artscouncil.mb.ca">artscouncil.mb.ca</a>): query failed (SERVFAIL) for <a href="http://artscouncil.mb.ca/IN/A">artscouncil.mb.ca/IN/A</a> at query.c:7769<br />20-Apr-2016 00:37:28.276 query-errors: debug 2: fetch completed at resolver.c:3658 for <a href="http://artscouncil.mb.ca/A">artscouncil.mb.ca/A</a> in 0.215778: SERVFAIL/success [domain:<a href="http://artscouncil.mb.ca">artscouncil.mb.ca</a>,referral:1,restart:3,qrysent:2,timeout:0,lame:0,neterr:2,badresp:0,adberr:0,findfail:0,valfail:0]<br /><br />For manual tests it only logs one, for the real-life sendmail problem <br />ones, I'll see dozen/hundreds of the same thing trying hour after hour <br />(usually around the sendmail queue retry times).<br /><br />One of my boxes is *extremely* well connected in the US, and while it <br />seems to have errors
slightly less often, it still has them. All the rest <br />are on various levels of Shaw or MTS, res or business.<br /><br />This seems to have just started popping up maybe 6 months ago. It feels <br />like it's getting worse.<br /><br />I've setup an easy test, on the actual domains with the most problems:<br /><br />rndc flush<br />dig +short <a href="http://sportmanitoba.ca">sportmanitoba.ca</a><br />dig +short <a href="http://gymcan.org">gymcan.org</a><br />dig +short <a href="http://brandoneagles.ca">brandoneagles.ca</a><br />dig +short <a href="http://interactivegym.org">interactivegym.org</a><br />dig +short <a href="http://artscouncil.mb.ca">artscouncil.mb.ca</a><br /><br />rndc flush<br />dig +trace <a href="http://sportmanitoba.ca">sportmanitoba.ca</a><br />dig +trace <a href="http://gymcan.org">gymcan.org</a><br />dig +trace <a href="http://brandoneagles.ca">brandoneagles.ca</a><br />dig +trace <a href="http://interactivegym.org">interactivegym.org</a><br />dig +trace
<a href="http://artscouncil.mb.ca">artscouncil.mb.ca</a><br /><br />Maybe some others can run those tests on their boxes (but only if you're <br />running BIND as caching resolver, which many/most people won't be).<br /><br />Here's where it gets interesting the +short tests I can get to fail at <br />least 1 of the domains (at random) about 1/8 of the time! On at least 5 <br />different boxes out there! But +trace has never failed once on any box or <br />any domain. It's like +trace does something different, maybe slowing the <br />process down or something, that allows it to always succeed. (Failure for <br />the +short is a missing line in output, +trace you have to look at the <br />domain/ip returned near the bottom.)<br /><br />So, is it just these particular domains?? Something wrong on their (DNS) <br />side? Or is it more domains, not just these? Is there any way to <br />diagnose what exactly is failing? I find it bizarre that *all* of these <br />domains
regularly go down for 4+ hours causing an email bounce!?! Or is <br />there something horribly wrong on my BIND caching DNS servers?<br /><br />Perhaps they are slow, and I'm my BINDs are just not waiting long enough. <br />Is there a way to tell BIND to be more patient waiting for DNS packets to <br />come in?<br /><br />Maybe it's something regarding IPv6? (I'm doing this all in IPv4 and have <br />no current interest in 6. And I'm only looking for A records.)<br /><br />I have packet traces of the above sample commands when the lookups fail, <br />but I can't really figure out what it's doing, other than one boatload of <br />traffic for a tiny dns query. I can provide a trace privately on demand <br />if you think you can help.<br /><br />Even more odd, I never seem to have a problem with interactive things like <br />web browsing. If this was a problem with all domains, I should see this <br />in firefox all the time, but I don't. Maybe Firefox doesn't even obey <br
/>resolv.conf and does its own thing, or retries heavily itself?<br /><br />I also checked to ensure my iptables aren't dropping packets related to <br />this.<br /><br />Lastly, answers of "just use <a href="http://8.8.8.8">8.8.8.8</a>" aren't helpful because I also need <br />to handle dynamic local, and in some cases, external DNS (often with <br />multiple views), all in the same BIND/box (and I like uniformity across <br />boxes for ease of admin). Sure, I could try another resolver, but I see <br />no reason BIND can't be made to work, as it has for me for 20 years. And <br />if this is a BIND bug, I want to submit it to help solve it.<br /><br />Thanks guys!<br /><hr /><br />Roundtable mailing list<br />Roundtable@muug.mb.ca<br /><a href="http://www.muug.mb.ca/mailman/listinfo/roundtable">http://www.muug.mb.ca/mailman/listinfo/roundtable</a><br /></pre></blockquote></div><br>
-- <br>
Sent from my Android device with K-9 Mail. Please excuse my brevity.</body></html>