Showing posts with label DNS Latency. Show all posts
Showing posts with label DNS Latency. Show all posts

Wednesday, November 16, 2016

Custom Domain Publishing, And DNS Latency

The Internet is diverse and large - and the Internet directory system, aka the DNS Infrastructure, supports the diversity and size.

When we setup a new domain, the registrar frequently advises us to wait patiently, before continuing.
Congrats on your new domain! Now, we suggest that you wait 24 - 48 hours for the new domain to propagate, before trying to publish to the domain.

When we update a domain - and add or delete a single DNS address - we won't always get advice about waiting, from the registrar.

Responsible helpers will occasionally provide advice to be patient, in Blogger Help Forum: Get Help with an Issue.

After you correct the addresses, please wait 8 hours (2 x "14400" seconds) before continuing. This will let all DNS servers on the Internet receive the corrected addresses.

This advice is generally given, for domains that use a 4 hour ("14400" second) or greater Time To Live or TTL.

Many registrars use a 1 ("3600") or 2 ("7200") hour TTL, for individual domain DNS addresses. Domains with 2 hour or less TTL generally do not require latency advice, because most forum topics take more than 2 - 4 hours to complete.

Adding a domain - and updating a domain - involve different latency.

The difference between "new domain" and "domain update" advice - and irregular presence of either - causes confusion.

Some advice about domain updates involves unnecessary waiting. In other cases, new domain owners jump right in, and try using their new domains, immediately - and regret later.

Moving too quickly, with Blogger custom domain publishing, can lead to Google database corruption. Fear of the apocryphal "Another blog ..." error - and similar Blogger access disruptions - leads many technical helpers to routinely advise unnecessary waiting periods.

People who make changes, and see their changes work immediately, cause similar confusion. Somebody who is advised to wait 2 to 4 hours, before publishing the blog to the domain - and who observe that the domain, perversely, is immediately operational - may misunderstand.

Remember that everybody who wants to read a blog won't have the same DNS service. Differences between one reader (or search engine), and another, will cause various confusions.

Waiting too long wastes time - but not waiting long enough can cause worse problems. For best results, there will be unnecessary waiting, sometimes.

Just be aware of the different latency periods - and try to provide relevant advice.

  • Domain addition latency can be 24 to 48 hours ("86400" to "172800" seconds).
  • Domain update latency can be 10 minutes, to 48 hours ("600" to "172800" seconds).

Domain addition latency can be 24 to 48 hours ("86400" to "172800" seconds).

Domain addition latency results from the different name servers - and server owners and update policies - all over the Internet. Any given name server updates their local domain master database periodically, by retrieving the zone file from the various master name servers.

If the domain master database on a given server is updated daily - and if your domain was setup 1 minute after the daily update, your domain will not be added to that server until the next daily update.

Now consider that there are thousands of name servers, owned by hundreds of registrars, ISPs, and other Internet services. All local name servers are updated on a schedule considered appropriate, by the owner of each service.

For best results, your domain needs to be indexed on any name server that might be used by any one of your readers - or any one of the search engines that provide necessary search results, for any one of your would be readers. Do you see the possibilities for confusion, for any new domain?

This justifies waiting "24 to 48 hours", so your new domain will be visible to everybody, at the same time. This strategy hopefully will prevent possible Google database corruption.

Domain update latency can be 10 minutes, to 48 hours ("600" to "172800" seconds).

Update of individual DNS addresses will generally be more prompt - and can be researched, using a Dig log or similar display. The TTL latency for any given DNS address - unlike the domain master updates - can be easily adjusted, and determined.

As an example, here is the DNS address which controls visibility of this blog, as shown in a Dig log.


See the "3600" for "blogging.nitecruzr.net"?



See the "3600" for "blogging.nitecruzr.net"? That tells us that "blogging.nitecruzr.net" updates on a 1 hour (3600 second) basis.

All things being equal, and every name server on the Internet properly updating, if I was to correct addressing for "blogging.nitecruzr.net", every name server would be issuing the updated Blogger addresses within 3600 seconds - and the average latency would be 1/2 hour. To increase reliability when advising address correction, I would advise
After you correct the addresses, please wait 2 hours (2 x "3600" seconds) before continuing. This will let all DNS servers on the Internet receive the corrected addresses.

Since most Blogger forum discussions about DNS addressing take place over hours or days, a 1 hour latency is generally inconsequential. Keeping it simple, I don't bother with a 1 to 2 hour latency, when providing address correction / update advice.

A 4 hour or greater latency, on the other hand, I generally mention in my advice.
After you correct the addresses, please wait 8 hours (2 x "14400" seconds) before continuing. This will let all DNS servers on the Internet receive the corrected addresses.

When adding a DNS address, as part of address correction, latency will be close to 0 seconds. If you need to retrieve an address for a new host, just defined by the blog owner, you'll either get it immediately from cache on the local name server, or from the local server requesting an immediate update from the domain master server.

Deletions and updates, which reference out of date / redundant addresses, are subject to address latency.

Understand the different latency numbers.

Additions are only relevant for NEW domains - and they lead to the "24 to 48 hour" advice, as the latency involves the Internet infrastructure in general. Address corrections are relevant for UPDATED domains, such as DNS address corrections - and latency is involved on an individual address basis.

By being more selective when we advise waiting "24 to 48 hours", we can provide more consistent and vigourous testing advice - and encourage more effective domain additions and updates. And this will be better for custom domain publishing, in general.

Need More Detail?

For more discussion, you can read:

And, try these DNS Propagation Checkers, for experimentation.




Not every Blogger blog owner understands the cause, and the effects, of DNS latency - or the detail that there are multiple latency figures, which affect custom domain publishing in various ways.

To better plan a custom domain update, it is good to understand the different causes of latency.

Wednesday, May 4, 2016

The DNS TTL Setting Is Chosen By The Registrar

Some blog owners, publishing their blog to a custom domain, wonder about the mysterious TTL setting for the domain.
What is "3600"? Why do I see "7200" in my Dig log?
and
Why do I have to wait 8 hours, after changing my DNS addresses?
DNS latency (aka "TTL") is not understood, by many blog owners. A mistake in setting TTL can cause anger and frustration.

"Time To Live" aka "TTL" is a registrar individual setting, that is used for providing proper performance from the registrars name servers and network.

There are multiple TTL settings - and each can have a different effect on your domain.

TTL controls caching of the DNS addresses, used to access your domain.

TTL indicates how long a DNS address will remain in cache, on a computer, before that computer requests an updated value. It's an essential (but obscure) component, in setting up a custom domain.

Since you're reading this blog, your computer has the address of this blog in cache - and won't be requesting an update, until "TTL" for "blogging.nitecruzr.net" expires. GoDaddy, the registrar for "nitecruzr.net", uses "3600" seconds, or 1 hour TTL.

Your computer probably uses the nameservers provided by your ISP - and your ISPs nameservers get updates from the nameservers provided by GoDaddy. Changes to "nitecruzr.net" go from Google, to GoDaddy, to your ISP, to your computer - and TTL applies to each DNS cache.

A typical Dig log, for a domain with a low TTL value.

Look at a Dig log extract, for "nitecruzr.net".

http://www.digwebinterface.com/?hostnames=nitecruzr.net%0D%0Ablogging.nitecruzr.net&type=A&useresolver=8.8.4.4&ns=auth&nameservers=

nitecruzr.net. 3600 IN A 216.239.32.21
nitecruzr.net. 3600 IN A 216.239.34.21
nitecruzr.net. 3600 IN A 216.239.36.21
nitecruzr.net. 3600 IN A 216.239.38.21
blogging.nitecruzr.net. 3600 IN CNAME ghs.google.com.

You set the TTL, for each individual DNS address, when you setup your domain. The registrar provides a recommended TTL - but most registrars let you select a different setting, within a given range.

A lower TTL ("600" seconds or 10 minutes) provides faster refresh after a change is made - but puts more load on the registrars name servers, and requires a more responsive network.

A higher TTL ("7200" or 2 hours) provides slower refresh - and puts less load on the registrars name servers. And when a change is made, by Google, a higher TTL can cause problems.

If you don't understand DNS, you will be better off not changing from the registrar recommended setting. Some registrars won't even have a TTL setting, in their zone editor - and others may hide it, behind an "Advanced" menu or similar.

If you generate a Dig log for your domain, you'll see the TTL settings for your addresses.

A typical Dig log, for a domain with a high TTL value.

Look at a Dig log, for "rockchickenz.com".


Here, the current DNS addresses (correction pending).




Here, we see a TTL of "14400" (displayed as "14399") - or 4 hours.



rockchickenz.com. 14399 IN A 78.137.164.51
www.rockchickenz.com. 14399 IN CNAME ghs.google.com.

For best results, we suggest that you wait 2 x TTL, after making DNS changes.

I recommend waiting 2 x "TTL", after making any DNS changes, before acting from those changes. Having diagnosed the painful "Another blog ..." domain database corruption, too many times, I suggest this whenever possible.

After you correct the addresses, please wait 8 full hours (2 x "14400" seconds) before continuing. This will let all DNS servers on the Internet receive the corrected addresses.

In many cases, TTL will be "3600" or "1 hour" - and with a typical forum diagnostic session taking several hours between posts, a one or two hour TTL is transparent to the forum discussion.

With domains using 4 hour TTL, suggesting that the blog owner wait 8 full hours after making the change, before re publishing the blog to the domain, makes sense. And a 2 full days, for "86400" seconds, makes even more sense.

And getting a righteous DNS address complement verified - before re publishing - makes the most sense.



DNS TTL latency, which affects accessibility of every #Blogger custom domain published blog, is not understood by many blog owners. To provide a stable domain - and a more visible blog after it's published to a custom domain - every blog owner should understand this obscure detail.

Tuesday, March 22, 2016

Custom Domain Publishing, And World Culture

People who speak (read / write) languages, that do not use Roman character sets, need to publish in their native language - and the Internet now supports that need.

Some registrars will register domains which use non Roman character sets, in the URL. Not all Internet services will accept non Roman characters, however.

My favourite online DNS diagnostic services, DigWebInterface, and intoDNS, and Rex Swain's HTTP Viewer, only use ASCII. One cannot use non ASCII ("non Roman") characters, with either service.

"بحث-سناب.com", as converted, shows an example of an Internationalized domain name.

To use DigWebInterface to retrieve DNS addresses, and intoDNS to check the domain setup, we have to convert the native URL to Ascii.

I use three reliable online services, which will convert URLs to ASCII.


Converting "بحث-سناب.com" to ASCII involves use of one of the latter services.


DomainTools




WhoIs - Identity for everyone




Who.is - Powered by Name.com



Using the 3 tools, we see that "بحث-سناب.com" == "xn----zmcbcml8b0j.com", "XN----ZMCBCML8B0J.COM", and "xn----zmcbcml8b0j.com", respectively. Look carefully, in the body of the display for each service, for the IDN equivalent.

We can then use "xn----zmcbcml8b0j.com", with DigWebInterface, and intoDNS - and get the necessary diagnostics.

And using the latter tools, we see a properly setup domain - which is now being used for a properly published Blogger blog, verified in Rex Swain.


DigWebInterface




intoDNS




Rex Swain's HTTP Viewer



And when assistance is requested, in Blogger Help Forum: Get Help with an Issue, we can use these tools with non Roman character URLs.

There may be limitations, though.

Here is a blog published to "www.राजभाषाअनुभाग.भारत" - aka "www.xn--l1b0bn3cxacq2d2ccbd1b.xn--h2brj9c".







The IDN concept has now led to another domain name concept - "emoji" domains.

Some people have taken the ability to setup a domain, using non Roman characters, to another level. How about a domain name, which includes a smiley face?

This is a limited use feature. Maybe, the limit will save us?

The availability of emoji domains is limited. As of August 2017, there are eight top-level domains for which registration is possible, all of which are ccTLDs: .ai, .cf, .ga, .gq, .ml, .tk, .to, and .ws.

This is intriguing - but is it useful?



If people don't know how to "type" it, how will new readers access the blog? Will search engines index it? Does the Blogger "Create a blog" wizard actually support it?

The jury is still out, here. ".whatever" vanity TLDs started this trend. I'm not sure where this will go, with TLD "emoji" domains.

Houston, we may have a problem with TLDs that are IDN encoded.

For a TLD that is in English ("بحث-سناب.com"), both of these work:

http://www.digwebinterface.com/?hostnames=xn----zmcbcml8b0j.com%0D%0Awww.xn----zmcbcml8b0j.com&type=A&useresolver=8.8.4.4&ns=auth&nameservers=

https://intodns.com/xn----zmcbcml8b0j.com

For a TLD that is IDN encoded (राजभाषाअनुभाग.भारत"), we have mixed success.

This works:

http://www.digwebinterface.com/?hostnames=xn--l1b0bn3cxacq2d2ccbd1b.xn--h2brj9c%0D%0Awww.xn--l1b0bn3cxacq2d2ccbd1b.xn--h2brj9c&type=A&useresolver=8.8.4.4&ns=auth&nameservers=

This does not work:

https://intodns.com/check/?domain=%E0%A4%B0%E0%A4%BE%E0%A4%9C%E0%A4%AD%E0%A4%BE%E0%A4%B7%E0%A4%BE%E0%A4%85%E0%A4%A8%E0%A5%81%E0%A4%AD%E0%A4%BE%E0%A4%97.%E0%A4%AD%E0%A4%BE%E0%A4%B0%E0%A4%A4



The Internet now supports use of non ASCII characters, in URLs. In order to publish #Blogger blogs to Internationalised Domain Names, we need to convert native URLs to ASCII - using any one of three identified online DNS services.


https://productforums.google.com/forum/#!category-topic/blogger/qKWZ0807m60

https://productforums.google.com/forum/#!category-topic/blogger/L1pv6CPUK18

Wednesday, November 19, 2014

'Good Enough' Never Is, With Custom Domains

This month, we're seeing an interesting collection of problem reports, in Blogger Help Forum: Get Help with an Issue, about custom domain setup.
Why is my blog offline - occasionally?
And
Why do some (just some!) of my readers tell me they can't see my blog?
Investigating the problem, we see that the domain is, indeed online - some of the time.

Investigating the DNS addressing, robustly, we find an all too frequently seen set of name servers:
mydomain.com. 14400 IN A 216.239.32.21
www.mydomain.com. 14400 IN CNAME ghs.google.com.
What we see here is a domain, setup by someone who thought that one server would be 'good enough' to get by.

One server will, sometimes, get by - for a while, and for some people.

What some blog owners fail to realise, though, is that DNS service is never the same, all over the world (or the Internet). People in some places will use one DNS name server - and people in other places will use another.

And similarly, DNS service is never the same, from day to day. Today, you might need the services of one DNS name server, and tomorrow you might need another.

This month, we're seeing a number of reports that suggest that the Google name server at "216.239.32.21" is not responding for everybody. Now - don't everybody, who has a custom domain published Blogger blog start spitting coffee, and calling their registrar for emergency support.

People who have a properly setup domain should not worry, needlessly.

mydomain.com. 14400 IN A 216.239.32.21
mydomain.com. 14400 IN A 216.239.34.21
mydomain.com. 14400 IN A 216.239.36.21
mydomain.com. 14400 IN A 216.239.38.21
www.mydomain.com. 14400 IN CNAME ghs.google.com.

People with domains that provide a robust set of addresses are in better shape. The odds that all 4 name servers will fail, simultaneously, is near zero. It's much better than those with one name server, that is 'good enough for me' - and that, right now, provides the basis for this post.

Really, folks.

mydomain.com. 14400 IN A 216.239.32.21
www.mydomain.com. 14400 IN CNAME ghs.google.com.

One "A" name server was good enough for you, last month. It's not, now. How many of your readers, since last month, saw

404 Not Found

How about your search reputation. What happens, when your blog is being indexed, and the search engine gets

404 Not Found

Neither your readers, nor the search engines, appreciate

404 Not Found

And both will react, negatively - and you won't like the results.

'Good enough' isn't - for everybody, all of the time. And this affects the reputation of your blog, both with people, and search engines.

Thursday, September 25, 2014

Why Are Some Custom Domain Published Blogs Inconsistently Online?

I've been using affinity analysis, and differential analysis, as tools for diagnosing intermittent problems with custom domain published blogs, for several years.

Some people may be using a browser which is more forgiving of DNS problems than others - and people who use the right browser, when accessing a domain with righteous DNS addresses, should expect to generally get more consistent service. But that's only a small part of the story.

DNS is the basis for custom domain publishing - and DNS is used in a number of ways. This creates a number of bases for inconsistencies.
  • Inconsistencies From The Browser
  • Inconsistencies From Individual DNS servers
  • Inconsistencies From Large ISP DNS Servers
  • Inconsistencies From The Google Name Servers
  • Inconsistencies From WorldWide DNS Services
  • Inconsistencies From The WorldWide Internet Infrastructure

The Browser
Some browsers, if they have a problem getting an IP address for the domain root (using the 4 x "A" servers) may automatically use the "www" alias.

If your domain provides only 1 x "A" address, it may be more susceptible to problems, when this aliasing of the domain root and "www" host are in use - since an outage which involves 1 "A" server may be more common than an outage which might involve all 4 x "A" servers simultaneously.

If your domain is not setup with both the domain root and "www" host properly addressed - or if the domain root to "www" host redirect is not enabled, the domain may not perform consistently.

Individual DNS servers
No people (you, or your readers) access the domain registrar's servers directly, to get the address of your domain. You use "local" DNS servers - the DNS severs provided by your ISP (or maybe a custom third party service like like GoogleDNS or OpenDNS). The "local" DNS servers that you use retrieve DNS addresses from the "authoritative" DNS servers provided by the registrar. The registrar servers reference the 4 x "A" / "CNAME" servers, provided by Google, using "referral", to get the address of your blog.

The servers that you use only access the registrars servers periodically, thanks to DNS cache. That's the mysterious TTL, which your domain's authoritative DNS servers specify. If the address for your domain is in cache, on any local DNS server, and cache is not expired (the address was retrieved any time in the previous TTL period) the local DNS server that you use simply issues what it already has.

Since almost every different reader of your blog uses a different "local" DNS server, you're going to see inconsistency in problem reports, from your readers.

Large ISP DNS Servers
If your domain is popular, people accessing your domain from a large ISP - or an ISP that is close to you geographically (if your readers are geographically concentrated) - is more likely to have the DNS addresses, for your domain, in cache. In this case, no retrieval from the registrar's servers will be necessary. The higher the TTL, the greater the chance this will be the case.

The Google Name Servers
If your readers are accessing your domain using the "www" alias, they are using the "ghs.google.com" name server for accessing your blog. The design of "ghs.google.com" means that everybody on the Internet, accessing any Blogger blog published to a custom domain, uses the same DNS addresses, cached locally. This too means that large ISPs - who are more likely to have somebody (if not you) accessing some Blogger blog published to a custom domain - are more likely to have the address for your blog in cache.

Your blog, and your domain, are addressed separately, and require separate DNS servers. This is why a righteously addressed domain uses "CNAME" referral, when mapping your domain to your blog. And this is why you have to purchase (or setup) DNS hosting, when you register your domain.

WorldWide DNS Services
Some of the 4 x "A" and "CNAME" servers, provided by Google, are not completely unique. Many large DNS infrastructures - like those provided by eNom, GoDaddy, and Google - have "unique" IP addresses replicated worldwide, using "Anycast DNS". This feature can sometimes create regionally concentrated outages, with some domains - as we have seen periodically, with eNom.

The WorldWide Internet Infrastructure
The combinations of browser used, domain popularity, ISP size (customer population), and regional outages, will always cause discrepancies between domains, and people in different locations and using different ISPs. What you (one person) see, where you are, may be completely different from what one of your readers (or my readers) sees, when using a different ISP, in another country, and a different browser.

Thanks to the transient nature of IP networking, you may "test" or observe problems over minutes or hours - but many DNS problems can appear and disappear in seconds. That''s in spite of what you would expect from cache / TTL latency.

All of these seemingly insignificant details, when fully understood, may help to explain why I continually insist on using righteous DNS addresses, whenever a custom domain publishing problem is encountered. Righteous addresses won't necessarily solve all of the above problems - but it may eliminate some of the inconsistencies, and make it possible that we could, eventually, diagnose the problem.

>> Top

Friday, March 12, 2010

Diagnosing Problems With Custom Domains: An Alternative Dig Tool

As I wrote earlier, when I'm diagnosing a problem with a custom domain - and occasionally, with other blog and network problems - my most frequently used tool is a Kloth online Dig. With Dig being an important diagnostic procedure, it's a good idea to have more than one Dig utility available.
  • The Kloth server may be down.
  • The Kloth server may not offer the right options.
  • It's good practice to occasionally (and sometimes, intentionally) cross check results against another server.

For many Dig investigations, I like to use a second online Dig service - WhoIs, provided by All-NetTools. The All-NetTools WhoIs Dig actually provides more than a standard Dig. A WhoIs Dig, for a domain, provides much valuable information about the domain / registrar relationship. Here, for instance, we can see an All-NetTools WhoIs Dig for this domain, nitecruzr.net.
http://www.who.is/whois/nitecruzr.net/

REGISTRY WHOIS FOR NITECRUZR.NET

Domain Name: nitecruzr.net

Registrar: GODADDY.COM, INC.
Whois Server: whois.godaddy.com
Referral URL: http://registrar.godaddy.com
Status: clientDeleteProhibited, clientRenewProhibited,
clientTransferProhibited, clientUpdateProhibited

Expiration Date: 2010-03-24
Creation Date: 2008-03-24
Last Update Date: 2009-05-01

Name Servers:
ns11.domaincontrol.com
ns12.domaincontrol.com
ns53.domaincontrol.com
ns54.domaincontrol.com


Here, we see useful information about this domain, including the fact that it expires within the month. In cases where the domain in question is seen to be parked, knowing that a domain recently expired helps us advise a blogger that it's necessary to contact the registrar immediately, and avoid loss of domain registration.

A an All-NetTools DNS Dig against nitecruzr.net provides still more interesting and useful information.

http://www.who.is/dns/nitecruzr.net/

The WhoIs SOA Dig shows us the SOA record.
NITECRUZR.NET SOA RECORD
Name Server ns11.domaincontrol.com
Email (masked)
Serial Number 2009121603
Refresh 8 hours
Retry 2 hours
Expiry 7 days
Minimum 1 day

Here, we see the name of the domain master server, and the various expiry times for the domain.

The Whois DNS Dig shows us the well known host records.
NITECRUZR.NET DNS RECORDS
Record Type TTL Priority Content
mail.nitecruzr.net CNAME 1 hour ghs.google.com
nitecruzr.net A 1 hour 216.239.36.21 (Mountain View, CA, US)
nitecruzr.net A 1 hour 216.239.32.21 (Mountain View, CA, US)
nitecruzr.net A 1 hour 216.239.34.21 (Mountain View, CA, US)
nitecruzr.net A 1 hour 216.239.38.21 (Mountain View, CA, US)
nitecruzr.net MX 1 hour 10 aspmx.l.google.com
nitecruzr.net MX 1 hour 20 alt1.aspmx.l.google.com
nitecruzr.net MX 1 hour 30 alt2.aspmx.l.google.com
nitecruzr.net MX 1 hour 40 aspmx2.googlemail.com
nitecruzr.net MX 1 hour 50 aspmx3.googlemail.com
nitecruzr.net NS 1 hour ns11.domaincontrol.com
nitecruzr.net NS 1 hour ns12.domaincontrol.com
nitecruzr.net NS 1 hour ns53.domaincontrol.com
nitecruzr.net NS 1 hour ns54.domaincontrol.com
nitecruzr.net SOA 1 day ns11.domaincontrol.com. dns.jomax.net. 2009121603 28800 7200 604800 86400
www.nitecruzr.net CNAME 1 hour ghs.google.com

This will never replace the Kloth Dig log, completely. The Kloth server offers many more options, such as the ability to selectively Dig against aliases besides the domain root and well known aliases. It is a worthy complement to a Kloth Dig in many cases, and can be used as a backup or cross check.

>> Top

Navigate» Become author for this Blog