Custom Domain Redirection, And "A" Referral Servers

One of the advantages of custom domain publishing, and the domain root to published URL redirection, used to be transparent redirection of the URLs.

As Blogger blog owners become more sophisticated with their blogs, and custom home pages (aka "landing pages"), become more popular, we have blog owners with concerns with home page access and the root redirection. We are also seeing concern with direct access to specific posts - with posts advertised, using the domain root URL.

In some cases, redirection only shows the reader the blog home page when the casual reader first sees the blog - even with a specific post being linked in the advertised URL.

With this blog, which is the redirect target of my domain, nitecruzr.net, this post "Custom Domain Redirection, And "A" Referral Servers" has a URL of "http://blogging.nitecruzr.net/2018/08/custom-domain-redirection-and-referral.html".

Since "blogging.nitecruzr.net" is the redirect target of the root redirect for "nitecruzr.net", this post "Custom Domain Redirection, And "A" Referral Servers" should have an alternate URL of "http://nitecruzr.net/2018/08/custom-domain-redirection-and-referral.html".

Click on the 4 links, in the 2 paragraphs immediately above, and compare the results - if you don't yet see the seriousness here. Please. It's not a cosmetic detail.

This post "Custom Domain Redirection, And "A" Referral Servers", advertised as "http://nitecruzr.net/2018/08/custom-domain-redirection-and-referral.html", redirects to "http://nitecruzr.net/".

This post "Custom Domain Redirection, And "A" Referral Servers", advertised as "https://nitecruzr.net/2018/08/custom-domain-redirection-and-referral.html", redirects to "https://nitecruzr.net/".

Note that this blog does not yet use "HTTPS Redirect", so if you click on a link to "Custom Domain Redirection, And "A" Referral Servers" you will see "http://blogging.nitecruzr.net/2018/08/custom-domain-redirection-and-referral.html".

Here's an example of the problem.

Here's a an example of this problem - diagnosed using a previous post blogging.nitecruzr.net/2016/08/delete-permanently-means-delete.html

A normal redirect - to "blogging.nitecruzr.net/2016/08/delete-permanently-means-delete.html"

http://blogging.nitecruzr.net/2016/08/delete-permanently-means-delete.html

HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Expires: Mon, 23 Apr 2018 03:25:09 GMT
Date: Mon, 23 Apr 2018 03:25:09 GMT
Cache-Control: private, max-age=0
Last-Modified: Mon, 23 Apr 2018 03:23:01 GMT
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Server: GSE
Accept-Ranges: none
Vary: Accept-Encoding
Transfer-Encoding: chunked

A normal redirect - to the specific post, using HTTPS.



A broken redirect - to "blogging.nitecruzr.net"

http://nitecruzr.net/2016/08/delete-permanently-means-delete.html

HTTP/1.1 301 Moved Permanently
Location: http://blogging.nitecruzr.net
Date: Mon, 23 Apr 2018 03:23:18 GMT
Content-Type: text/html; charset=UTF-8
Server: ghs
Content-Length: 226
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN

http://blogging.nitecruzr.net

HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Expires: Mon, 23 Apr 2018 03:23:19 GMT
Date: Mon, 23 Apr 2018 03:23:19 GMT
Cache-Control: private, max-age=0
Last-Modified: Mon, 23 Apr 2018 03:23:01 GMT
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Server: GSE
Accept-Ranges: none
Vary: Accept-Encoding
Transfer-Encoding: chunked

A broken redirect - to the blog home page, using HTTPS.



Unfortunately, some blog owners are observing that redirection of a specific root URL - maybe a custom landing page, or a specific post advertised using the root URL - appears to lead only to the home page of the published blog.

This is a problem with domain / website design, in general - and with Blogger custom domain design, in particular.

The domain root to published URL (for most blogs, the "www" host) appears to use a version of frame forwarding, with some blogs being reported.

I never recommend using Frame Forwarding, in the custom domain DNS setup.

One of the disadvantages of frame forwarding is that redirects using frame forwarding drop the post URL.

We've been seeing this problem reported, for several years.

The problem, rarely - but not never - reported, started several years ago. This was before HTTPS for BlogSpot - and even farther before HTTPS for custom domains - was rolled out.

This was shortly after we started observing other problems involved, in the deployment of HTTPS.

The first HTTPS issue was reported in 2014. HTTPS for BlogSpot was rolled out in 2015.

I'm betting we first saw reports of the post URL being lost, in the ""http://xxxxxxx.whatever.com/whatever.html" -> ... -> "https://www.whatever.com"" was maybe 2 years ago. During deployment of HTTPS.

My guess is that Blogger Engineers upgraded the domain root servers ("A" address) redirection code, to support HTTPS - and made a business decision to use frame forwarding.

From what I've seen of Blogger Engineering in general - and custom domain design in particular - I am convinced that if they decided that frame forwarding is necessary, it's necessary. Until they can correct the problem, that is. My advice to every domain owner to not use registrar supplied frame forwarding in the domain DNS, notwithstanding.

I can only hope that we will see this problem corrected, this decade. Remain optimistic - and report the problem, when it can be identified.

---

It appears that some of the #Blogger custom domain "A" address name servers are using "Frame Forwarding", instead of "Referral", to redirect the domain root to the published URL. Frame forwarding strips away the post address portion of the URL - leaving only the domain root.

Some blog owners use the domain root to advertise their blog posts - and are seeing only the blog home page, instead of the individual posts, when linking to the posts.

https://productforums.google.com/d/topic/blogger/NtQHoVNJ5PM/discussion

https://productforums.google.com/d/topic/blogger/pePwPhr_H-Y/discussion

https://productforums.google.com/forum/#!category-topic/blogger/pePwPhr_H-Y

https://productforums.google.com/forum/#!category-topic/blogger/nr-cIIgjBGg

Custom Domains And HTTPS Redirection Code

As most of us know, Blogger HTTPS support does not include custom domain publishing.

The advantages offered by HTTPS access are widely advertised - and have led to envy between blog owners who publish to custom domains, and native BlogSpot blog owners proudly advertising their new HTTPS connectivity.

Long ago, we saw possibly malicious code which helps our readers avoid using country code aliases, to read our blogs from an aliased country. Recently, there was dodgy code which blocked HTTPS mode, to read a customised blog.

Now, we have custom code to force HTTPS access, for BlogSpot published blogs.

Along with providing code to help blog owners avoid country local domain aliasing, some marginally helpful hackers are providing code to help blog owners force reader access to HTTPS.

Some blog owners always wanted HTTPS to be used, to access their blog.

Some blog owners wanted their readers always using HTTPS to access their blogs, before forced HTTPS access became an option. They Googled, and found, semi helpful hackers who provide clever code to force the "HTTP --> HTTPS" redirection.

<script type='text/javascript'>
$(document).ready(function() { 
  $("a[href^='http://']").each( 
    function(){ 
      if(this.href.indexOf(location.hostname) == -1) { 
        $(this).attr('target', '_blank'); 
      } 
    } 
  );
  $("a[href^='https://']").each
    function(){ 
      if(this.href.indexOf(location.hostname) == -1) { 
        $(this).attr('target', '_blank'); 
      } 
    } 
  );
});
</script>

This is clever code - when only BlogSpot access is involved. When you add BlogSpot to custom domain redirection, it becomes another "404".



Adding this clever code is an excellent solution - until the blog owner forgets about it, and later upgrades to a non BlogSpot custom domain.

With a custom domain published blog, the redirection becomes a problem.

The added code contains no exception to permit custom domain published blogs to remain in HTTP mode. When accessing an otherwise properly setup custom domain published blog, from a reader using the "blogspot.com" URL, this prevents the BlogSpot to domain redirect from operating.

BlogSpot URLs, which should redirect to the HTTP published custom domain URL, instead redirect to a non existent HTTPS URL - and result in another "404". As the custom domain URL becomes more commonly used for a recently published blog, confusion increases when the rarer BlogSpot URL reference is encountered.

My blog has been using the domain URL for months, why is this happening now?

The problem involves dual redirection - to "https:" mode, and to the custom domain.

After painful problem diagnosis, we find the clever redirection code buried in template HTML - and we see that the blog reader is starting from the BlogSpot URL, and using the BlogSpot to domain redirection, to access the blog.

With blog access redirected to "https:" mode, then subsequently to the custom domain URL, the readers sees a "404" - because the custom domain URL is not available as "https:" content.

This problem will become increasingly rarer - but not extinct.

As self caused custom domain victims become rarer, this way of breaking ones own blog will become more obscure - and it's likely that some cases will go, unsolved. This will be similar to the problem of un migrated classic templates, which has increasingly less experienced support.

If you must install unsupported template tweaks into your template - consider the long term effects. Learn to recognise a problem that you have caused, to your own blog.

Not every helper will realise that you have added custom redirection code - and when looking at the problem code, when a problem is reported, will recognise it for what it is. Your problem may remain your problem - at least, until Blogger Engineering completes Blogger SSL integration (may this happen soon).

---

Some blog owners have added clever HTTP to HTTPS redirection code, acquired from helpful third party providers, installed in the template. When later publishing a blog to a custom domain, this code will prevent proper blog access - and as installed, may not be easily recognised.

https://productforums.google.com/forum/#!category-topic/blogger/bl5W39BKX4U

Blogger Magic - Custom Domain Redirects

Money is a popular artifact used in magic tricks - everybody loves looking at it, and playing with it.

The disappearing quarter - which starts out in your hand, and is found behind your ear - is intriguing. Equally as intriguing is the magician who can take two 50 cent pieces, and turn them into a dollar bill.

The two into one conversion is a neat trick - and that's part of a properly designed and properly setup custom domain.

1. The BlogSpot URL is redirected by URL forwarding, to the domain URL. This picks up the traffic from the BlogSpot blog, and transfers it to the domain. And, it picks up the identity of the domain.
2. The domain is redirected by "A" / "CNAME" referral, to Google (and within Google to the Blogger blog). This picks up the traffic from the domain (including the traffic forwarded from the BlogSpot URL), and transfers it to the Blogger blog.
3. "CNAME" referral retains the identity of the domain, with the traffic.

"CNAME" Referral merges traffic to the blog, using the domain identity.

What we have here is traffic (BlogSpot + domain) merged together, to the domain. Blog content gets indexed by the search engines under the domain URL - and contributes to page rank, and search engine reputation, for the blog, as published to the custom domain URL.

Identity * Traffic == Search Engine Reputation, for the domain.

Remember that the value of a custom domain is based on the increased reputation - to the blog, when seen under the domain URL.

Frame / URL Forwarding merges traffic to the blog, using the blog identity.

An alternative to "A" / "CNAME" referral, provided and recommended by some registrars, is Frame / URL Forwarding.

1. When re published to the domain, the BlogSpot URL is redirected by URL forwarding, to the domain URL. This picks up the traffic from the BlogSpot blog, and transfers it to the domain. And, it picks up the identity of the domain.
2. The domain is redirected by Frame or URL Forwarding to the BlogSpot URL. This picks up the traffic from the domain (including the original traffic, forwarded from the BlogSpot URL), and transfers it to the blog - and back to the BlogSpot URL.
   
3. Forwarding picks up the identity of the BlogSpot URL, with the traffic.

No Identity * Traffic == No Search Engine Reputation, for the domain.

All that work - and the identity (and search engine reputation) goes back to the BlogSpot URL.

If you pay extra to just merge traffic to the blog, what's the benefit?

In terms of magic, the latter trick is a dud. Nobody sees your domain. There goes your dollar bill, in a puff of smoke.

There's no benefit to the blog, if it's just seen under the BlogSpot URL.

Don't let your registrar sell you a puff of smoke!

If your registrar tries to tell you to use Frame or URL Forwarding, because they don't provide "CNAME" referral, or because they won't allow 4 x "A" referrals, your best choice is to find another registrar - or possibly, use a (free) third party DNS hosting service.

If you're here because you got technical advice to use URL Forwarding anyway, because it will work just as well as "4 x A" / "CNAME" referral, get better advice.

It's your domain - you want the benefits of publishing your blog to a domain URL? Publish it properly, using "A" / "CNAME" referral.

Custom Domains Use 301 Redirect, From BlogSpot

When you setup a Custom Domain, the setup wizard provides encouraging advice

We won't leave your readers behind!
http://myblog.blogspot.com will redirect to your custom domain.

Blogger accomplishes this bit of trickery, using standard Internet protocol - a 301 Redirect - but it's done in two steps.

Blogger redirects a Blogger blog, using two steps.

• When you publish your blog, "myblog.blogspot.com", to your custom domain, "mydomain.com", all internal links within the blog are changed, from "myblog.blogspot.com" to "mydomain.com".
• When the transition period ends 72 hours later, Blogger sets up a standard server based "301 Moved Permanently" for your blog.
• Note that only the blog internal links are changed - any links which you added, from post to post, are your responsibility. Likewise, all external links are your responsibility.

The search engine spiders, which previously indexed your blog as "myblog.blogspot.com", now index it as "www.mydomain.com".

Everybody who looks at the URL sees the blog published as the domain URL.

Similarly, your readers see "www.mydomain.com". To everybody, even those intending to look at the old BlogSpot URL, you now have "www.mydomain.com" for a blog URL.

The "301 Moved Permanently" referral from BlogSpot is similar to the "301 Moved Permanently" referral used in Custom Domain DNS setups, to equate "mydomain.com" to "www.mydomain.com".

In both cases, when this technique is used properly, the clients - either the people who read the blog, or the search engine spiders that index the blog - see the contents of the blog as being part of the redirect target. Now, they see "www.mydomain.com", instead of "myblog.blogspot.com".

The BlogSpot to domain redirect is installed in the Blogger server, not in the blog.

This "301 Redirect" is server based. If you put a meta tag, or JavaScript, based "301 Redirect" or "302 Redirect" into your blog, you will cause problems for your readers. Both are treated similar to a "302 Moved Temporarily", and your blog loses search engine weight.

In some cases, your readers will see

Too many redirects!

or a similar error, following a gratuitous redirection to block country local domain redirection, installed by you. In other cases, the BlogSpot URL simply does not operate.

The "Too many redirects!" error is more frequently seen, with "HTTPS" accessible blogs published to custom domains - and with the "HTTP" to "HTTPS" redirection. Here, too, a properly setup custom domain is essential.

Some non Google services, which use a URL to identify you, must be updated.

Note that this redirect applies mainly to search engines, following links. It won't apply to feeds, which need the updated URL in their database. It will apply to your readers - but readers who monitor the address window in the browser may become confused, if they see the new, non-BlogSpot URL where they are accustomed to seeing a BlogSpot URL.

You still need to notify your readers, and you still need to update all external fixed URL references to the blog, such as FeedBurner and other custom feeds, Google Webmaster Tools, and the various visitor meters.