We've been experiencing - and discussing - referer spam, since 2011.
We still see unaware blog owners, asking in Blogger Help Forum: Get Help with an Issue.
When I checked my stats for my blog, and looked at traffic sources, I noticed a link from a different country - and it just seems weird that Russian readers would then go to a US site.
When we explain that they're probably seeing another referer spam attack - and that clicking on the links is not always a good idea, we get a variety of responses. Some want to know if there is a purpose, to this noise.
Referer spam has a variety of purposes - ranging from commercial, to dangerous, to deceptive.
Garbage? Or a purpose?
Here are just 3 examples, from Blogger Help Forum: Get Help with an Issue.
- Advertise any paying customer.
- Lead you to a website with hacking content.
- Attack innocent third parties.
Advertise any paying customer.
This is the commercial possibility. If there's money to be made from publishing informative, interesting, and original content, there is probably lots more money to be made herding innocent third parties to ads on any website needing traffic - and lacking informative, interesting, and original content.
This may even represent a variation of GPT. And with the volume of referer spam that's possible, there are surely enough naive blog owners to make this a very lucrative activity.
Lead you to a website with hacking content.
This is the dangerous possibility. With hacking activity as a possible destination, clicking on a Stats link blindly is like playing "Russian Roulette", with your computer.
And, don't expect the website URL to provide a clue as to the destination. "www.innocous-name.com" could itself be hacked - and might unknowingly serve content from "www.hacking-website.com" - or redirect to "www.hacking-website.com".
Attack innocent third parties.
This is the deceptive possibility.
My blog was a "victim" of referer spam attack, in 2011. I have seen similar referer spam reports, that suggest this is not an unusual use of referer spam.
Protect yourself - if you must investigate your referers.
Surely, there are still actual people surfing - and some referer links are genuine. Eventually, you will want to check out some of the more intriguing URLs, in "Traffic sources".
If you decide to investigate one of the links, copy the text of the link URL - then use a proxy server.
Just don't investigate links, without protection. That's what proxy servers are for.
Some #Blogger blog owners want to know if referer spam has a purpose - or if it is simply random noise. It actually has a variety of purposes - commercial, dangerous, and deceptive.
Occasionally, we see odd questions about Stats, and the history of someone's blog (or maybe, the URL of someone's blog), in Blogger Help Forum: Get Help with an Issue.If I started my blog last year, why does my Stats display show pageviews from 2, or even 3, years ago?
andIf I rename my blog to a better URL, how do I carry the Stats numbers to the new URL?
Neither blog owner shows an accurate understanding of the historical nature of Stats pageview counts.
The Blogger servers record access activity by URL - not by blog.
Stats extracts pageview counts as needed, by URL, from the Blogger server logs. If the URL of your blog is "myfineblog.blogspot.com", and you request pageview counts for your blog, for "All time", you will see pageview counts against "myfineblog.blogspot.com", for Stats since May 2006 (as currently the case).
If the URL of your blog was "myexcellentblog.blogspot.com" until you renamed the blog to "myfineblog.blogspot.com" 6 months ago, you will see historical pageview counts for "myfineblog.blogspot.com" - which will include your blog, starting 6 months ago. You won't see pageview counts for "myexcellentblog.blogspot.com", from a year ago, because you'll be seeing historical pageview counts for "myfineblog.blogspot.com".
If someone else had a blog, published to "myfineblog.blogspot.com", before you renamed your blog to its current URL, your Stats displays could include pageview counts reflecting access to that blog. If "myfineblog.blogspot.com" was never used before you renamed your blog to that URL, it's possible that your pageview counts include attempted access to a non existent URL.
If you're using Google Webmaster Tools with your blog (and you should be doing that), you can check the access logs for "404 Not Found" events in the WMT logs. Just as a tree, falling in the forest, makes a sound even when nobody is around, so can access attempts exist against URLs where no blog is published.
Some referer spam appears to be sent to all likely URLs, ignoring whether or not a blog actually exists, at that URL. Referer spam is not unique to Blogger, and has been a problem since before Blogger became a major player in the Internet world.
If you just started a blog, or just renamed your blog to its current URL, your pageview counts can reflect historical referer spam, against your current URL, from before your blog was published to the current URL. As Blogger identifies specific referer spam campaigns, and eliminates some referer spam from Stats logs, this is one more possible cause of fluctuations in pageview counts.
If the possibility of seeing bogus pageview counts, for your blog as published to the current URL, does not please you, I will again point out that you'll get more out of your blog if you spend less time worrying about the details of the Stats displays, and more time working on your blog.
Look at activity before the blog was published as "noise", and look at current activity as "signal" - and work on improving the "signal to noise" ratio. Be aware of the value of your blog, and work on improving the value.
>> Top
That is the unfortunate truth.
Every day, some new member of Blogger Help Forum: Something Is Broken asks, innocentlyWhat is all this traffic from dodgy websites?
and after we explain what the dodgy traffic is, and why it does not reflect real traffic, the next question isSo why doesn't Google block it? Why should I have it polluting my Stats displays, and be unable to find actual traffic in my counts?
and the unfortunate truth is simply that Google cannot block it, because it's not significantly different from normal traffic - and the insignificant difference is not easily detected.
Referer spam cannot be identified, because it is identical in structure to legitimate Stats pageviews - and because its content changes, constantly.
What does a normal blog page "pageview request" look like?
When you click on a link from, say, a post in Blogger Help Forum: Something Is Broken, to this article, your computer sends a single message to the Blogger server, containing three essential details.
- The IP address of your computer.
- The URL of a forum discussion, which contains a link to this article.
- The URL of this article.
From the message, the Blogger server creates a server activity record.
- An IP address.
- The URL of the page containing a link to the webpage requested.
- The URL of the webpage requested.
That server activity record, in the Stats display for the blog, is known as a "pageview".
Finally, the Blogger server starts sending web page content back to your computer, so your computer can display this article to you. As the webpage content is sent back to your computer, your computer receives, and displays, the received content - and asks for more content.
What does a referer spam "pageview request" look like?
Simple enough? So, what is referer spam? Simply, a single message from a spammer computer, to the Blogger computer, containing three essential details.
- An IP address - possibly, but not predictably, of their computer.
- The URL of the website being pimped (the spammed website).
- The URL of the blog being spammed (your blog).
From the message, the Blogger server creates a server activity record.
- An IP address.
- The URL of the page (supposedly) containing a link to the webpage requested.
- The URL of the webpage (supposedly) requested.
That server activity record, in the Stats display for the blog, is also known as a "pageview".
Finally, the Blogger server starts sending web page content back to the IP address provided. If the IP address does refer to the spammers computer, what is received is simply ignored. The spammer computer moves on, and sends another fake pageview message to another server - maybe referencing this blog.
The "pageview request" is generated, before referer spam discontinues.
The problem is simply that no web server can detect a message from a client computer, that results in a response that is just ignored. Web traffic is lossy, and clients drop offline constantly. Even if the response could be detected as ignored, the ignored request might still reflect legitimate activity, initiated by a client that immediately went offline.
There is simply no way for Google to block the spam - because the spam is simply one message that results in a response, by the Blogger server, that is subsequently ignored by the client computer.
That's it.
So why can't Google block the numbers generated by referer spam, as the referer spam hits the servers? Simply because the numbers may not really represent actual spam. They can, just as easily, reflect intense, legitimate activity - or possibly a devious attack against a legitimate website.
Google can only detect referer spam in context, against multiple blogs.
Specific pageview counts and details are observed in context - are blocked only after the same activity is observed against multiple blogs, over long periods of time (similar, in concept, to stateful network traffic analysis) - and the numbers are removed, retroactively.
All of this is a simple unavoidable side effect, of blog owners needing site activity figures that are not affected by script filtering by the blog readers, complicated by fraudulent activity by hackers and spammers.
Referer spam is not unique to Blogger - it is simply tuned to abuse Stats logs.
Please note that referer spam did not start with Blogger - it's an Internet wide problem. Even though it appears mindless and random, some of it is craftily designed and executed.
For a comprehensive look at how referer spam works, outside Blogger, see Wikipedia: Referer spam.
The problem here is threefold.
- Too many blog owners obsess over raw pageview counts.
- Too many blog owners do not understand the origins of referer spam.
- Too many blog owners are not interested in understanding the real problem.
That's it!
Not all blog owners realise how unique Blogger Stats is, in its design.
Some owners may idly suggest that Stats can easily be replaced by any third party visitor activity log / meter.Why bother to use Stats? Since Blogger Stats shows referrer spam, it's pretty useless - just use SiteMeter, StatCounter - or Google Analytics.
They have no idea why Stats was designed as it is, nor what information Stats provides, that no competing product can possibly provide.
Every add-on accessory, such as any visitor activity counter / log / meter, has to be manually installed in your blog.
Most visitor activity counter / log / meter products require installation.
Simple accessories, which depend upon your visitor clicking on a link, can be installed easily - just add a clickable link on your blog - either in a post, or anywhere in the body of the blog. Visitor logs or meters such as SiteMeter or StatCounter, in order to produce usable statistics, can't depend upon the blog readers clicking on a link.
Most such accessories use add-on JavaScript code, installed in the body of the blog (as an accessory gadget), or in the blog template (installed using "Edit HTML"). With add-on code, that references any external server, the installed location on the blog page, of the accessory code, is crucial.
- Install the add-on at the top of the web page (or in the template code), and as your reader loads each page, he / she gets to watch the page load pause, as it waits for a distant accessory server to respond (while recording the visit).
- Install the add-on at the bottom of the page, and any reader, who closes the display before the page finishes loading, will not be counted by Analytics / SiteMeter / StatCounter.
In either case, the page takes longer to load. This causes reader impatience, and motivates the reader to close the display before the page finishes loading. The result - your blog gets one less new reader.
Most visitor activity counter / log / meter products are affected by filters.
Besides reader impatience causing statistical inaccuracy, all third party accessories that use JavaScript have a second problem - script filtering by our readers.
Analytics, SiteMeter, and StatCounter are known to be explicitly blocked, by some browser setting or third party application that may run on any given client computer. Some security products specifically list "sitemeter.com" or "statcounter.com" in their Block Lists.
A lot of malicious activity, encountered when surfing the web, can easily be blocked by proactive script filtering - just permit scripts, from any given domain, only when explicitly told to do so.
Every reader uses security accessories in the browser and on the computer - and many security accessories and settings provide script filtering. Most security is now provided as "deny by default, permit only on demand".
Most security products update automatically, as updates are produced - and this leads to other problems. Many of our readers, who are concerned with security, or who use computers that are well protected, may not be counted, consistently, by Analytics, SiteMeter, or StatCounter.
Stats does not require installation, and is not affected by filters.
Blogger Stats avoids the issues of page load delay induced impatience, and client filtering, by not using JavaScript add-on code. Since Stats is a Blogger accessory, it can retrieve data directly from the access logs produced by the Blogger servers. Access to server access logs:
- Does not cause page load delays.
- Is not subject to page load delay impatience.
- Is not subject to reader security settings.
- Does not require installation of any accessory, on individual blogs.
Besides the problems of our readers visits not being counted, we have the issue of what information is provided, and for what period of time. If you have installed SiteMeter or StatCounter on your blog, look at the displays provided. Each product will mention a limit of either 100 or 500 log entries - and will display details or statistics based upon the log used.
Many Blogger blogs get more than 500 visits in a single day - requiring blog owner visits to the log website at least daily, or to pay for extra service, to provide any benefit. And of course, that log was started only after the product was installed.
Blogger Stats, on the other hand, is able to provide statistics for the current day, week, month, and for "all time" (starting in May 2009, for all blogs in existence at that time).
Stats does not discard old statistics, they just extract from the server access logs, for any time range provided.
- All time.
- Last 30 days ("Month").
- Last 7 days ("Week").
- Last 24 hours ("Day").
- Last 2 hours ("Now").
Any blog owner can see any available statistics, at any time. Stats never has to be installed, by any blog owners - it is already there.
Stats is vulnerable to bogus activity records, created by spammers.
Unfortunately, the biggest strength of Stats - use of the server access logs to gather the visitor activity data - leads to its best known weakness - abuse by referer spammers, which leads to inflated blog read counts.
It may help to understand that referer spam did not start with Stats - nor is referer spam unique to Stats. Referer spam has been around ever since people published websites, and displayed a visitor log extract ("My Recent Visitors") to make their website more interesting to new readers (The suggestion that "This website should be more interesting to YOU, because it has readers from all over the world!").
Stats, like every visitor activity counter / log / meter product, resets totals.
Besides the concern of referer spam, we see various evidences of confusion about Stats displays.
All of these limitations are simply the direct result of how Stats is designed.
Every visitor activity counter / log / meter product differs, from every other product.
In reality, both Blogger Stats and third party visitor activity logs or meters have their respective advantages - and neither choice can ever replace another.
- If you want to see demographic details about some readers of your blog, you can use Analytics, SiteMeter, StatCounter - or any number of third party visitor activity logs and meters - after the chosen accessory has been installed.
- If you want to see comprehensive statistics about all readers of your blog - without being limited by install time or reader security policy - only Blogger Stats will help you.
Fortunately, all products are free - and Stats requires no installation. Your Stats data is there, waiting for you - on the Blogger dashboard menu.
We've been discussing referer spam for many years.
Every week, besides many people who wonderWhy doesn't Google put an end to this, for good?
there are occasionally some folks who wonderBut is this all bad? Doesn't this help us in our overall Blogger statistics as far as traffic counts go? If so, it's not all bad - for those of us with less than ginormous followings.
And both attitudes reflect people who just don't understand what it is.
We've already discussed, repeatedly, why Google can't just put a end to it, unilaterally.
The latter musing, considering that it may possibly be beneficial to any Blogger blogs, is no more valid than the former. Referer spam simply cannot be blocked, because it is not different from legitimate traffic.
Stats logs entries, from referer spam, do not reflect people viewing the blog.
The numbers reflected in our Stats logs do not represent any actual traffic against our blogs, or any websites with links to our blogs. The only person who benefits from referer spam is the owner of the advertised (fake referer URL) sites - and that happens only when we click on the links in the Stats display.
Only Stats, which builds its pageview counts and graphs using the Blogger server activity logs, is subject to referer spam. All third party visitor logs and meters, which depend upon snippets or widgets added to the blog template, are not subject to this problem.
Don't click on the links - and use a third party visitor log for verification.
So, besides my facetious advice that you simplyDon't click on the links.
comes equally facetious advice that youGet a third party visitor log / meter, for accurate and comprehensive pageview counts.
For several days now, we've seen various reports from anxious Blogger blog owners, in Blogger Help Forum: Something Is Broken, lamenting the lack of detail information in their Stats logs.My Stats logs does not show the page views for particular posts.
This is just one example, of the many ways that blog owners perceive, and report, the ongoing problem.
If you look at the Stats Overview or Posts display, for any blog with any popularity, in the "Now" or "Day" time range, you'll likely see the well known adviceNo stats yet, check back later.
When initially reported, this was only visible in the "Now" time range. Now, the effect is seen in "Day", and for some, less consistently trafficked blogs, in "Week" also. The effect is visible, consistently, in both the Classic GUI (brown / dark blue display), and the New GUI (2011) (brown / orange on white display).
This problem is apparent, for most blogs, in the "Posts" tab.
Most blogs with consistent traffic will show this affecting only the detail Posts lists. The "Now" and "Day" Posts lists, as noted, show "No stats yet, check back later.". The "Week", Month", and "All time" Posts displays show the Posts lists from before the problem first occurred. Both the Posts names in the lists, and the pageview counts for each post listed, are frozen.
This blog shows no effect in pageview counts, outside the Posts detail lists. I've carefully examined the graphs for "Now", "Day", "Week", "Month", and "All time", in both the Classic and New GUI, for this blog - and I don't see any count discrepancies. Your observations may agree or disagree - and I await your comments.
We have a Rollup discussion, in Blogger Help Forum.
We have an ongoing rollup discussion, in Blogger Help, where the scope of the problem is being discussed. My original perception of the problem began as we explored the latest wave of referer spam; I am today trying to determine if the two events are related.
An earlier outage may be related to this problem.
We also had an interesting 2 hour outage, a week ago, where no Stats data (counts, or details) was available for the previous week - and some blog owners are wondering if that issue is related. We have no references for the latter episode, as we escalated that problem in real time, and Blogger Support was able to fix it, immediately.
That problem was complete (as noted, both counts and details were available), it was immediate (it showed on all Stats displays for the previous week, retroactively), and it was total (the immediate nature and effect on the forums made it an obvious BloggerFire). This problem, according to my observations, is neither.
(Update 18:00): Blogger Support has fixed this problem.
This morning, we're seeing a number of questions in Blogger Help Forum: Something Is Broken.Why are my Stats displays showingNo stats yet, check back later.
yet again?
and more interestingly"blogging.nitecruzr.net" is showing multiple hits in my Stats log!
And none of this is really surprising, personally.
The bad guys, who have been serving various waves of referer spam against the many Blogger blogs, have recently included "nitecruzr.net" in their lists of websites, falsely advertised by their attacks.
I've been using this blog to talk about spam, in general - and about referer spam, in particular - for years. This week, the people who operate the referer spam networks are targeting me, in a Joe Job attack, using Stats logs to confuse Blogger blog owners.
In crime novels, it's called a "frame up". In networking, what they are doing is vaguely similar to a "smurf attack", using the various Blogger blog owners, in an attempt to overwhelm the forums.
I guess I should be flattered. Let's try to keep it in perspective, though - and keep working on your blog.
Blogger blog owners have been suffering from the onslaught of referer spam, in their Stats logs, for over 6 months now. Some blog owners, who use third party visitor logs / meters like FlagCounter, Sitemeter, and StatCounter, have started to wonder.How do SiteMeter and StatCounter manage to filter out the referer spam, yet Google cannot do anything about it?
This would be quite a magic trick indeed - if this was happening.
When you setup a third party visitor information product, like FlagCounter, Sitemeter, and StatCounter, the installation process involves addition of a JavaScript code snippet, as part of the blog template. Any time a page in the blog is loaded, by a blog visitor, the JavaScript code references the FlagCounter, Sitemeter, or StatCounter server, and adds a visitor record describing the pageview.
That's such a simple way to gather information about your visitors - and for its simplicity, it is unreliable, in various ways.- Your visitors who use computers that filter content from domains - like BlogSpot, FlagCounter, Sitemeter, StatCounter, or possibly your non BlogSpot custom domain - won't be counted, reliably.
- Your visitors who share a caching proxy server won't be counted.
- Depending upon where in the page, the JavaScript code is added, your visitors may or may not be counted.
When Blogger / Google designed the Stats visitor information accessory, they used another technique for counting visitor activity. Blogger, unlike FlagCounter, Sitemeter, and StatCounter, has access to the server activity logs. When Stats was activated, for the Blogger blogosphere, they activated it on a blog by blog basis - and without requiring any updates to the blog code.
Every Blogger blog was given Stats access, and the statistics initially provided preceded actual Stats availability, because nobody had to install any accessory code to the blogs. That is because Blogger uses the actual server activity logs - and does not require add-on accessory code.
Unfortunately, server activity logs only record server activity - and this is how referer spam works - and why it is so hard for Google to block it. Referer spam is simply a normal access request from the spammer, which generates a server access record, and a Stats pageview. The spammer simply goes away after generating the initial access request. The Blogger server, as with any non Blogger server, has no way to detect when the spammer drops the connection and goes away.
Since third party products like FlagCounter, Sitemeter, and StatCounter use add-on code, they are not susceptible to referer spam techniques. Only products which work from the server activity logs are susceptible. FlagCounter, Sitemeter, and StatCounter, and other third party visitor information accessories, have no need to filter bogus activity.
And this is why I have stated that the only way that referer spam will ever go away is for us to make it unprofitable for the spammers, by not clicking on the links in the Stats logs.
>> Top
We've been observing the ongoing problem with referer spam since the beginning of the year - yet we see that some blog owners still haven't heard of the problem. Some folks who are advised later, to be more cautious
Don't click on the links!
come back with anxious queries
I already clicked on the link! What will happen now? Have I been hacked?
Here, there's no authoritative answer.
When you click on a Stats traffic source link, and get an eyeful of porn or spam, you're not going to be happy - but hopefully, if your computer is well protected, chances are probably good that you're OK. On the other hand, if this makes you anxious, it would not be a bad idea to get the computer checked out.
In the future, if you must check out your Stats traffic sources, use a proxy server. Again, don't click on the links - copy the URLs, and paste them into the proxy server address window.
If you surf to a referring website and like what you see, then go back, click on the link, and bookmark the URL. Just do this after you screen the traffic source.
If, one day, Blogger uses SafeSearch to screen the URLs, we'll be safe again. And shortly afterwards, referer spam will dry up as the people who pay for spam based advertising stop paying.
>> Top
