Are We There Yet?

I should feel honored. I've read all over the place about how "referral spam" is rapidly making its way through the Internet. What's referral spam, you ask? Well, most web servers keep logs by default. One of the more interesting things that web servers log is the referring link that led a visitor to your web site (this is a part of the HTTP protocol - that's how the web server gets the information). Geeks and other sys admins are notoriously interested in this data because being linked to from another web page is supposed to make up for our lack of popularity in high school. Or something.

Anyway, spammers and advertisers never miss an opportunity to sully every crevice of the Internet that they can seep in to and now Web referrals are no different. As I browsed through my web server log not long ago, I came upon this nugget:

2004-01-05 20:08:40 206.129.0.135 - HEAD /index.htm 200 288 HTTP/1.0 StarProse+Referrer+Advertising+System+2004 http://blog.johnkerry.com

Now, the orignal intent of referrals (as I said before) is that it's supposed to indicate that the referring page linked to yours and that's how the traffic was directed to your site. Do you honestly believe John Kerry's blog linked to mine? That would be especially funny since 1) I'm not a Democrat, and 2) I've never talked politics on this site.

Of course, it's trivial to forge the HTTP header information that does this. I could write such a utility in a few hours. A quick search of Google for "StarProse+Referrer+Advertising+System+2004" links this whole phenomenom to a company - StarProse.com. Imagine a script that will crawl through pages it finds on the Web pretending to visit the site using a URL of your choice as the referring link. Ta-da. You've just stamped your URL in to the Web server logs of thousands of servers out there. Better yet, for the thousands of these sites that publish their log statistics on their sites (many of which interpret URLs and change them in to hyperlinks), you've just added a link from their page to yours which does wonders for your Google page rank consequently increasing your position in the search engine.

The funny part is that a stroll through those Google results shows you that two people are doing this the most: porn sites and Democratic Presidential hopefuls from John Kerry and Joe Lieberman to the supposedly Geek Friendly™ Howard Dean. I'll leave you to draw your own conclusions.