FEED Validator

for Atom and RSS and KML

Congratulations!

This is a valid Atom 1.0 feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

line 1, column 0: Use of unknown namespace: http://schemas.google.com/blogger/2008 [help]
```
<?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href="http://www.blog ...
```
line 1, column 0: Unable to validate namespace: http://schemas.google.com/g/2005. See the Google Data specification at https://developers.google.com/gdata/docs/1.0/elements [help]
```
<?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href="http://www.blog ...
```

line 1, column 2562: Self reference doesn't match document location [help]

... s/default?alt=atom&amp;redirect=false'/><link rel='alternate' type='text ...
                                             ^

line 1, column 9893: style attribute contains potentially dangerous content: thin (2 occurrences) [help]

... ctive with regard to their real mission.</content><link rel='replies' ty ...
                                             ^

line 1, column 20046: content should not contain script tag [help]

... 0&amp;width=450&quot;&gt;&lt;/script&gt;</content><link rel='replies' ty ...
                                             ^

line 1, column 110047: title should not contain HTML unless declared in the type attribute: ™ [help]

... itle type='text'>LZZR Linking&amp;trade;</title><content type='html'>&lt ...
                                             ^

line 1, column 112183: link should not contain HTML: ™ [help]

... .html' title='LZZR Linking&amp;trade;'/><author><name>Sebastian</name><u ...
                                             ^

Source: http://sebastianx.blogspot.com/atom.xml

<?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href="http://www.blogger.com/styles/atom.css" type="text/css"?><feed xmlns='http://www.w3.org/2005/Atom' xmlns:openSearch='http://a9.com/-/spec/opensearchrss/1.0/' xmlns:blogger='http://schemas.google.com/blogger/2008' xmlns:georss='http://www.georss.org/georss' xmlns:gd="http://schemas.google.com/g/2005" xmlns:thr='http://purl.org/syndication/thread/1.0'><id>tag:blogger.com,1999:blog-14517520</id><updated>2024-03-23T14:08:33.907-04:00</updated><category term="Google"/><category term="SEO"/><category term="nofollow"/><category term="Webmaster Central"/><category term="crap"/><category term="fun"/><category term="microformats"/><category term="search quality"/><category term="Erol"/><category term="JavaScript redirects"/><category term="cloaking"/><category term="paid links"/><category term="robots.txt"/><category term="Blogger"/><category term="crawler directives"/><category term="e-commerce"/><category term="risky linkage"/><category term=".htaccess"/><category term="Yahoo"/><category term="robots meta tags"/><category term="spam"/><category term="XML-Sitemaps"/><category term="reciprocal links"/><category term="recommendations"/><category term="tools"/><category term="Anchor Text"/><category term="link building"/><category term="social Web"/><category term="testing"/><category term="twitter"/><category term="AJAX"/><category term="AdSense"/><category term="CTR"/><category term="Copy+paste-penalties"/><category term="Web development"/><category term="Webspam"/><category term="X-Robots-Tag"/><category term="hotlinking"/><category term="spam report"/><category term="404grabber"/><category term="AdWords"/><category term="IIS"/><category term="Netscape"/><category term="Progress EasyAsk"/><category term="URL removal"/><category term="analytics"/><category term="copyrights"/><category term="duplicate content"/><category term="ego food"/><category term="languages"/><category term="plagiarism"/><category term="pownce"/><category term="site-search"/><category term="spoofing"/><category term="translations"/><category term="trolling"/><title type='text'>Sebastian's Pamphlets</title><subtitle type='html'>If you've read my articles somewhere on the Internet, expect something different here.</subtitle><link rel='http://schemas.google.com/g/2005#feed' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/posts/default'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default?alt=atom&redirect=false'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/'/><link rel='hub' href='http://pubsubhubbub.appspot.com/'/><link rel='next' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default?alt=atom&start-index=26&max-results=25&redirect=false'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><generator version='7.00' uri='http://www.blogger.com'>Blogger</generator><openSearch:totalResults>170</openSearch:totalResults><openSearch:startIndex>1</openSearch:startIndex><openSearch:itemsPerPage>25</openSearch:itemsPerPage><entry><id>tag:blogger.com,1999:blog-14517520.post-1816050928354584157</id><published>2007-08-20T10:25:00.000-04:00</published><updated>2007-08-25T05:09:34.416-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="Blogger"/><category scheme="http://www.blogger.com/atom/ns#" term="recommendations"/><title type='text'>This is my last blog post ...</title><content type='html'>... on blogspot. Farewell Blogger. Sebastian's Pamphlets moved to <a href="http://sebastians-pamphlets.com/">sebastians-pamphlets.com</a>. Please visit me over there, change the feed URL in your reader to <a href="http://feeds.sebastians-pamphlets.com/SebastiansPamphlets">http://feeds.sebastians-pamphlets.com/SebastiansPamphlets</a> and update your blogrolls. Thank you! This site will be kept as an archive for a while, but I'll soon make it uncrawlable and even uglier with blinking links pointing to the new location. You dear readers don't deserve that, so please move with me to <a href="http://sebastians-pamphlets.com/">sebastians-pamphlets.com</a>! I'm looking forward to seeing you there :) Sebastian</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/1816050928354584157/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=1816050928354584157' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/1816050928354584157'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/1816050928354584157'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/08/this-is-my-last-blog-post.html' title='This is my last blog post ...'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-4587333288381211155</id><published>2007-08-15T11:07:00.000-04:00</published><updated>2007-08-15T12:27:08.454-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="fun"/><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="nofollow"/><title type='text'>Google's 5 sure-fire steps to safer indexing</title><content type='html'><a href="http://www.smart-it-consulting.com/img/misc/safer-indexing-nofollow-google.png"><img src="http://www.smart-it-consulting.com/img/misc/safer-indexing-nofollow-google-thumbnail.png" border="0" width="165" height="209" align="left" alt="Nofollow plague" title="Nofollow plague" style="margin: 3px; border: thin dashed firebrick;" /></a>Are you wondering why <a href="http://www.grayhatnews.com/">Gray Hat Search Engine News</a> (GHN) is so quiet recently? One reason may be that I've borrowed their Google savvy spy. I've sent him to Mountain View <a href="http://sebastianx.blogspot.com/2007/07/why-ebay-and-wikipedia-rule-googles.html">again</a> to learn more about Google's nofollow strategy. He returned with a copy of Google's recently revised <a href="http://www.google.com/corporate/">mission statement</a>, discovered in the wastebasket of a conference room near office 211 in building 43. Read the shocking and unbelievable head note printed in bold letters:Google's mission is to condomize the world's information and make it universally uncrawlable and useless. Read and reread it, then some weird facts begin to make sense. Now you'll understand why:<ol> <li>The rel-nofollow plague was designed to maximize collateral damage by devaluing all hyperlinked votes by honest users of nearly all platforms you're using everyday, for example <a href="http://sebastianx.blogspot.com/2007/08/just-another-victim-of-nofollow-plague.html">Twitter</a>, <a href="http://sebastianx.blogspot.com/2007/02/nofollow-universe-of-black-holes.html">Wikipedia</a>, <a href="http://sebastianx.blogspot.com/2007/06/google-nofollows-itself.html">corporate blogs</a>, <a href="http://www.jlh-design.com/2007/08/more-nofollow-bs/">GoogleGroups</a> ... ostensibly to nullify the efforts of a <a href="http://sebastianx.blogspot.com/2007/06/fraud-from-desk-of-sebastian-foss.html">few</a> <a href="http://sebastianx.blogspot.com/2007/06/which-sebastian-foss-is-spammer.html">spammers</a>.</li> <li>Nobody bothers to comment on your <a href="http://sebastianx.blogspot.com/2007/06/blogger-abuses-rel-nofollow-due-to.html">nofollow'ed blog</a>.</li> <li>Google invented the supplemental index (to store scraped resources suffering from too many condomized links) and why it grows faster than the main index.</li> <li>Google installed the Bigdaddy infrastructure (to prevent Ms. Googlebot from following nofollow'ed links).</li> <li>Google switched to BlitzCrawling (to list timely contents for a moment whilst fat resources from large archives get buried in the supplemental index). RIP deep crawler and freshbot.</li> </ol> Seriously, the deep crawler isn't defunct, it's called supplemental crawler nowadays, and the freshbot is still alive as Feedfetcher. Disclaimer: All these hard facts were gathered by torturing sources close to Google, robbery and other unfair methods. If anyone bothers to debunk all that as bad joke, one question still remains: Why does Google next to nothing to stop the nofollow plague? I mean, ongoing mass abuse of rel-nofollow is obviously counterproductive with regard to their real mission.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/4587333288381211155/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=4587333288381211155' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/4587333288381211155'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/4587333288381211155'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/08/googles-5-sure-fire-steps-to-safer.html' title='Google's 5 sure-fire steps to safer indexing'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-4909469076883882501</id><published>2007-08-13T09:22:00.000-04:00</published><updated>2007-08-13T09:33:43.693-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="ego food"/><title type='text'>Ego food from John's barbecue</title><content type='html'><a href="http://johnmu.com/">JohnMu</a> <a href="http://johnmu.com/interview-with-sebastian">grilled me</a> ;) Check out his <a href="http://johnmu.com/category/people/">folks bin</a> frequently for readable Webmaster interviews. Thanks John, it was fun :)</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/4909469076883882501/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=4909469076883882501' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/4909469076883882501'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/4909469076883882501'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/08/ego-food-from-johns-barbecue.html' title='Ego food from John's barbecue'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-4242940512833131326</id><published>2007-08-13T04:11:00.000-04:00</published><updated>2007-08-13T04:57:15.162-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="nofollow"/><category scheme="http://www.blogger.com/atom/ns#" term="twitter"/><title type='text'>Just another victim of the nofollow plague</title><content type='html'>It's <a href="http://twitter.com/SebastianX/statuses/202991852">evil</a>, it <a href="http://twitter.com/SebastianX/statuses/195714772">sucks</a> even more than the crappy tinyurl nonsense obfuscating link destinations, nobody outside some SEO cliques really <a href="http://twitter.com/johnweb/statuses/202985732">cares</a> <a href="http://twitter.com/johnweb/statuses/195582502">about</a> or <a href="http://twitter.com/graywolf/statuses/195173642">noticed</a> it, I'm not sure it's <a href="http://twitter.com/SebastianX/statuses/197844332">newsworthy</a> because it's perfectly in line with rel-nofollow semantics, but it annoys me and others so here is the news of late last week: Twitter drank the <a href="http://twitter.com/johnweb/statuses/196786372">nofollow kool-aid</a>.Folks, remove Twitter from your list of PageRank sources and drop links for fun and traffic only. I wonder whether particular people change their linking behavior on Twitter or not. I won't.<a href="http://sphinn.com/story/2728"><img src="http://www.smart-it-consulting.com/img/misc/twitter-nofollow-crap.png" border="0" width="366" height="34" align="left" style="margin-right:5px;" alt="Nofollow crap on Twitter" title="Nofollow crap on Twitter" /></a>Following Nofollow's questionably tradition of maximizing collateral damage Twitter nofollows even links <a href="http://blessinghands.blogspot.com/">leading</a> to Matt's mom's <a href="http://blessing-hands.org/">charity site</a>. More <strike>PageRank</strike> power to you, Betty Cutts! Your son deserves a bold <a href="http://www.mattcutts.com/blog/" rel="nofollow">nofollow</a> for inventing the beast ;)</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/4242940512833131326/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=4242940512833131326' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/4242940512833131326'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/4242940512833131326'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/08/just-another-victim-of-nofollow-plague.html' title='Just another victim of the nofollow plague'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-1939547931367168306</id><published>2007-08-11T10:17:00.000-04:00</published><updated>2007-08-14T03:21:29.925-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="fun"/><title type='text'>ɹǝɟɟıp oʇ bǝq ı</title><content type='html'>:sdıʇ ɹǝpısuı <a href="http://fantomaster.com/fantomNews/archives/2007/08/10/%c9%a5%c9%94%ca%87%c9%90%c9%94-o%ca%87-%ca%871n%c9%94i%c9%9f%c9%9fip-%ca%87%c9%90%c9%a5%ca%87-%ca%87usi-%ca%8e11%c9%90%c7%9d%c9%b9-%ca%87i-%c9%94i%c9%9f%c9%9f%c9%90%c9%b9%ca%87-1i%c9%90%ca%87buo1/" style="text-decoration:none;">sʞ1oɟ</a> <a href="http://www.cornwallseo.com/search/index.php/2007/07/24/%ca%8e11is-%ca%87sn%d8%b8-si-u%ca%8dop-%c7%9dpisdn-bui%ca%87i%c9%b9%ca%8d/" style="text-decoration:none;">ɹǝɥʇo</a> ʇdʎɹɔuǝ oʇ unɟ s,ʇı ʇnq .sdɐɥɹǝd ¿uoɹoɯʎxo uɐ ʇɐɥʇ sı .ʎ1ɟʎɐp ɐ ǝʞı1 ʇsnظ 'ǝɹnʇnɟ ɐ sɐɥ ɔıɟɟɐɹʇ 1ıɐʇ buo1 ǝsɹǝʌǝɹ buı11nd oǝs uʍopǝpısdn <a href="http://www.smart-it-consulting.com/img/misc/reverse-cornwall-seo.png"><img src="http://www.smart-it-consulting.com/img/misc/reverse-cornwall-seo.png" border="0" width="99%" alt="Lyndon's insider tip" title="Lyndon's insider tip" /></a> <a href="http://www.smart-it-consulting.com/img/misc/reverse-fantomaster.png"><img src="http://www.smart-it-consulting.com/img/misc/reverse-fantomaster.png" border="0" width="99%" alt="Ralph's insider tip" title="Ralph's insider tip" /></a> <a href="http://www.i-am-bored.com/bored_link.cfm?link_id=24226">If you're bored, give it a try</a>. Mark <a href="http://www.alistseo.com/25/%ca%87i%c9%90q-%ca%9eui1-%ca%8e11is-%ca%87sn%d8%b8-si-u%ca%8dop-%c7%9dpisdn-bui%ca%87i%c9%b9%ca%8d/">did it</a>.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/1939547931367168306/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=1939547931367168306' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/1939547931367168306'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/1939547931367168306'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/08/p-o-bq.html' title='ɹǝɟɟıp oʇ bǝq ı'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-5198171575635111518</id><published>2007-08-10T08:43:00.000-04:00</published><updated>2007-08-14T05:52:18.156-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="testing"/><title type='text'>Lijit SERP</title><content type='html'>I'm testing the <a href="http://www.lijit.com/">lijit</a> search results page. In theory, when you <a href="#lijit-search-box">submit a lijit search</a> you should land here to view the results. Unfortunately it doesn't work as expected when you're surfing with HTTP_REFERER turned off. Sigh. <script type="text/javascript" src="http://www.lijit.com/pvs_on_site/SebastianX?height=1000&width=450"></script></content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/5198171575635111518/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=5198171575635111518' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/5198171575635111518'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/5198171575635111518'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/08/lijit-serp.html' title='Lijit SERP'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-2952458973087960680</id><published>2007-08-09T04:14:00.000-04:00</published><updated>2007-08-09T04:19:29.738-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="link building"/><category scheme="http://www.blogger.com/atom/ns#" term="recommendations"/><category scheme="http://www.blogger.com/atom/ns#" term="social Web"/><title type='text'>How to bait link baiters and attention whores properly</title><content type='html'>What a <a href="http://www.johnon.com/381/sphinn-this-page.html" style="text-decoration:none; font-weight:normal;">brilliant marketing stunt</a>. Click here! Err... click: <a href="http://sphinn.com/story/2432" title="SPHINN THIS">Brilliant</a>. <a href="http://www.johnon.com/" title="John Andrews">Marketing</a>. <a href="http://sphinn.com/greatesthits/" title="Sphinn's hall of fame">Stunt</a>. Best of luck John :)</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/2952458973087960680/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=2952458973087960680' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/2952458973087960680'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/2952458973087960680'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/08/how-to-bait-link-baiters-and-attention.html' title='How to bait link baiters and attention whores properly'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-7818283285901115030</id><published>2007-08-08T10:45:00.000-04:00</published><updated>2007-08-13T16:18:44.616-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="link building"/><category scheme="http://www.blogger.com/atom/ns#" term="reciprocal links"/><category scheme="http://www.blogger.com/atom/ns#" term="risky linkage"/><category scheme="http://www.blogger.com/atom/ns#" term="SEO"/><title type='text'>Google manifested the axe on reciprocal link exchanges</title><content type='html'>Yesterday <a href="http://fantomaster.com/fantomNews/archives/2007/08/07/more-disgusting-fud-tactics-google-bans-reciprocal-linking/">Fantomaster</a> via <a href="http://www.threadwatcher.com/2007/08/07/66695-the-reciprocal-link-ban/">Threadwatcher</a> pointed me to this <a href="http://www.google.com/support/webmasters/bin/answer.py?answer=66356&query=link+exchange&topic=&type">page</a> of Google's Webmaster help system. The cache was a few days old and didn't show a difference, I don't archive each and every change of the guidelines, so I asked and a friendly and helpful Googler told me that this item was around for a while now. Today this page made it on <a href="http://sphinn.com/story/2363">Sphinn</a> and probably a few other Webmaster hangouts too. So what the heck is the scandal all about? When you ask Google for help on "<a href="http://www.google.com/support/webmasters/bin/answer.py?answer=66356&query=link+exchange&topic=&type">link exchange</a>", the help machine rattles for a second, sighs, coughs, clears its throat and then yells out the answer in bold letters: "Link schemes", bah! Ok, we already <a href="http://www.google.com/support/webmasters/bin/answer.py?answer=35769">knew</a> what Google thinks about <a href="http://www.smart-it-consulting.com/article.htm?node=155&page=89#define-artificial-link">artificial linkage</a>: "Don't participate in link schemes designed to increase your site's ranking or PageRank". Honestly, what is the intent when I suggest that you link to me and concurrently I link to you? Yup, it means I boost your PageRank and you boost mine, also we chose some nice anchor text and that makes the link deal perfect. In the eyes of Google even such a tiny deal is a link scheme, because both links weren't put up for users but for search engines. Pre-Google this kind of link deal was business as usual and considered natural, but frankly back then the links were exchanged for traffic and not for search engine love. We can rant and argue as much as we want, that will not revert the changed character of link swaps nor Google's take on manipulative links. Consequently Google has devalued artificial reciprocal links for ages. Pretty much simplified these links nullify each other in Google's search index. That goes for tiny sins. Folks raising the concept onto larger link networks got caught too but penalized or even banned for link farming. Obviously all kinds of link swaps are easy to detect algorithmically, even triangular link deals, three way link exchanges and whatnot. I called that plain vanilla link 'swindles', but only just recently Google has caught up with a scalable solution and seems to detect and penalize most if not all variants covering the whole search index, thanks to the search quality folks in Dublin and Zurich even overseas in whatever languages. The knowledge that the days of free link trading are numbered was out for years before the exodus. Artificial reciprocal links as well as other linkage considered link spam by Google was and is a pet peeve of Matt's team. Google sent lots of warnings, and many sane SEOs and Webmasters heard their traffic master's voice and acted accordingly. Successful link trading just went underground leaving the great unwashed alone with their obsession about exchanging reciprocal links in the public. Also old news is, that Google does not penalize reciprocal links in general. Google almost never penalizes a pattern or a technique. Instead they try to figure out the Webmaster's intent and judge case by case based on their findings. And yes, that's doable with algos, perhaps sometimes with a little help from humans to compile the seed, but we don't know how perfect the algo is when it comes to evaluations of intent. <a href="http://www.smart-it-consulting.com/article.htm?node=155&page=89#define-natural-link">Natural reciprocal links</a> are perfectly fine with Google. That applies to well maintained blogrolls too, despite the often reciprocal character of these links. Reading the <a href="http://www.google.com/support/webmasters/bin/answer.py?answer=66356">link schemes page</a> completely should make that clear. <a href="http://www.google.com/support/webmasters/bin/answer.py?answer=66356">Google defines link scheme</a> as "[...] Link exchange and reciprocal links schemes ('Link to me and I'll link to you.') [...]". The "I link to you and vice versa" part literally addresses link trading of any kind, not a situation where I link to your compelling contents because I like a particular page, and you return the favour later on because you find my stuff somewhat useful. As <a href="http://www.threadwatcher.com/2007/08/07/66695-the-reciprocal-link-ban/#comment-141">Perkiset</a> puts it "linking is now supposed to be like that well known sex act, '68? - or, you do me and I'll owe you one'" and there is truth in this analogy. Sometimes a favor will not be returned. That's the way the cookie crumbles when you're keen on Google traffic. The fact that Google openly said that link exchange schemes designed "exclusively for the sake of cross-linking" of any kind violate their guidelines indicates that first they were sure to have invented the catchall algo, and second that they felt safe to launch it without too much collateral damage. Not everybody agrees, I quote <a href="http://sphinn.com/story/2363#c3358">Fantomaster's critique</a> not only because I like his inimitably parlance: <blockquote id="flak">This is essentially a theological debate: Attempting to determine any given action's (and by inference: actor's) "intention" (as in "sinning") is always bound to open a can of worms or two. It will always have to work by conjecture, however plausible, which makes it a fundamentally tacky, unreliable and arbitrary process. The delusion that such a task, error prone as it is even when you set the most intelligent and well informed human experts to it (vide e.g. criminal law where "intention" can make all the difference between an indictment for second or first degree murder...) can be handled definitively by mechanistic computer algorithms is arguably the most scary aspect of this inane orgy of technological hubris and naivety the likes of Google are pressing onto us.</blockquote> I've seen some collateral damage already, but pragmatic Webmasters will find --respectively have found long ago-- their way to build inbound links under Google's regime. And here is the context of Google's definition <a href="http://www.google.com/support/webmasters/bin/answer.py?answer=66356">link exchanges = link schemes</a> which makes clear that not each and every reciprocal link is evil: <blockquote>[…] However, some webmasters engage in link exchange schemes and build partner pages exclusively for the sake of cross-linking, disregarding the quality of the links, the sources, and the long-term impact it will have on their sites. This is in violation of Google’s webmaster guidelines and can negatively impact your site’s ranking in search results. Examples of link schemes can include: &bull; Links intended to manipulate PageRank &bull; Links to web spammers or bad neighborhoods on the web &bull; Link exchange and reciprocal links schemes ('Link to me and I'll link to you.') &bull; Buying or selling links [...]</blockquote>Again, please read the whole page. Bear in mind that all this is Internet history, it just boiled up yesterday as the help page <a href="http://www.seoish.com/what-is-a-link-scheme/">was</a> <a href="http://groups.google.com/group/Google_Webmaster_Help-Requests/browse_thread/thread/7d8dcd4df2be7f70/34d758e9bb37dcb9">discovered</a>. Related article: <a href="http://searchengineland.com/070813-071742.php">Eric Ward on reciprocal links</a>, why they do good, and where they do bad.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/7818283285901115030/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=7818283285901115030' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/7818283285901115030'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/7818283285901115030'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/08/google-manifested-axe-on-reciprocal.html' title='Google manifested the axe on reciprocal link exchanges'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-5277547113690850141</id><published>2007-08-07T09:11:00.000-04:00</published><updated>2007-08-07T14:31:10.470-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="crawler directives"/><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="robots meta tags"/><category scheme="http://www.blogger.com/atom/ns#" term="search quality"/><category scheme="http://www.blogger.com/atom/ns#" term="X-Robots-Tag"/><title type='text'>NOPREVIEW - The missing X-Robots-Tag</title><content type='html'>Google provides <a href="http://www.google.com/help/features.html#pdf">previews of non-HTML resources</a> listed on their SERPs:<a href="http://www.smart-it-consulting.com/img/misc/view-pdf-file-as-html-on-google-serp.png"><img src="http://www.smart-it-consulting.com/img/misc/view-pdf-file-as-html-on-google-serp.png" border="0" width="99%" alt="View PDF as HTML document" title="View PDF as HTML document" /></a> These "view as text" and "view as HTML" links are pretty useful when you for example want to scan a PDF document before you clutter your machine's RAM with 30 megs of useless digital rights management (aka Adobe Reader). You can view contents even when the corresponding application is not installed, Google's transformed previews should not stuff your maiden box with unwanted malware, etcetera. However, under some circumstances it would make sound sense to have a NOPREVIEW <a href="http://sebastianx.blogspot.com/2007/07/handling-googles-neat-x-robots-tag.html">X-Robots-Tag</a>, but unfortunately <a href="http://googleblog.blogspot.com/2007/07/robots-exclusion-protocol-now-with-even.html">Google forgot to introduce it</a> yet. Google is rightfully proud of their capability to transform various file formats to readable HTML or plain text: Adobe Portable Document Format (pdf), Adobe PostScript (ps), Lotus 1-2-3 (wk1, wk2, wk3, wk4, wk5, wki, wks, wku), Lotus WordPro (lwp), MacWrite (mw), Microsoft Excel (xls), Microsoft PowerPoint (ppt), Microsoft Word (doc), Microsoft Works (wks, wps, wdb), Microsoft Write (wri), Rich Text Format (rtf), Shockwave Flash (swf), of course Text (ans, txt) plus a couple of "<a href="http://www.google.com/help/faq_filetypes.html#results">unrecognized</a>" file types like <a href="http://sphinn.com/story/2123#c2968">XML</a>. New formats are added from time to time. According to <a href="http://groups.google.com/group/Google_Webmaster_Help-chit-chat/msg/6cd6133565d62098">Adam Lasnik</a> currently there is no way for Webmasters to tell Google not to include the "View as HTML" option. You can <a href="http://www.seroundtable.com/archives/014264.html">try to fool Google's converters</a> by messing up the non-HTML resource in a way that a sane parser can't interpret it. Actually, when you search a few minutes you'll find e.g. PDF files without the preview links on Google's SERPs. I wouldn't consider this attempt a bullet-proof nor future-proof tactic though, because Google is pretty intent on improving their conversion/interpretation process. I like the previews not only because sometimes they allow me to read documents behind a login screen. That's a loophole Google should close as soon as possible. When for example PDF documents or Excel sheets are crawlable but not viewable for searchers (at least not with the second click) that's plain annoying both for the site as well as for the search engine user. With HTML documents the Webmaster can apply a NOARCHIVE crawler directive to prevent non paying visitors from lurking via Google's cached page copies. Thanks to the <a href="http://sebastianx.blogspot.com/2007/07/handling-googles-neat-x-robots-tag.html">newish REP header tags</a> one can do that with non-HTML resources too, but neither NOARCHIVE nor NOSNIPPET etch away the "view-as HTML" link. &lt;speculation&gt;Is the lack of a NOPREVIEW crawler directive just an oversight, or is it stuck in the pipeline because Google is working on supplemental components and concepts? Google's <a href="http://groups.google.com/group/Google_Webmaster_Help-chit-chat/browse_thread/thread/dd7f65d589023380/">yet</a> inconsistent handling of subscription content comes to mind as an ideal playground for such a robots directive in combination with a policy change.&lt;/speculation&gt; Anyways, there is a need for a NOPREVIEW robots tag, so why not implement it now? Thanks in advance.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/5277547113690850141/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=5277547113690850141' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/5277547113690850141'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/5277547113690850141'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/08/nopreview-missing-x-robots-tag.html' title='NOPREVIEW - The missing X-Robots-Tag'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-8010108623946147087</id><published>2007-08-01T11:09:00.000-04:00</published><updated>2007-08-01T16:54:51.896-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="fun"/><category scheme="http://www.blogger.com/atom/ns#" term="search quality"/><title type='text'>SEOs home alone - Google's nightmare</title><content type='html'>Being a single parent of three monsters at the moment brings me newish insights. I now deeply understand the pain of father Google dealing with us, and what doing the chores all day long means to Matt's gang in building 43, Dublin, and whereever. What a nightmare of a household. If you don't suffer from an offspring plague you won't believe what sneaky and highly intelligent monsters having too much time on their tiny greedy hands will do to gain control over their environment. Outsmarting daddy is not a hobby, it's their mission, and everything in perfect order is attackable. Each of them tries to get as much attention as possible, and if nothing helps, negative attention is fine too. There's no such thing as bad traffic, err ... mindfulness. Every rule is breakable, and there's no way to argue seriously with a cute 5 yo gal burying her 3 yo brother in the mud whilst honestly telling me that she has nothing to do with the dirty laundry because she never would touch anything hanging on the clothesline. Then my little son speaks out telling me that's all her fault, so she promises to do it never, never, never again in her whole life and even afterwards. In such a situation I've not that much options: I archive my son's paid links report, accept her reconsideration request but throttle her rankings for a while, recrawl and remove the unpurified stuff from the ... Oups ... I clear the scene with a pat on her muddy fingers, forgive all blackhatted kids involved in the scandal and just do the laundry again, writing a note to myself to improve the laundry algo in a way that muddy monsters can't touch laundered bed sheets again. Anything not on the explicit don'ts list goes, so while I'm still stuffing the washer with muddy bed sheets I hear a weird row in the living room. Running upstairs I spot my 10 yo son and his friend playing soccer with a ball I had to fish out of a heap of broken crockery and uprooted indoor plants to confiscate it just two hours ago. Yelling that's against our well known rules and why the heck is that [...] ball in the game again I get stopped immediately by the boys. First, they just played soccer and the recent catastrophe was the result of a strictly forbidden basketball joust. I've to admit that I said they must not play basketball in the house. Second, it's my fault when I don't hide the key to the closet where I locked the confiscated ball away. Ok, enough is enough. I banned my son's friend and grounded himself for a week, took away the ball, and ran to the backyard to rescue two bitterly crying muddy dwarfs from the shed's roof. Later on, while two little monsters play games in the bath tub which I really don't want to watch too closely currently, I read a thread titled "Daddy is soooo unfair" in the house arrest forum where my son and his buddy tell the world that they didn't do anything wrong, just sheer whitehatted stuff, but I stole their toy and banned them from the playground. Sigh. I'm exhausted. I'm supposed to deliver a script to merge a few feeds giving fresh contents, a crawlability review, and whatnot tonight, but I just wonder what else will happen when I leave the monsters alone in their beds after supper and story hour, provided I get them into their beds without a medium-size flame war. Now I understand why another <a href="http://www.mattcutts.com/blog/" title="Matt Cutts">daddy</a> supplemented the <a href="http://www.google.com/" title="Google">family</a> with a <a href="http://googlewebmastercentral.blogspot.com/" title="Webmaster Central">mom</a>.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/8010108623946147087/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=8010108623946147087' title='6 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/8010108623946147087'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/8010108623946147087'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/08/seos-home-alone-googles-nightmare.html' title='SEOs home alone - Google's nightmare'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>6</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-5566110381611580083</id><published>2007-07-31T18:58:00.000-04:00</published><updated>2007-07-31T19:09:56.084-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term=".htaccess"/><category scheme="http://www.blogger.com/atom/ns#" term="crawler directives"/><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="robots meta tags"/><category scheme="http://www.blogger.com/atom/ns#" term="SEO"/><category scheme="http://www.blogger.com/atom/ns#" term="X-Robots-Tag"/><title type='text'>Handling Google's neat X-Robots-Tag - Sending REP header tags with PHP</title><content type='html'>It's a bad habit to tell the bad news first, and I'm guilty of that. Yesterday I linked to Dan Crow telling Google that the unavailable_after tag is useless IMHO. So todays post is about a great thing: REP header tags aka X-Robots-Tags, unfortunately mentioned as second news somewhat concealed in <a href="http://googleblog.blogspot.com/2007/07/robots-exclusion-protocol-now-with-even.html">Google's announcement</a>. The REP is not only a <a href="http://therep.org/">theatre</a>, it stands for <a href="http://www.robotstxt.org/wc/exclusion.html">Robots Exclusion Protocol</a> (<a href="http://www.smart-it-consulting.com/article.htm?node=140&page=46">robots.txt</a> and <a href="http://www.smart-it-consulting.com/article.htm?node=140&page=47">robots meta tag</a>). Everything you can shove into a robots meta tag on a HTML page can now be delivered in the HTTP header for any file type:<ul> <li>INDEX|NOINDEX - Tells whether the page may be indexed or not</li> <li>FOLLOW|NOFOLLOW - Tells whether crawlers may follow links provided on the page or not</li> <li>ALL|NONE - ALL = INDEX, FOLLOW (default), NONE = NOINDEX, NOFOLLOW</li> <li>NOODP - tells search engines not to use page titles and descriptions from the ODP on their SERPs.</li> <li>NOYDIR - tells Yahoo! search not to use page titles and descriptions from the Yahoo! directory on the SERPs.</li> <li>NOARCHIVE - Google specific, used to prevent archiving (cached page copy)</li> <li>NOSNIPPET - Prevents Google from displaying text snippets for your page on the SERPs</li> <li>UNAVAILABLE_AFTER: <a href="http://www.ietf.org/rfc/rfc0850.txt">RFC 850</a> formatted timestamp - Removes an URL from Google's search index a day after the given date/time</li></ul> So how can you serve X-Robots-Tags in the HTTP header of PDF files for example? Here is one possible procedure to explain the basics, just adapt it for your needs: Rewrite all requests of PDF documents to a PHP script knowing wich files must be served with REP header tags. You could do an external redirect too, but this may confuse things. Put this code in your root's .htaccess: <pre> RewriteEngine On RewriteBase /pdf RewriteRule ^(.*)\.pdf$ serve_pdf.php </pre> In /pdf you store some PDF documents and serve_pdf.php: <pre> ... $requestUri = $_SERVER['REQUEST_URI']; ... if (stristr($requestUri, "my.pdf")) { header('X-Robots-Tag: index, noarchive, nosnippet', TRUE); header('Content-type: application/pdf', TRUE); readfile('my.pdf'); exit; } ... </pre> This setup routes all requests of *.pdf files to /pdf/serve_pdf.php which outputs something like this header when a user agent asks for /pdf/my.pdf: <pre> Date: Tue, 31 Jul 2007 21:41:38 GMT Server: Apache/1.3.37 (Unix) PHP/4.4.4 X-Powered-By: PHP/4.4.4 X-Robots-Tag: index, noarchive, nosnippet Connection: close Transfer-Encoding: chunked Content-Type: application/pdf </pre> You can do that with all kind of file types. Have fun and say thanks to Google :)</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/5566110381611580083/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=5566110381611580083' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/5566110381611580083'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/5566110381611580083'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/handling-googles-neat-x-robots-tag.html' title='Handling Google's neat X-Robots-Tag - Sending REP header tags with PHP'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-501782309624335979</id><published>2007-07-30T20:20:00.000-04:00</published><updated>2007-08-02T17:15:42.856-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="crawler directives"/><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="robots meta tags"/><category scheme="http://www.blogger.com/atom/ns#" term="search quality"/><category scheme="http://www.blogger.com/atom/ns#" term="SEO"/><title type='text'>Unavailable_After is totally and utterly useless</title><content type='html'>I've a lot of respect for Dan Crow, but I'm struggling with my understanding, or possible support, of the <a href="http://googleblog.blogspot.com/2007/07/robots-exclusion-protocol-now-with-even.html">unavailable_after</a> tag. I don't want to put my reputation for <a href="http://sebastianx.blogspot.com/2007/05/yahoo-search-going-to-torture.html">bashing such initiatives from search engines</a> at risk, so sit back and grab your popcorn, here comes the roasting: As a Webmaster, I did not find a single scenario where I could or even would use it. That's because I'm a greedy traffic whore. A bazillion other Webmasters are greedy too. So how the heck is Google going to sell the newish tag to the greedy masses? Ok, from a search engine's perspective unavailable_after makes sound sense. Outdated pages bind resources, annoy searchers, and in a row of useless crap the next bad thing after an outdated page is intentional Webspam. So convincing the great unwashed to put that thingy on their pages inviting friends and family to granny's birthday party on 25-Aug-2007 15:00:00 EST would improve search quality. Not that family blog owners care about new meta tags, RFC 850-ish date formats, or search engine algos rarely understanding that the announced party is history on Aug/26/2007. Besides there may be painful aftermaths worth submitting a desperate call for aspirins the day after in the comments, what would be news of the day after expiration. Kinda dilemma, isn't it? Seriously, unless CMS vendors support the new tag, tiny sites and clique blogs aren't Google's target audience. This initiative addresses large sites which are responsible for a huge amount of outdated contents in Google's search index. So what is the large site Webmaster's advantage of using the <a href="http://www.highrankings.com/advisor/getting-into-google/">unavailable_after tag</a>? A loss of search engine traffic. A loss of link juice gained by the expired page. And so on. Losses of any kind are not that helpful when it comes to an overdue raise nor in salary negotiations. Hence the Webmaster asks for the sack when s/he implements Google's traffic terminator. Who cares about Google's search quality problems when it leads to traffic losses? Nobody. Caring Webmasters do the right thing anyway. And they don't need no more useless meta tags like unavailable_after. "We don't need no stinking metas" from "Another Brick in the Wall Part Web 2.0" expresses my thoughts perfectly. So what separates the caring Webmaster from the 'ruthless traffic junky' who Google wants to implement the unavailable_after tag? The traffic junkie lets his stuff expire without telling Google about it's state, is happy that frustrated searchers click the URL from the SERPs even years after the event, and enjoys the earnings from tons of ads placed above the content minutes after the party was over. Dear Google, you can't convince this guy. [It seems this is a post about repetitive "so whats". And I came to the point before the 4th paragraph ... wow, that's new ... and I've put a message in the title which is not even meant as link bait. Keep on reading.] So what does the caring Webmaster do without the newish unavailable_after tag? Business as usual. Examples: Say I run a news site where the free contents go to the subscription area after a while. I'd closely watch which search terms generate traffic, write a search engine optimized summary containing those keywords, put that on the sales pitch, and move the original article to the archives accessible to subscribers only. It's not my fault that the engines think they point to the original article after the move. When they recrawl and reindex the page my traffic will increase because my summary fits their needs more perfectly. Say I run an auction site. Unfortunately particular auctions expire, but I'm sure that the offered products will return to my site. Hence I don't close the page, but I search my database for similar offerings and promote them under a H3 heading like "[product] (stuffed keywords) is hot" /H3 P buy [product] here: /P followed by a list of identical products for sale or similar auctions. Say I run a poll expiring in two weeks. With Google's newish near real time indexing that's enough time to collect keywords from my stats, so the textual summary under the poll's results will attract the engines as well as visitors when the poll is closed. Also, many visitors will follow the links to related respectively new polls. From Google's POV there's nothing wrong with my examples, because the visitor gets what s/he was searching for, and I didn't cheat. Now tell me, why should I give up these valuable sources of nicely targeted search engine traffic just to make Google happy? Rather I'd make my employer happy. Dear Google, you didn't convince me. Update: <a href="http://internethunger.blogspot.com/2007/08/nobody-seems-to-realize-that.html">Tanner Christensen</a> posted a remarkable <a href="http://sphinn.com/story/1796#c2628">comment</a> at <a href="http://sphinn.com/story/1796">Sphinn</a>:<blockquote>I'm sure there is some really great potential for the tag. It's just none of us have a need for it right now. Take, for example, when you buy your car without a cup holder. You didn't think you would use it. But then, one day, you find yourself driving home with three cups of fruit punch and no cup holders. Doh! I say we wait it out for a while before we really jump on any conclusions about the tag.</blockquote><a href="http://www.johnon.com/">John Andrews</a> was the first to report an <a href="http://www.johnon.com/375/unavailable_after.html">evil use of unavailable_after</a>. Also, Dan Crow from Google announced a pretty neat thing in the same <a href="http://googleblog.blogspot.com/2007/07/robots-exclusion-protocol-now-with-even.html">post</a>: With the <a href="http://sebastianx.blogspot.com/2007/07/handling-googles-neat-x-robots-tag.html">X-Robots-Tag</a> you can now apply crawler directives valid in robots meta tags to non-HTML documents like PDF files or images.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/501782309624335979/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=501782309624335979' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/501782309624335979'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/501782309624335979'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/unavailableafter-is-totally-and-utterly.html' title='Unavailable_After is totally and utterly useless'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-2218993848098715041</id><published>2007-07-28T18:37:00.000-04:00</published><updated>2007-08-07T12:25:09.750-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="analytics"/><category scheme="http://www.blogger.com/atom/ns#" term="CTR"/><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="SEO"/><category scheme="http://www.blogger.com/atom/ns#" term="tools"/><title type='text'>Analyzing search engine rankings by human traffic</title><content type='html'>Recently I've discussed <a href="http://sebastianx.blogspot.com/2007/07/rediscover-googles-free-ranking-checker.html">ranking checkers</a> at several places, and I'm quite astonished that folks still see some value in ranking reports. Frankly, ranking reports are --in most cases-- a useless waste of paper and/or disk space. That does not mean that SERP positions per keyword phrase aren't interesting. They're just useless without context, that is traffic data. Converting traffic pays the bills, not sole rankings. The truth is in your traffic data. That said, I'd like to outline a method to get a particular useful information out of raw traffic data: underestimated search terms. That's not a new attempt, and perhaps you have the reports already, but maybe you don't look at the information which is somewhat hidden in stats ordered by success, not failure. And you should be --respective employ-- a programmer to implement it. The first step is gathering data. Create a database table to record all hits, then in a footer include or so, when the complete page got outputted already, write all data you have in that table. All data means URL, timestamp, and variables like referrer, user agent, IP, language and so on. Be a data rat, log everything you can get hold of. With dynamic sites it's easy to add page title, (product) IDs etcetera, with static sites write a tool to capture these attributes separately. For performance reasons it makes sense to work with a raw data table, which has just a primary key, to log the requests, and normalized working tables which have lots of indexes to allow aggregations, ad hoc queries, and fast reports from different perspectives. Also think of regular purging the raw log table and historization. While transferring raw log data to the working tables in low traffic hours or on another machine you can calculate interesting attributes and add data from other sources which were not available to the logging process. You'll need that traffic data collector anyway for a gazillion of purposes where your analytics software fails, is not precise enough, or just can't deliver a particular evaluation perspective. It's a prerequisite for the method discussed here, but don't build a monster sized cannon to chase a fly. You can <a href="http://oyoy.eu/site/sereferrer/">gather search engine referrer data from logfiles</a> too. For example an interesting information is on which SERP a user clicked a link pointing to your site. Simplified you need three attributes in your working tables to store this info: search engine, search term, and SERP number. You can extract these values from the HTTP_REFERER. http://www.google.com/search?q=keyword1+keyword2~ &ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a 1. "google" in the server name tells you the search engine. 2. The "q" variable's value tells you the search term "keyword1 keyword2". 3. The lack of a "start" variable tells you that the result was placed on the first SERP. The lack of a "num" variable lets you assume that the user got 10 results per SERP, so it's quite safe to say that you rank in the top 10 for this term. Actually, the number of results per page is not always extractable from the URL because it's pulled from a cookie usually, but not so many surfers change their preferences (e.g. less than 0.5% surf with 100 results <a href="http://www.cre8asiteforums.com/forums/index.php?showtopic=51682">according to</a> <a href="http://johnmu.com/">JohnMu</a> and my data as well). If you've got a "num" value then add 1 and divide the result by 10 to make the data comparable. If that's not precise enough you'll spot it afterwards, and you can always recalculate SERP numbers from the canned referrer. http://www.google.co.uk/search?q=keyword1+keyword2~ &hl=en&start=10&sa=N 1. and 2. as above. 3. The "start" variable's value 10 tells you that you got a hit from the second SERP. When start=10 and there is no "num" variable, most probably the searcher got 10 results per page. http://www.google.es/search?q=keyword1+keyword2~ &rls=com.microsoft:*&ie=UTF-8&oe=UTF-8&startIndex=~ &startPage=1 1. and 2. as above. 3. The empty "startIndex" variable and startPage=1 are useless, but the lack of "start" and "num" tells you that you've got a hit from the 1st spanish SERP. http://www.google.ca/search?q=keyword1+keyword2~ &hl=en&rls=GGGL,GGGL:2006-30,GGGL:en&start=20~ &num=20&sa=N 1. and 2. as above. 3. num=20 tells you that the searcher views 20 results per page, and start=20 indicates the second SERP, so you rank between #21 and #40, thus the (averaged) SERP# is 3.5 (provided SERP# is not an integer in your database). You got the idea, here is a <a href="http://www.joostdevalk.nl/wp-content/uploads/2007/07/google-url-parameters.pdf">cheat sheet</a> and <a href="http://code.google.com/apis/soapsearch/reference.html#2_1">official</a> <a href="http://code.google.com/enterprise/documentation/xml_reference.html">documentation</a> on Google's URL parameters. Analyze the URLs in your referrer logs and call them with cookies off what disables your personal search preferences, then play with the values. Do that with other search engines too. Now a subset of your traffic data has a value in "search engine". Aggregate tuples where search engine is not NULL, then select the results for example where SERP number is lower or equal 3.99 (respectively 4), ordered by SERP number ascending, hits descending and keyword phrase, break by search engine. (Why sorted by traffic descending? You have a report of your best performing keywords already.) The result is a list of search terms you rank for on the first 4 SERPs, beginning with keywords you've probably not optimized for. At least you didn't optimize the snippet to improve CTR, so your ranking doesn't generate a reasonable amount of traffic. Before you study the report, throw away your site owner hat and try to think like a consumer. Sometimes those make use of a vocabulary you didn't think of before. Research promising keywords, and decide whether you want to push, bury or ignore them. Why bury? Well, in some cases you just don't want to rank for a particular search term, [your product sucks] being just one example. If the ranking is fine, the search term smells somewhat lucrative, and just the snippet sucks in a particular search query's context, <a href="http://sebastianx.blogspot.com/2007/07/google-assists-serp-click-through.html">enhance your SERP listing</a>. Every once in a while you'll discover a search term making a killing for your competitors whilst you never spotted it because your stats package reports only the best 500 monthly referrers or so. Also, you'll get the most out of your rankings by optimizing their SERP CTRs. Be crative, over time your traffic database becomes more and more valuable, allowing other unconventional and/or site specific reports which off-the-shelf analytics software usually does not deliver. Most probably your competitors use standard analytics software, individually developed algos and reports can make a difference. That does not mean you should throw away your analytics software to reinvent the wheel. However, once you're used to self developed analytic tools you'll think of more interesting methods not only to analyse and monitor rankings by human traffic than you can implement in this century ;) Bear in mind that the method outlined above does not and cannot replace <a href="http://learn.wordtracker.com/articles/keyword-inspiration-aaron-wall-of-seobookcom-shares-his-secrets/">serious keyword research</a>. Another --very popular-- approach to get this info would be automated ranking checks mashed up with hits by keyword phrase. Unfortunately, Google and other engines do not permit automated queries for the purpose of ranking checks, and this method works with preselected keywords, that means you don't find (all) search terms created by users. Even when you compile your ranking checker's keyword lists via various keyword research tools, you'll still miss out on some interesting keywords in your seed list. Related thoughts: <a href="http://www.seo-scoop.com/2007/08/07/why-i-check-rankings/">Why regular and automated ranking checks are necessary when you operate seasonal sites</a> by <a href="http://www.seo-scoop.com/">Donna</a></content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/2218993848098715041/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=2218993848098715041' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/2218993848098715041'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/2218993848098715041'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/analyzing-search-engine-rankings-by.html' title='Analyzing search engine rankings by human traffic'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-2157938551174839056</id><published>2007-07-27T17:29:00.000-04:00</published><updated>2007-07-27T17:41:52.944-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="recommendations"/><category scheme="http://www.blogger.com/atom/ns#" term="SEO"/><category scheme="http://www.blogger.com/atom/ns#" term="tools"/><title type='text'>Rediscover Google's free ranking checker!</title><content type='html'>Nowadays we're searching via toolbar, personalized homepage, or in the browser address bar by typing in "google" to get the search box, typing in a search query using "I feel lucky" functionality, or -my favorite- typing in google.com/search?q=free+pizza+service+nearby. Old fashioned, uncluttered and nevertheless sexy user interfaces are forgotten, and pretty much disliked due to the lack of nifty rounded corners. Luckily Google still maintains them. Look at this beautiful SERP: <a href="http://www.smart-it-consulting.com/img/misc/google-free-ranking-checker.png"><img src="http://www.smart-it-consulting.com/img/misc/google-free-ranking-checker.png" width="99%" border="0" alt="Google's free ranking checker" title="Google's free ranking checker" /></a> It's free of personalized search, wonderful uncluttered because the snippets appear as tooltip only, results are nicely numbered from 1 to 1,000 on just 10 awesome fast loading pages, and when I've visited my URLs before I spot my purple rankings quickly. <a href="http://www.google.com/ie?num=100&hl=en&safe=off&q=minimalist">http://google.com/ie?num=100&q=keyword1+keyword2</a> is an ideal free ranking checker. It supports &filter=0 and other URL parameters, so it's a perfect tool when I need to lookup particular search terms. Mass ranking checks are totally and utterly useless, at least for the average site, and <a href="http://www.google.com/support/webmasters/bin/answer.py?answer=66357">penalized</a> by Google. Well, I can think of ways to semi-automate a couple queries, but honestly, I almost never need that. Providing fully automated ranking reports to clients gave SEO services a more or less well deserved snake oil reputation, because nice rankings for preselected keywords may be great ego food, but they don't pay the bills. I admit that with some setups automated mass ranking checks make sense, but those are off-topic here. By the way, <a href="http://sebastianx.blogspot.com/2007/07/google-assists-serp-click-through.html">Google's query stats</a> are a pretty useful resource too.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/2157938551174839056/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=2157938551174839056' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/2157938551174839056'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/2157938551174839056'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/rediscover-googles-free-ranking-checker.html' title='Rediscover Google's free ranking checker!'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-4104560206068393798</id><published>2007-07-25T17:13:00.000-04:00</published><updated>2007-07-26T04:16:46.814-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="Blogger"/><category scheme="http://www.blogger.com/atom/ns#" term="duplicate content"/><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="robots.txt"/><title type='text'>Blogger to rule search engine visibility?</title><content type='html'>Via <a href="http://groups.google.com/group/Google_Webmaster_Help-Indexing/browse_thread/thread/1164d12a090ecd00/">Google's Webmaster Forum</a> I found this curiosity: <a href="http://www.stockweb.blogspot.com/robots.txt" title="Equal to http://stockweb.blogspot.com/robots.txt">http://www.stockweb.blogspot.com/robots.txt</a><blockquote>User-agent: * Disallow: /search Disallow: /</blockquote>A standard robots.txt at *.blogspot.com looks different:<blockquote>User-agent: * Disallow: /search Sitemap: http://*.blogspot.com/feeds/posts/default?orderby=updated</blockquote> According to the <a href="http://groups.google.com/group/Google_Webmaster_Help-Indexing/msg/33203891c7fbbdb6">blogger</a> the blog is not private, what would explain the crawler blocking:<blockquote>It is a public blog. In the past it had a standard robots.txt, but 10 days ago it changed to "Disallow: /"</blockquote> Copyscape thinks that the blog in question shares a fair amount of content with other Web pages. So does blog search: <code style="font-size:6pt; color:black; font-weight:bold; background:white;">http://stockweb.blogspot.com/2007/07/ukraine-stock-index-pfts-gained-97-ytd.html</code> has a duplicate, posted by the same author, at <code style="font-size:6pt; color:black; font-weight:bold; background:white;">http://business-house.net/nokia-nok-gains-from-n-series-smart-phones/</code>, <code style="font-size:6pt; color:black; font-weight:bold; background:white;">http://stockweb.blogspot.com/2007/07/prague-energy-exchange-starts-trading.html</code> is reprinted at <code style="font-size:6pt; color:black; font-weight:bold; background:white;">http://business-house.net/prague-energy-exchange-starts-trading-tomorrow/</code> and so on. Probably a further investigation would reveal more duplicated contents. It's understandable that Blogger is not interested in wasting Google's resources by letting Ms. Googlebot crawl the same contents from different sources. But why do they block other search engines too? And why do they block the source (the posts reprinted at business-house.net state "Originally posted at [blogspot URL]")? Is this really censorship, or just a software glitch, or is it all the blogger's fault? Update 07/26/2007: The robots.txt reverted to standard contents for unknown reasons. However, with a shabby link neigborhood as expressed in the blog's footer I doubt the crawlers will enjoy their visits. At least the indexers will consider this sort of spider fodder nauseous.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/4104560206068393798/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=4104560206068393798' title='11 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/4104560206068393798'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/4104560206068393798'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/blogger-to-rule-search-engine.html' title='Blogger to rule search engine visibility?'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>11</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-4235401016853055260</id><published>2007-07-24T21:38:00.000-04:00</published><updated>2007-07-24T21:52:45.961-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="recommendations"/><title type='text'>Hey, there is content in the widgets!</title><content type='html'>Yeah, I do know the layout of this blog is somewhat cluttered. Especially the sidebar with all the JS script calls slowing down page loads. Not that Blogger page load times are exiting at all, especially not with the classic template. Forgive me, I just can't stay away from fancy stuff. Perhaps you're not exactly interested in my <a href="http://twitter.com/SebastianX">twits</a> telling you that my monsters are asleep and I can code untroubled, or that I've dugg or sphunn my friends' posts. Perfectly legit votings of course, since we share that many interests so that I often like what my buddies write and submit to whatever social bookmarking services or communities. Of course you couldn't care less on stats like how many blogs in the Technorati universe (which is a tiny subset of the GoogleBlogSearch universe, which is a tiny subset of the blogosphere, which is a tiny subset of the Web ... Ok, you don't give a f***) link to my pamplets. Actually, here you could help me out, just put me on your blogroll. Honestly, the lack of backlinks is scandalous. Everybody reads my stuff but very few of you dear readers link to me. I don't consider scrapers readers, so their links don't count. Since my audience consists of 99% Webmasters, I hope all of you understand the syntax of my beloved <a href="http://www.smart-it-consulting.com/article.htm?node=155&page=90">A element</a>. I promote lots of nice folks in my diverse blogroll sections, but very few return the honor. Not even the <a href="http://googleblog.blogspot.com/">Google blog</a> lists me under "What We're Reading" (please notice the capital "W" indicating a pluralis majestatis), although I spam FeedFetcher with Google bashing quite frequently. Weird ... And no, the MBL users list doesn't count as content (but it's nice to see who visited), and the AdSense stuff is just informational (and remains unclicked by the way, you guys and gals are way too savvy). Oups, I did it again: four inexpressively paragraphs before I come to the point - vice. Since I add widgets when I discover them, you've to scroll down for the <a href="http://www.google.com/reader/">GoogleReader</a> <a href="http://www.google.com/reader/shared/17784309474383380743">thingy</a>. It's titled "Sebastian's picked gems", and I mean that. When I stumble upon a great post, I share it. That does not mean that I agree 100%, perhaps I even disagree 100%, but when I share a post I believe it's worth reading. Honestly, you wouldn't read my pamplets if you wouldn't share (a few of) my pet peeves, would you? I guess it's safe to assume that you'll enjoy reading my shared articles. Good news is, you can subscribe to the <a href="http://www.google.com/reader/public/atom/user/17784309474383380743/state/com.google/broadcast">feed of my selected readings</a>. I don't recycle news, so I don't blog every tidbit I find on the 'Net. Hence you should subscribe to the feed and read the content I'd like to have on my blog although I'm too busy (Ok Ok, that's just a lame excuse for laziness) to publish it myself. If you read my blog in your preferred feed reader, you'll miss out on some exciting stuff!</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/4235401016853055260/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=4235401016853055260' title='4 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/4235401016853055260'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/4235401016853055260'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/hey-there-is-content-in-widgets.html' title='Hey, there is content in the widgets!'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>4</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-8580325342633645050</id><published>2007-07-16T19:10:00.000-04:00</published><updated>2007-07-16T19:17:33.036-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term=".htaccess"/><category scheme="http://www.blogger.com/atom/ns#" term="404grabber"/><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="hotlinking"/><category scheme="http://www.blogger.com/atom/ns#" term="SEO"/><category scheme="http://www.blogger.com/atom/ns#" term="testing"/><category scheme="http://www.blogger.com/atom/ns#" term="tools"/><category scheme="http://www.blogger.com/atom/ns#" term="Webmaster Central"/><title type='text'>Getting the most out of Google's 404 stats</title><content type='html'>The 404 reports in Google's Webmaster Central panel are great to debug your site, but they contain URLs generated by invalid --respectively truncated-- URL drops or typos of other Webmasters too. Are you sick of wasting the link love from invalid inbound links, just because you lack a suitable procedure to 301-redirect all these 404 errors to canonical URLs? Your pain ends here. At least when you're on a *ix server running Apache with PHP 4+ or 5+ and .htaccess enabled. (If you suffer from IIS <a href="http://www.google.com/search?num=100&hl=en&safe=off&q=funeral+services">go</a> search another hobby.) I've developed a tool which grabs all 404 requests, letting you map a canonical URL to each 404 error. The tool captures and records 404s, and you can add invalid URLs from Google's 404-reports, if these aren't recorded (yet) from requests by Ms. Googlebot. It's kinda layer between your standard 404 handling and your error page. If a request results in a 404 error, your .htaccess calls the tool instead of the error page. If you've assigned a canonical URL to an invalid URL, the tool 301-redirects the request to the canonical URL. Otherwise it sends a 404 header and outputs your standard 404 error page. Google's 404-probe requests during the Webmaster Tools verification procedure are unredirectable (is this a word?). Besides 1:1 mappings of invalid URLs to canonical URLs you can assign keywords to canonical URLs. For example you can define that all invalid requests go to <code>/fruit</code> when the requested URI or the HTTP referrer (usually a SERP) contain the strings "apple", "orange", "banana" or "strawberry". If there's no persistent mapping, these requests get 302-redirected to the guessed canonical URL, thus you should view the redirect log frequently to find invalid URLs which deserve a persistent 301-redirect. Next there are tons of bogus requests from spambots searching for exploits or whatever, or hotlinkers, resulting in 404 errors, where it makes no sense to maintain URL mappings. Just update an ignore list to make sure those get 301-redirected to <code>example.com/goFuckYourself</code> or a cruel and scary image hosted on your domain or a free host of your choice. Everything not matching a persistent redirect rule or an expression ends up in a 404 response, as before, but logged so that you can define a mapping to a canonical URL. Also, you can use this tool when you plan to change (a lot of) URLs, it can 301-redirect the old URL to the new one without adding those to your .htaccess file. I've tested this tool for a while on a couple of smaller sites and I think it can get trained to run smoothly without too many edits once the ignore lists etcetera are up to date, that is matching the site's requisites. A couple of friends got the script and they will provide useful input. Thanks! <a href="http://www.smart-it-consulting.com/contact.htm?cSubject=404Grabber_BETA">If you'd like to join the BETA test drop me a message</a>. Disclaimer: All data get stored in flat files. With large sites we'd need to change that to a database. The UI sucks, I mean it's usable but it comes with the browser's default fonts and all that. IOW the current version is still in the stage of "proof of concept". But it works just fine ;)</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/8580325342633645050/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=8580325342633645050' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/8580325342633645050'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/8580325342633645050'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/getting-most-out-of-googles-404-stats.html' title='Getting the most out of Google's 404 stats'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-769881317777957433</id><published>2007-07-13T05:32:00.000-04:00</published><updated>2007-07-13T09:30:21.823-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="Blogger"/><category scheme="http://www.blogger.com/atom/ns#" term="social Web"/><title type='text'>Sphinn rocks</title><content type='html'>Thanks to <a href="http://searchengineland.com/070712-112832.php">Danny's crew</a> we've got a promising <a href="http://sphinn.com/">search geek community site</a>. Since I've recently started to <a href="http://sebastianx.blogspot.com/2007/07/now-powncing.html">deal with invites</a>, here is the top secret link where you get your <a href="http://sphinn.com/register">free Sphinn invite</a>. Click it now and join today, as Gorbachev said 'those who are late will be punished by life itself' <a href="http://en.wikipedia.org/wiki/Special:Search?search=those+who+are+late+will+be+punished+by+life&go=Go" rel="nofollow nocontent" title="laughable nofollow nocontent">;)</a> <a href="http://sebastianx.blogspot.com/2007/02/why-proper-error-handling-is-important.html">Previous</a> <a href="http://sebastianx.blogspot.com/2007/04/please-dont-run-your-counter-on-my.html">experiments</a> revealed that my pamphlets aren't <a href="http://www.vanessafoxnude.com/2007/04/27/when-you-dont-make-an-effort-on-digg-you-still-get-buried/">diggworthy</a>, despite the presence of OL/UL lists. Because I mention search and stuff like that every once in a while, I decided to submit a <a href="http://sebastianx.blogspot.com/2007/07/why-ebay-and-wikipedia-rule-googles.html">horror story</a> to Sphinn to test the waters over there. Adding <a href="http://sphinn.com/tools.php">Sphinn-it! widgets</a> to my posts hopefully helps promoting Sphinn, but with Blogger that turned into kinda nightmare. To prevent you from jumping through infinite try-and-error hoops, here is how it works: Classic templates: Search for $BlogItemBody$ and below the &lt;/div&gt; put <code style="color:black; font-size:8pt;">&lt;script type='text/javascript'&gt;submit_url='&lt;$BlogItemPermalinkUrl$&gt;';&lt;/script&gt; &lt;script src='http://sphinn.com/evb/button.php' type='text/javascript'/&gt;&lt;/script&gt;</code> (Blogger freaks out when you omit the non-standard ;&lt;/script&gt; after the self-closing second tag, hence stick with the intentional syntax error.) Newish templates: Check "Expand Widget Templates" Search for data:post.body/ and below the &lt;/p&gt; put <code style="color:black; font-size:8pt;">&lt;b:if cond='data:post.url'&gt; &lt;p&gt;&lt;script type='text/javascript'&gt;submit_url='&lt;data:post.url/&gt;';&lt;/script&gt; &lt;script src='http://sphinn.com/evb/button.php' type='text/javascript'/&gt;&lt;/p&gt; &lt;/b:if&gt;</code> (After saving the changes Blogger replaces some single quotes with HTML entities, but it works though. Most probably one could do that in a more elegant way, but once I saw the badges pointing to the correct URL --both in the posts and on the main page-- I gave up.) Have fun sphinning my posts!</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/769881317777957433/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=769881317777957433' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/769881317777957433'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/769881317777957433'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/sphinn-rocks.html' title='Sphinn rocks'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-2640723045840111922</id><published>2007-07-11T05:36:00.000-04:00</published><updated>2007-07-12T09:16:21.653-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="SEO"/><category scheme="http://www.blogger.com/atom/ns#" term="Web development"/><category scheme="http://www.blogger.com/atom/ns#" term="Webmaster Central"/><title type='text'>Google helps those who help themselves</title><content type='html'>And if that's not enough to survive on Google's SERPs, try <a href="http://groups.google.com/group/Google_Webmaster_Help">Google's Webmaster Forum</a> where you can study <a href="http://groups.google.com/group/Google_Webmaster_Help/web">Adam Lasnik's FAQ</a> which covers even questions the <a href="http://www.google.com/support/webmasters/">Webmaster Help Center</a> provides no comprehensive answer for (<a href="http://sebastianx.blogspot.com/2007/06/help-google-revealing-secret-sauce.html">yet</a>), and where Googlers working in Google's Search Quality, Webspam, and Webmaster Central teams hang out. Google dumps all sorts of questioners to the forum, where a crowd of <a href="http://www.redcardinal.ie/google/03-07-2007/help-with-google/">hardcore volunteers</a> (aka regulars as Google calls them) invests a lot of time to help out Webmasters and site owners facing problems with the almighty Google. Despite the sporadic posts by Googlers, the backbone of Google's Webmaster support channel is this crew of regulars from all around the globe. Google monitors the forum for input and trends, and intervenes when the periodic scandal escalates every once in a while. Apropos scandal ... although the <a href="http://groups.google.com/group/Google_Webmaster_Help/about">list of top posters</a> mentions a few of the regulars, bear in mind that trolls come with a disgusting high posting cadency. Fortunately, currently the signal drowns the noise (again), and I appreciate very much that the Googlers participate more and more. Some of the regulars like <a href="http://groups.google.com/groups/profile?enc_user=VJCBDC8AAAC0ZCEBAysSlShC_gPAdXUZ0pfPHXAsFTfIf7fdVSPr2Fhwp5n7Y7Kfs3bwt7zc7Yo">seo101</a> don't reveal their URLs and stay anonymous. So here is an incomplete list of folks giving good advice:<ul> <li>Cass-Hacks - <a href="http://cass-hacks.com/">Powerful XHTML DHTML presentation and accessibility tools</a> (<a href="http://groups.google.com/groups/profile?enc_user=-wy2HTcAAABqX0ba682YTKfOntXCz4UiVbJfFxQtFW39Oq5yZAXhmW6c35L4nLp1ZqYcPETKBxF4M27fpdsypU4zjOmH7itB&amp;group=Google_Webmaster_Help">profile</a>)</li> <li>Christina - <a href="http://www.asymptoticdesign.com/">Accessible design and development</a> (<a href="http://groups.google.com/groups/profile?enc_user=HXkX4j4AAABqX0ba682YTKfOntXCz4Uig5xLrBsWWrtQZGhr0KlVQXe0216uj6t62Ejd4bYn5Yx6CsHMf11kBABi5zMsco0n&amp;group=Google_Webmaster_Help">profile</a>)</li> <li>Dockarl - <a href="http://www.utheguru.com/blix-krieg-info">BlixKrieg Wordpress Theme</a> <a href="http://groups.google.com/groups/profile?enc_user=o6OJJjIAAABqX0ba682YTKfOntXCz4UiS0I-G8MSm5BjSJQUJ687ToPT_UQzvfikPsfcDjCkbYAvPp-8dun_mzBPI6iup1z0&amp;group=Google_Webmaster_Help">profile</a></li><li>JHL - <a href="http://www.jlh-design.com/">JHL web design</a> (<a href="http://groups.google.com/groups/profile?enc_user=svA9jjAAAABqX0ba682YTKfOntXCz4UikJteFc_k5bLSZcoW-6W3asoe68QBJH_RySTVmHahkXs&amp;group=Google_Webmaster_Help">profile</a>)</li> <li>JohnMu - <a href="http://oyoy.eu/">Search Engine Tools</a> (<a href="http://groups.google.com/groups/profile?enc_user=Dbr0CzEAAABqX0ba682YTKfOntXCz4Ui0Fr9UyFshVn01J5UosTckbWuXo9Ypep3UVfam6wN95A77j2jf7J8ZFK392Ir2mQh&amp;group=Google_Webmaster_Help">profile</a>)</li> <li>Webado - <a href="http://www.webado.net/">Web Hosting and Design Canada</a> (<a href="http://groups.google.com/groups/profile?enc_user=jhT_ri8AAABqX0ba682YTKfOntXCz4Ui-cixqMc2C_Rh1Vk1bSakNNyquXRioEQ36gDaLSFyPfU&amp;group=Google_Webmaster_Help">profile</a>)</li><li>Phil Payne - <a href="http://www.isham-research.co.uk/">Website rescue, redesign and maintenance</a> (<a href="http://groups.google.com/groups/profile?enc_user=Rdjm8DwAAABqX0ba682YTKfOntXCz4UiJ6jRY2r_NyxnDjTOW_BCrXkQDm5YYQAOiT1hULWIT3aHrWZjt4iounepE6FMYRck&amp;group=Google_Webmaster_Help">profile</a>)</li> <li>Red Cardinal - <a href="http://www.redcardinal.ie/search-engine-optimisation/">Search Engine Optimization in Ireland</a> (<a href="http://groups.google.com/groups/profile?enc_user=AGS4Xj0AAAC0ZCEBAysSlShC_gPAdXUZSil49yEbUdO_O6oAxpVzUtl-jRbV0l5CNYEPjP3TyhjK3kEjetniRPMwo6pAlkBB&amp;group=Google_Webmaster_Help">profile</a>)</li> <li>DJC (Dori) - <a href="http://coplien.com/">We're coming to Wisconsin</a> (<a href="http://groups.google.com/groups/profile?enc_user=3GZb1SwAAABqX0ba682YTKfOntXCz4Ui6CdtEfxoaMXR79FFgv5alQwzuFtfKTpYKoeNA9f7mhY&group=Google_Webmaster_Help">profile</a>)</li> </ul>If I've missed anyone, please drop me a line (I stole the list above from JLH and Red Cardinal, so it's all their fault!). So when you're a Webmaster or site owner, don't hesitate to <a href="http://groups.google.com/group/Google_Webmaster_Help-Indexing/post">post</a> your Google related question (but read the FAQ before posting, and search for your topics), chances are one of these regulars or even a Googler offers assistance. Otherwise when you're questionless carrying a swag of valuable answers, <a href="http://groups.google.com/group/Google_Webmaster_Help/subscribe">join</a> the group and share your knowledge. Finally, when you're a Googler, donate the sites linked above a boost on the SERPs ;) Micro-meme started by <a href="http://www.jlh-design.com/2007/07/pa-rum-pum-pum-pum/">John Honeck</a>, supported by <a href="http://www.redcardinal.ie/google/03-07-2007/help-with-google/">Richard Hearne</a>, <a href="http://bertvierstra.com/2007/07/12/google-forum/">Bert Vierstra</a> ...</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/2640723045840111922/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=2640723045840111922' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/2640723045840111922'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/2640723045840111922'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/google-helps-those-who-help-themselves.html' title='Google helps those who help themselves'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-1064929488911272482</id><published>2007-07-09T05:54:00.000-04:00</published><updated>2007-07-09T16:39:16.980-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="pownce"/><category scheme="http://www.blogger.com/atom/ns#" term="twitter"/><title type='text'>Now Powncing</title><content type='html'><a href="http://www.jlh-design.com/2007/07/pownce/"><img src="http://www.jlh-design.com/wp-content/uploads/2007/07/idontgetit.jpg" border="0" width="50%" alt="John, thanks for the invite!" title="John, thanks for the invite!" align="left" style="margin:5px;" /></a>Inspired by all the <a href="http://twitter.com/SebastianX">twits</a> about <a href="http://pownce.com/SebastianX">pownce</a> I submitted my email addy too. What a useless procedure. From inside there's no list of submitted email addresses to pick friends from. Or I'm too blind to find that page. Probably the best procedure to get rid of the 6 invites is to sell them at eBay. Perhaps Pownce releases 6 new invites then and I get rich quick. Wait ... I've a better idea. Submit your honest review of this blog in the comments and <a href="http://www.smart-it-consulting.com/contact.htm?cSubject=I_need_a_Pownce_invite">send me</a> the email addy for your invite. If your piece is funny or honest or vilifying enough to make me laugh I might invite you ;) Ok, so what separates Pounce from Twitter and WS_FTP? Here are my first impressions. Unfortunately, I will not see the ads, never. Hectic clicking on all links signed me up as a pro-member by accident. <a href="http://pownce.com/SebastianX"><img src="http://www.smart-it-consulting.com/img/misc/pownce-pro-26098_large-red-crab.png" border="0" width="25%" align="right" style="margin:5px;" alt="pro-crab" title="professional crab" /></a> Now Pownce blemishes my cute red crab with a "pro" label. I guess I got what I paid for. Paid? Yep, that's the first difference, Pownce is not completely free. Spamming friends in 100 meg portions costs an annual fee of 20 bucks. Next difference. There is no 140 bytes per message limit. Nice. And the "Send to" combo box is way more comfortable than the corresponding functionality at Twitter. I miss Twitter's "command line options" like "d username" and "@username". Sounds schizophrenic perhaps, but I'm just greedy. I figured out how to follow someone without friending. Just add somebody as friend and (you don't need to) wait for the decline, this makes you a fan of other users. You get their messages but not the other way round. Twitter's "add as friend" and "follow user" is clearer I think. <a href="http://pownce.com/SebastianX/notes/106187/">Searching</a> for the IM setup I learned there's none. Pownce expert John <a href="http://pownce.com/SebastianX/notes/106175/">said</a> I've to try the desktop thingy but it looks like AIM 1999, so I refuse the download and stick with the Web interface until Pownce interacts with GTalk. The personal pounce page has a refresh link at least, but no auto-refresh like Twitter. There's no way to bookmark messages or threads yet, and the link to the particular messages is somewhat obfuscated. The "email a bug report" is a good replacement for a "beta" label. I guess I'll use it to tell Pownce that I hate their link manipulation applying rel-nofollow crap. I'll play with the other stuff later on, the daddy-cab is due at the kindergarden. Hopefully, when I return, there will be a Pownce badge available for this blog, I've plenty of white space left on my sidebar. <hr color="gray" height="2"> Back, still no badge, but I realized that I forgot to mention the <a href="http://www.uncov.com/2007/6/29/kevin-rose-you-are-doing-it-wrong">FTP similarities</a>. And there is no need to complete this post, since I found Tamar's brilliant <a href="http://www.techipedia.com/2007/07/02/pownce-vs-twitter/">Twitter vs. Pownce</a> article. Update: <a href="http://www.downloadsquad.com/2007/07/09/posting-to-pownce-and-twitter-at-the-same-time/">How to post to Twitter and Pownce at the same time</a> (a <a href="http://www.twitterfeed.com/">Twitterfeed</a> work around, I didn't test this configuration)</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/1064929488911272482/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=1064929488911272482' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/1064929488911272482'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/1064929488911272482'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/now-powncing.html' title='Now Powncing'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-133086009096526788</id><published>2007-07-06T04:10:00.000-04:00</published><updated>2007-07-07T20:27:08.790-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="link building"/><category scheme="http://www.blogger.com/atom/ns#" term="nofollow"/><category scheme="http://www.blogger.com/atom/ns#" term="SEO"/><category scheme="http://www.blogger.com/atom/ns#" term="Yahoo"/><title type='text'>LZZR Linking&trade;</title><content type='html'><img src="http://www.smart-it-consulting.com/img/misc/lzzr-link-love.png" border="0" width="80" height="15" align="left" style="margin-right:5px;" alt="LZZR Link Love" title="LZZR Link Love" />In <a href="http://lzzr.com/blog/why-it-is-good-to-link-to-lzzr/">why it is a good thing to link out loud</a> <a href="http://del.icio.us/lzzr">LZZR</a> explains a nicely designed <a href="http://anthologyoi.com/blogish/black-hat-seo-tricks-are-getting-trickier.html">method</a> to accelerate the power of inbound links. Unfortunately this technique involves <a href="http://pipes.yahoo.com/" rel="nofollow">Yahoo! Pipes</a>, which is evil. Certainly that's a nice tool to compose feeds, but <a href="http://sebastianx.blogspot.com/2007/04/yahoo-pipes-jeopardizes-integrity-of.html">Yahoo! Pipes automatically inserts the evil nofollow crap</a>. Hence using <a href="http://pipes.yahoo.com/pipes/pipe.info?_id=mqGJ4FEq3BGm84wwyzUFzw" rel="nofollow">Pipes' feed output</a> to amplify links faults caused by the auto-nofollow. I'm sure <a href="http://www.squidoo.com/lzzrnet/">LZZR</a> can replace this component with ease, if that's not done already.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/133086009096526788/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=133086009096526788' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/133086009096526788'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/133086009096526788'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/lzzr-linking.html' title='LZZR Linking&trade;'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-937016130678882064</id><published>2007-07-05T17:42:00.000-04:00</published><updated>2007-07-23T07:36:10.477-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="crap"/><category scheme="http://www.blogger.com/atom/ns#" term="fun"/><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="search quality"/><title type='text'>Why eBay and Wikipedia rule Google's SERPs</title><content type='html'><img src="http://www.smart-it-consulting.com/img/misc/eBaydia.jpg" align="left" border="0" style="margin:3px;" /> It's hard to find an obscure search query like [<a href="http://www.google.com/search?num=100&hl=en&safe=off&q=artificial+link">artificial link</a>] which doesn't deliver eBay spam or a Wikipedia stub within the first few results at Google. Although both Wikipedia and eBay are large sites, the Web is huge, so two that different sites shouldn't dominate the SERPs for that many topics. Hence it's safe to say that many nicely ranked search results at Googledia, pulled from eBaydia, are plain artificial positioned <a href="http://www.wolf-howl.com/case-study/wikipedia-im-in-ur-index-rulin-ur-serpz/">non-results</a>. Curious why my beloved search engine fails so badly, I borrowed a Google-savvy spy from <a href="http://www.grayhatnews.com/">GHN</a> and sent him to Mountain View to uncover the eBaydia ranking secrets. He came back with lots of pay-dirt scraped from DVDs in the safe of building 43. Before I sold Google's ranking algo to Ask (the price Yahoo! and MSN offered was laughable), I figured out why Googledia prefers eBaydia from comments in the source code. Here is the unbelievable story of a miserable failure: When Yahoo! launched <a href="http://mindset.research.yahoo.com/search.php?p=Lary+Page+Sergey+Brin">Mindset</a>, Larry Page and Sergey Brin threw chairs out of anger because Google wasn't able to accomplish such a simple task. The engineers, eager to fulfill their founder's wishes asap, tried to integrate mindset-functionality without changing Google's fascinating simple search interface (that means without a shopping/research slider). Personalized search still lived in the labs, but provided a somewhat suitable API (mega beta): <code style="color:black;">scanSearchersBrainForContext([search query])</code>. Not knowing that this function of personalized search polls a nano-bugging-device (pre alpha) which Google had not yet released nor implemented into any searcher's brain at this time, they made use of that piece of experimental code to evaluate the search query's context. Since the method always returned "false", though they had to deliver results quickly, they made up some return values to test their algo tweaks: <code style="color:black; font-size:9pt;">/* debug - praying S&L don't throw more chairs */ if (scanSearchersBrainForContext($searchQuery) === false) then { $contextShopping = "%ebay%"; $contextResearch = "%wikipedia%"; $context = both($contextShopping, $contextResearch); } else {[pretty complex algo])</code> This worked fine and found its way into the ranking algo under time pressure. The result is that with each and every search query where a page from eBay and/or Wikipedia is in the raw result set, those get a ranking boost. Sergey was happy because eBay is generally listed on page #1, and Larry likes the Wikipedia results on the first SERP. Tell me why the heck should the engineers comment out these made up return values? No engineer on this planet likes flying chairs, especially not in his office.<img src="http://www.smart-it-consulting.com/img/misc/Googledia.jpg" align="right" border="0" style="margin:3px;" width="60%" /> PS: <a href="http://searchengineland.com/070705-123736.php">Some SEOs</a> push Wikipedia <a href="http://www.wolf-howl.com/case-study/2007-tour-de-france-winner-wikipedia/">stubs</a> too.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/937016130678882064/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=937016130678882064' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/937016130678882064'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/937016130678882064'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/why-ebay-and-wikipedia-rule-googles.html' title='Why eBay and Wikipedia rule Google's SERPs'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-7351151380627540472</id><published>2007-07-05T13:13:00.001-04:00</published><updated>2007-07-05T14:30:13.853-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="fun"/><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="paid links"/><category scheme="http://www.blogger.com/atom/ns#" term="SEO"/><title type='text'>Who is responsible for the paid link mess?</title><content type='html'>Look at this graph showing the number of [<a href="http://www.google.com/trends?q=buy+link&ctab=0&hl=en&geo=all&date=all&sort=0">buy link</a>] searches since 2004: <a href="http://www.smart-it-consulting.com/img/misc/buy-link-trend.png"><img src="http://www.smart-it-consulting.com/img/misc/buy-link-trend.png" border="1" bordercolor="silver" width="99%" alt="" title="" /></a> Interestingly this search term starts out in September or October 2004, and shows a quite stable trend until the recent paid links debate started. Who or what caused SEOs to massively buy links since 2004? <ul> <li>The Playboy interview with Google cofounders Larry Page and Sergey Brin just before Google was about to go public?</li> <li>Google's IPO?</li> <li>Rumors that Google ran out of index space and therefore might restrict the number of doorway pages in the search index?</li> <li>Nick Wilson preparing the launch of Threadwatch?</li> <li>AdWords and Overture no longer running gambling ads?</li> <li>The Internet Advancement scandal?</li> <li>Google's shortage of beer at the SES Google dance?</li> <li>A couple <a href="http://maps.google.com/maps?um=1&tab=wl&hl=en&lr=&rls=GGGL%2CGGGL%3A2004-07%2CGGGL%3Aen&q=Ripon%20UK">UK</a> <a href="http://www.smart-it-consulting.com/img/misc/who-invented-bought-organic-rankings.png">based</a> SEOs invented bought organic rankings?</li> </ul> Seriously, buying links for rankings was an established practice way before 2004. If you know the answer, or if you've a somewhat plausible theory, leave it in the comments. I'm really curious. Thanks.</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/7351151380627540472/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=7351151380627540472' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/7351151380627540472'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/7351151380627540472'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/who-is-responsible-for-paid-link-mess.html' title='Who is responsible for the paid link mess?'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-3337297970136263560</id><published>2007-07-04T18:51:00.000-04:00</published><updated>2007-07-05T18:41:52.494-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="fun"/><category scheme="http://www.blogger.com/atom/ns#" term="social Web"/><title type='text'>Letting friends know you read their stuff</title><content type='html'>With various social tools and gadgets there are tons of opportunities to publically or privately show that you follow your friends. I can digg my friends' articles, or bookmark them at delicious, I can link to their posts via sharing in Google Reader, or after reading their posts in my preferred feed reader, I can click the link too just to push my red crab image to the top of their MBL and BUMPzee widgets. All that comes with common hassles. I want to use these social gadgets and services without jumps thru unintended hoops, that is I consider all the above mentioned methods to tell friends that I still love them diverting those services from their intended use. Also, not every friend of mine makes use of all these geeky tools, so I need to digg posts of A., to delicious articles by B., to share posts of C., and to visit the blogs of D., E. and F. just to show that I've read their stuff in my feed reader. I can't do that, at least not in a reliable manner, especially not when I'm swamped and just try to catch up after 12 or more hours of dealing with legacy applications or other painful tasks like meetings with wannabe-geeks (unexperienced controllers or chiefs of whichever-useless-service-center) respectively anti-geeks (know-it-all but utterly-clueless and dangerous-to-the-company's-safety IT managers). Doh! So when I'm not able to send my friends a twitter-great-job-message or IM, and don't have the time to link to their stuff, should I feel bad? Probably. Penalties are well deserved. Actually, the consequence is that nice guys like <a href="http://twitter.com/NickWilson">Nick Wilson</a> @<a href="http://metaversed.com/">Metaversed</a> unfriend me (among other well-meaning followers) at Twitter coz "I didn't provide useful input for a while", not knowing that I follow them with interest, read their posts and all that, but just can't contribute at the moment because their actual field of interest doesn't match my time schedule respectively my todays-hot-topic-list, nor my current centre of gravity, so to say. That does not mean I'm not interested in whatever they do and output, I just can't process it ATM but I know that'll change at some point in the future. Hey, geeks usually hop from today's hot thing to tomorrow's hot thing, and flashbacks are rather natural, so why expect continuousness? Bugger, I wrote four paragraphs and didn't come to the point expectable from the post's title. And I bored you dear readers with lots of title bait recently. Sorry, but I did enjoy it. Ok, here's the message: Everybody monitors referrer stats. Don't say you don't do it because that's first a lie and second a natural thing to do. That applies to ego searches too by the way. So why don't we make use of referrer spoofing to send a signal to our friends? It's that easy. Just add the <a href="http://sebastianx.blogspot.com/2007/06/referrer-spoofing-with-prefbar-341.html">referrer-spoofing widget</a> to your <a href="http://prefbar.mozdev.org/">PrefBar</a>, enter your URL, and surf on. Well, technically that's <a href="http://sebastianx.blogspot.com/2007/05/when-your-referrer-stats-turn-into-porn.html">referrer spamming</a>, so if you wear a tinfoil hat use a non-indexable server like example.com. I'm currently surfing with the HTTP_REFERER "http://www.example.com/gofuckyourself" but I'm going to change that to this blog's URL. Funny folks visiting my blog provide bogus referrers like "http://spamteam.google.com/" and "http://corp.google.com:8080/webspam/watchlist.py", so why the fuck shouldn't I use my actual address? This will tell my friends that I still love them. And real geeks shouldn't expect unforged referrer stats, since many <a href="http://www.mattcutts.com/blog/">nice guys</a> surf without spamming the server logs with a referrer. What do you think?</content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/3337297970136263560/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=3337297970136263560' title='8 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/3337297970136263560'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/3337297970136263560'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/letting-friends-know-you-read-their.html' title='Letting friends know you read their stuff'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>8</thr:total></entry><entry><id>tag:blogger.com,1999:blog-14517520.post-5191363964293073637</id><published>2007-07-04T12:24:00.000-04:00</published><updated>2007-07-09T16:20:06.969-04:00</updated><category scheme="http://www.blogger.com/atom/ns#" term="Anchor Text"/><category scheme="http://www.blogger.com/atom/ns#" term="CTR"/><category scheme="http://www.blogger.com/atom/ns#" term="Google"/><category scheme="http://www.blogger.com/atom/ns#" term="SEO"/><category scheme="http://www.blogger.com/atom/ns#" term="Webmaster Central"/><title type='text'>Google assists SERP Click-Through Optimization</title><content type='html'>Big Mama Google in her ongoing campaign to keep her search index clean assists Webmasters with reports allowing click-trough optimization of a dozen or so pages per Web site. Google launched these reports a while ago, but most Webmasters didn't make the best use of them. Now that <a href="http://www.vanessafoxnude.com/">Vanessa</a> has <a href="http://www.stonetemple.com/articles/interview-vanessa-fox.shtml">revealed her SEO secrets</a>, lets discuss why and how Google helps increasing, improving, and targeting search engine traffic. Google is not interested in gazillions of pages which rank high for (obscure) search terms but don't get clicked from the SERPs. This clutter tortures the crawler and indexer, and it wastes expensive resources the query engine could use to deliver better results to the searchers. Unfortunately, legions of clueless SEOs work hard to increase mount clutter by providing their clients with weekly ranking reports, what leads to even more pages which rank for (potentially money making) search phrases but appear on the SERPs with such crappy titles and snippets that not even a searcher coming with an IQ slightly below a slice of bread clicks them. High rankings don't pay the bills, converting traffic from SERPs on the other hand does. A nicely ranking page is an asset, which in most cases just needs a few minor tweaks to attract search engine users (mount clutter contains machine generated cookie-cutter pages too, but that's a completely other story). For example unattended pages gaining their SERP position from anchor text of links pointing to them often have a crappy click through rate (CTR). Say you've a page about a particular aspect of green widgets, which applies to widgets of all colors. For some reason folks preferring red widgets like your piece and link to it with "red widgets" as anchor text. The page will rank fine for [red widgets], but since "red widgets" is not mentioned on the page this keyword phrase doesn't appear on the SERP's snippets, not to speak of the linked title. Search engine users seeking for information on red widgets don't click the link about green widgets, although it might be the best matching search result. So here is the click-thru optimization process based on Google's query stats (it doesn't work with brand new sites nor more or less unindexed sites, because the data provided in Google's Webmaster Tools are available, reliable and quite accurate for somewhat established sites only): <a href="https://www.google.com/webmasters/">Login</a>, <a href="https://www.google.com/webmasters/tools/siteoverview?hl=en">choose a site</a> and go to <a href="https://www.google.com/webmasters/tools/querystats?siteUrl=http%3A%2F%2Fwww.example.com%2F&hl=en" rel="nofollow example">query stats</a>. In an ideal world you'll see two tables of rather identical keyword lists (all examples made up). <div class="smalltable"><table border="1" bordercolor="silver" cellspacing="0" cellpadding="3" width="99%"><tr><th style="margin-right:10px;">Top search queries</th><th style="margin-right:20px;">Avg. Pos.</th><th style="margin-right:5px;">Top SERP clicks</th><th style="">Avg. Pos.</th></tr><tr><td>1. web site design</td><td>5</td><td>1. web site design</td><td>4</td></tr><tr><td>2. google consulting</td><td>4</td><td>2. seo consulting</td><td>5</td></tr><tr><td>3. seo consulting</td><td>3</td><td>3. google consulting</td><td>2</td></tr><tr><td>4. web site structures</td><td>2</td><td>4. internal links</td><td>3</td></tr><tr><td>5. internal linkage</td><td>1</td><td>5. web site structure</td><td>3</td></tr><tr><td>6. crawlability</td><td>3</td><td>6. crawlability</td><td>5</td></tr></table></div> The "Top search queries" table on the left shows positions for search phrases on the SERPs, regardless whether these pages got clicks or not. The "Top search query clicks" table on the right shows which search terms got clicked most, and where the landing pages were positioned on their SERPs. If good keywords appear in the left table but not in the right one, you've CTR optimization potentials. The "average top position" might differ from todays SERPs, and it might differ for particular keywords even if those appear in the same line in both tables. Positioning fluctuation depends on a couple of factors. First, the position is recorded at the run time of each search query during the last 7 days, and within seven days a page can jump up and down on the SERPs. Second, positioning on for example UK SERPs can differ from US SERPs, so an average 3rd position may be a utterly useless value, when a page ranks #1 in the UK and gets a fair amount of traffic from UK SERPs, but ranks #8 on US SERPs and searchers don't click it because the page is about a local event near Loch Nowhere in the highlands. Hence refine the reports by selecting your target markets in "location", and if necessary "search type" too. Third, if these stats are generated based on very few searches and even fewer click throughs, they are totally and utterly useless for optimization purposes. Lets say you've got a site with a fair amount of <strike>Google</strike> search engine traffic, the next step is identifying the landing pages involved (you get only 20 search queries, so the report covers only a fraction of your site's pages). Pull these data from your referrer stats, or extract SERP referrers from your logs to create a crosstab of search terms from Google's reports per landing page. Although the click data are from Google's SERPs, it might make sense to do this job with a broader scope, that is including referrers from all major search engines. Now perform the searches for your 20 keyword phrases (just click on the keywords on the report) to check how your pages look at the SERPs. If particular landing pages trigger search results for more than one search term, extract them all. Then load your landing page, and view its source. Read your page first rendered in your browser, then check out semantic hints in the source code, for example ALT or TITLE text and stuff like that. Look at the <a href="http://www.smart-it-consulting.com/article.htm?node=155&page=90#a-anchor-text">anchor text</a> of incoming links (you can use <a href="https://www.google.com/webmasters/tools/links?siteUrl=http%3A%2F%2Fwww.example.com%2F&hl=en" rel="nofollow example">link stats</a> and <a href="https://www.google.com/webmasters/tools/pageanalysis?siteUrl=http%3A%2F%2Fwww.example.com%2F&hl=en#phrases" rel="nofollow example">anchor text stats</a> from Google, <a href="http://www.webuildpages.com/tools/default.htm">We Build Pages Tools</a>, ...) and other ranking factors to understand why Google thinks this page is a good match for the search term. For each page, let the information sink before you change anything. If the page is not exactly a traffic generator for other targeted keywords, you can optimize it with regard to a better CTR for the keyword(s) it ranks for. Basically that means use the keyword(s) naturally on all page areas where it makes sense, and provide each occurence with a context which hopefully makes it into the SERP's snippet. Make up a few natural sentences a searcher might have in mind when searching for your keyword(s). Write them down. Order them by their ability to fit the current page text in a natural way. Bear in mind that with personalized search Google could have scanned the searcher's brain to add different contexts to the search query, so don't concentrate too much on the keyword phrase alone, but on short sentences containing both the keyword(s), respectively their synonyms, and a sensible context as well. There is no magic number like "use the keywords 5 times to get a #3 spot" or "7 occurences of a keyword gain you a #1 ranking". Optimal keyword density is a myth, so just apply common sense by not annoying human readers. One readable sentence containing the keyword(s) might suffice. Also, emphasizing keywords (EM/I, STRONG/B, eye catching colors ...) makes sense because it helps catching the attention of scanning visitors, but don't over-emphasize because that looks crappy. The same goes for H2/H3/... headings. Structure your copy, but don't write in headlines. When you emphasize a word or phrase in (bold) red, then don't do that consistently but only in the most important sentence(s) of your page, and better only on the first visible screen of a longer page. Work in your keyword+context laden sentences, but -again!- do it in a natural way. You're writing for humans, not for algos which at this point already know what your page is all about and rank it properly. If your fine tuning gains you a better ranking that's fine, but the goal is catching the attention of searchers reading (in most cases just skimming) your page title and a machine generated snippet on a search result page. Convince the algo to use your inserted sentence(s) in the snippet, not keyword lists from navigation elements or so. Write a sensible summary of the page's content, not more than 200-250 characters, and put that into the description meta tag. Do not copy the first paragraph or other text from the page. Write the summary from scratch instead, and mention the targeted keyword(s). The first paragraph on the page can exceed the length of the meta description to deliver an overview of the page's message, and it should provide the same information, preferably in the first sentence, but don't make it longish. Check the TITLE tag in HEAD: when it is truncated on the SERP then shorten it so that the keyword becomes visible, perhaps move the keyword(s) to the beginning, or create a neat page title around the keyword(s). Do title changes very carefully, because the title is an important ranking factor and your changes could result in a ranking drop. Some CMSs change the URL without notice on changes of the title text, and you certainly don't want to touch the URL at this point. Make sure that the page title appears on the page too. Putting the TITLE tag's content (or a slight variation) in a H1 element in BODY cannot hurt. If you for some weird reasons don't use H-elements, then at least format it prominently (bold, different color but not red, bigger font size ...). If the page performs nice with a couple money terms and just has a crappy CTR for a particular keyword it ranks for, you can just add a link pointing to a (new) page optimized for that keyword(s), with the keyword(s) in the anchor text, preferably embedded in a readable sentence within the content (long enough to fill two lines under the linked title on the SERP), to improve the snippet. Adding a (prominent) link to a related topic should not impact rankings for other keywords too much, but the keywords submitted by searchers should appear in the snippet a short while after the next crawl. In such cases better don't change the title, at least not now. If the page gained its ranking solely from anchor text of inbound links, putting the search term on the page can give it a nice boost. Make sure you get an alert when Ms. Googlebot fetches the changed pages, and check out the SERPs and Google's click stats a few days later. After a while you'll get a pretty good idea of how Google creates snippets, and which snippets perform best on the SERPs. Repeat until success. Related post: <a href="http://www.naturalsearchblog.com/archives/2007/07/02/google-quality-scores-for-natural-search-optimization/">Google Quality Scores for Natural Search Optimization</a> by <a href="http://www.naturalsearchblog.com/archives/2007/07/02/google-quality-scores-for-natural-search-optimization/#comment-55146">Chris Silver Smith</a></content><link rel='replies' type='application/atom+xml' href='http://sebastianx.blogspot.com/feeds/5191363964293073637/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://www.blogger.com/comment.g?blogID=14517520&postID=5191363964293073637' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/5191363964293073637'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/14517520/posts/default/5191363964293073637'/><link rel='alternate' type='text/html' href='http://sebastianx.blogspot.com/2007/07/google-assists-serp-click-through.html' title='Google assists SERP Click-Through Optimization'/><author><name>Sebastian</name><uri>http://www.blogger.com/profile/11554855150060057562</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='16' height='16' src='https://img1.blogblog.com/img/b16-rounded.gif'/></author><thr:total>2</thr:total></entry></feed>

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

Download the "valid Atom 1.0" banner.
Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)
Add this HTML to your page (change the image src attribute if necessary):

<a href="http://www.feedvalidator.org/check.cgi?url=http%3A//sebastianx.blogspot.com/atom.xml"><img src="valid-atom.png" alt="[Valid Atom 1.0]" title="Validate my Atom 1.0 feed" /></a>

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=http%3A//sebastianx.blogspot.com/atom.xml

Home · About · News · Docs · Terms