- Re: Dont Aggregrate Me
I’m with you on how robots.txt is to be interpreted, but to a point there is a point to the dodge. F.ex, your example of pointing an enclosure to a large file on a foreign server ...
- About robots.txt File
Join Date Nov 2005 Location Dodge City, KS Posts 864 ... I use this as my robots.txt file. User-agent: * Disallow: /images/ Disallow: /forums/ajax.php
- The Internet Manifesto - SmartPlanet
John Dodge, Dana Blankenhorn Thinking Tech. Bio - John Dodge; Bio - Dana Blankenhorn ... There is a way for any site to control how its content is syndicated, called a robots.txt ...
- Daily SearchCast, Feb. 7, 2006: BMW Banned By Google Over Spamming ...
... Feb. 7, 2006: BMW Banned By Google Over Spamming; Google Sitemaps Rolls Out Robots.txt ... Don Dodge Chats with Memeorandum Founder, Gabe Rivera ; Google and Fiat Partner to Offer ...
- Re: Dont Aggregrate Me
Its sad that such an inane dodge would even garner any attention at all, much less require a response. http://www.robotstxt.org/wc/faq.html What is a WWW robot?
|
