It was thus said that the Great Ren? Wagner once stated:
> Hi,
>
> simple question:
> is the following robots.txt format valid in a form that the "disallow" is applied to all User-agents mentioned before?
> ---
> User-agent: researcher
> User-agent: indexer
> User-agent: archiver
> Disallow: about
> ---
That will work, but you need to add a leading '/' to the Disallow line:
Disallow: /about
That will match any request starting with '/about', like '/about',
'/aboutthis', '/about/that', etc.
> or do i need to be more chatty?
> ---
> User-agent: researcher
> Disallow: about
> User-agent: indexer
> Disallow: about
> User-agent: archiver
> Disallow: about
> ---
That will work too (same thing about the Disallow: line though). You can
read more about it at <
http://www.robotstxt.org/>.
-spc
![Atom Feed](data:image/svg+xml,<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg"
id="RSSicon"
viewBox="0 0 8 8" width="16" height="16">
<title>RSS feed icon</title>
<style type="text/css">
.button {stroke: none; fill: orange;}
.symbol {stroke: none; fill: white;}
</style>
<rect class="button" width="8" height="8" rx="1.5" />
<circle class="symbol" cx="2" cy="6" r="1" />
<path class="symbol" d="m 1,4 a 3,3 0 0 1 3,3 h 1 a 4,4 0 0 0 -4,-4 z" />
<path class="symbol" d="m 1,2 a 5,5 0 0 1 5,5 h 1 a 6,6 0 0 0 -6,-6 z" />
</svg>)