diff options
Diffstat (limited to 'static/stage.robots.txt')
-rw-r--r-- | static/stage.robots.txt | 21 |
1 files changed, 21 insertions, 0 deletions
diff --git a/static/stage.robots.txt b/static/stage.robots.txt new file mode 100644 index 00000000..c708b608 --- /dev/null +++ b/static/stage.robots.txt | |||
@@ -0,0 +1,21 @@ | |||
1 | # | ||
2 | # robots.txt | ||
3 | # | ||
4 | # This file is to prevent the crawling and indexing of certain parts | ||
5 | # of your site by web crawlers and spiders run by sites like Yahoo! | ||
6 | # and Google. By telling these "robots" where not to go on your site, | ||
7 | # you save bandwidth and server resources. | ||
8 | # | ||
9 | # This file will be ignored unless it is at the root of your host: | ||
10 | # Used: http://example.com/robots.txt | ||
11 | # Ignored: http://example.com/site/robots.txt | ||
12 | # | ||
13 | # For more information about the robots.txt standard, see: | ||
14 | # http://www.robotstxt.org/robotstxt.html | ||
15 | # | ||
16 | # For syntax checking, see: | ||
17 | # http://www.frobee.com/robots-txt-check | ||
18 | |||
19 | User-agent: * | ||
20 | Disallow: / | ||
21 | |||