0

I have this robots.txt file:

User-agent: *
Sitemap: Path_to_sitemap.xml

My Q is, should I have something else in there as well? Like allow All or something?

Thanks

Anonymous12345
  • 1,012
  • 1
  • 12
  • 17

2 Answers2

2

It is a valid robots.txt. You may read this post from google: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449

0

Depends on what you want ;-) You don´t have to allow sth. special, just block the things you wouldn´t have crawled by Google and Co.

Tobias H.
  • 119
  • 4