How to make changes in robot.txt so as to make sure certain pages are disallowed to be index
Printable View
How to make changes in robot.txt so as to make sure certain pages are disallowed to be index
Webmaster tools has a robots.txt generator at: crawler access>generate robots.txt tab.
You mean google webmaster tools?? i still confusing how to change, but i'll give a try soonQuote:
Originally Posted by wink0r
Yes, Google Webmaster tools. The crawler access link is under Site Configuration. The page opens to test robots.txt and shows the content of your current robots.txt file if one is available. The next tab is generate robots.txt and provides a wizard driven interface. The results will show up in the box below so that you may copy them and paste into your robots.txt file or a notepad page that you will then save as robots.txt and upload to the root directory for your site. If you have a WP blog in a sub-folder you will need to move the robots.txt to the root directory.
Quote:
Originally Posted by wink0r
New thing to me. I was also looking for some plugins but you have provided a simple way for this. Thanks, i am going to check it.
I had just noticed it shortly before franberries question. It may be a new feature, or one I had just overlooked in the past. There is a lot of depth to both Webmaster tools and Analytics, it is hard to keep track of all the features.Quote:
New thing to me.