How to make changes in robot.txt so as to make sure certain pages are disallowed to be index
How to make changes in robot.txt so as to make sure certain pages are disallowed to be index
Webmaster tools has a robots.txt generator at: crawler access>generate robots.txt tab.
Over 7000 collected Internet Marketing Articles | -0-Money From the Web-0- | About Music Blog
Affiliate links - HostGator 30% off Code | MicroNicheFinder
Yes, Google Webmaster tools. The crawler access link is under Site Configuration. The page opens to test robots.txt and shows the content of your current robots.txt file if one is available. The next tab is generate robots.txt and provides a wizard driven interface. The results will show up in the box below so that you may copy them and paste into your robots.txt file or a notepad page that you will then save as robots.txt and upload to the root directory for your site. If you have a WP blog in a sub-folder you will need to move the robots.txt to the root directory.
Over 7000 collected Internet Marketing Articles | -0-Money From the Web-0- | About Music Blog
Affiliate links - HostGator 30% off Code | MicroNicheFinder
I had just noticed it shortly before franberries question. It may be a new feature, or one I had just overlooked in the past. There is a lot of depth to both Webmaster tools and Analytics, it is hard to keep track of all the features.New thing to me.
Over 7000 collected Internet Marketing Articles | -0-Money From the Web-0- | About Music Blog
Affiliate links - HostGator 30% off Code | MicroNicheFinder