I had my website using dynamic urls and it got indexed from Google, now I changed it to SEF links, and I want to remove the dynamic URLS, because Google did not remove them although it indexed my website another 2 or 3 times after that... and I read it should be made via robots.txt, but how? will the following work?:
or I should useCode:User-agent: * Disallow: /index.php?cat=1 Disallow: /index.php?cat=7 Disallow: /index.php?cat=2 Disallow: /index.php?cat=4 Disallow: /index.php?cat=3 Disallow: /cool.php?url=flash/SammyCamilleri.swf Disallow: /review.php?sid=2&recat=4 Disallow: /review.php?sid=1&recat=3
Disallow: /index.php?
The google.com/robots.txt uses that code.but im not sure what to do!
You can simply use Disallow: ./index.php.
Well but that means that the bot will not crawl my website! But I want it to!
If you want the Googlebot to crawl through some pages, you simply don't but them in your robots.txt file. If you want the Googlebot to crawl through index.php then don't put it in your robots.txt - or do an:
You cannot allow - or disallow pages like index.php?id=X, only the page itself, index.php.Code:Allow: index.php
There are currently 1 users browsing this thread. (0 members and 1 guests)
Bookmarks