Robots.txt Http And Https - Part II

So I posted earlier on splitting up the robots.txt you are showing based on if the bot was connecting on http or https here. I commented at the end that if this is an existing site and Google has already indexed your https content you need to be careful. Once you block Google from reindexing your https content using the robots.txt file Google will ignore changes, but it will keep the old indexed pages.

You cannot use the URL removal tool because you can’t tell it to only remove https urls. The way you get around this is to remove the robots.txt restrictions from https and put meta tags for Google to remove the content only on our https content.

Tags you are going to want to user are:

and

This will tell Google to not index the pages and remove any old caches and indexes. This could take some time for Google to crawl all of your https content. Once everything you want out of their index is gone you can go ahead and change over to the easier https robots.txt file restrictions.

I do not have the code that allows you to set different meta tag info based on http or https. If someone has that please comment on this post. This should help out the SEO on your site. Thanks

Resource Page for You

Technorati Tags: robots.txt, IIS 6.0, HTTPS, Google Bot