Spider Getting in the Loop
by Shruti Gupta - October 20th, 2005
I was attempting to create ROR Sitemap for Ecommerce Partners website and I noticed interesting phenomenon. In the article SHTML Makes Updating Website Easy, I have added some example of the code in the content:
& lt ; a href – “index.shtml” & gt ; home & lt ; /a & gt ;
You will notice that after href I have a “-” instead of “=“. Initially, I had “=” after href but the link was not clickable as I had replaced “>” with “& gt ;”. I didn’t ever think that the link can still be spidered until I discovered it while creating ROR sitemap. I noticed that search engine spider was trying to crawl it and since it’s relative URL, it was getting in a loop. To avoid it, I had to replace “=” with “-”. If anyone has a more sophisticated solution to this problem, please let me know.
I tried putting the code inside a text area in a form. Also, I tried to insert it between <code> HTML tag. No success! The only way it was possible was by changing “=” to “-” after HREF. Any thoughts?
Same isssue was going on with the codes in this article, How to Add Alternate StyleSheets?
