PDA

View Full Version : Problem with image rating and yahoo slurp


reflejandome
04-12-2005, 11:00 PM
Hi

YahooSlurp, the web spider from yahoo usually visit my site for indexing, but the spider follow the url likes ?showimage=23&rating=2 , so, whe it try to follow the link ,it insert a vote that is not true
is any manner to stop this, so the site can be indexing but no insert false votes ?? thx

Rob
04-13-2005, 03:45 AM
I added rel='nofollow' to all of the tags so that search engines would not follow the links, so if yahoo is, then they are not paying attention to the tags :evil:

The only other way I can think of doing it is to add the URL for the tags to a robots.txt file in the root of your website. I am not sure if Yahoo supports wildcards, but if they do (google does), the robots.txt would look like;

User-agent: *
Disallow: /?showimage=*&rating=*
Disallow: /index.php?showimage=*&rating=*

Note: The * in the disallow line is not standard compliant, but I believe search engines are allowing it. It is the only way to do the disallow though without using URL rewriting.

Anyone have better ideas?

reflejandome
04-13-2005, 07:54 AM
Version 1.2 has rel='nofollow' but 1.1 , version i was using perhaps not, i am using now version 1.2 , i hope it works so human vote my images, not only yahoo and google :)

Rob
04-13-2005, 01:12 PM
I hope it works too. Let me know if you notice any search engines crawling those links because it is something that worries me ;)

reflejandome
04-13-2005, 02:29 PM
I tried with only rel='nofollow' with version 1.2 and it doesn't work :( , now i am goin to try with robots.txt to see what happen now....

raminia
04-13-2005, 03:04 PM
what if yahoo tries to crawl my site too?;)

reflejandome
04-13-2005, 04:55 PM
:cry: i dont know why yahoo craw mi syite so many times, but i think robots.txt doesnt work. I read that Yahoo! Slurp adheres to the 1994 Robots Exclusion Standard and so dont like the *. And it ignores rel='nofollow' (perhaps by the ' not " ??).
Any idea ?? :(

Rob
04-13-2005, 06:01 PM
If we modified the addon to reverse the order of the parameters, then we should be able to write a robots.txt that will work, although it will take more lines.

User-agent: *
Disallow: /?rating=
Disallow: /index.php?rating=

Anything blank is supposed to be a wildcard by the standard, so this should work. Let me know how it goes or if you need help modifying the addon.

reflejandome
04-13-2005, 08:04 PM
thx , i will try it to see what happen, if yahoo webcrawel will give me only five stars ... :D

reflejandome
04-14-2005, 10:42 AM
If we modified the addon to reverse the order of the parameters, then we should be able to write a robots.txt that will work, although it will take more lines.

User-agent: *
Disallow: /?rating=
Disallow: /index.php?rating=

Anything blank is supposed to be a wildcard by the standard, so this should work. Let me know how it goes or if you need help modifying the addon.

I try it and it doesn't work .... so i decided to ignore yahoo while i found a way to resolver it :(