Referers counting and SPAM

[posefius]

How does Pixelpost count the number of referers? The last week I have many visitors from non photography sites, about 1500-2000 each day. It looks like SPAM. When I look at the referers stats I see sites like:

ww. your-onlinepharmacy.com
ww. rxmeds-now.com
ww. io-pharmacy.com

How can I stop this?

I also have Nedstat running where I dont see those sites and number of visitors. Please help me, just take a look at:

tle.osefius.nl

Thanks

[Joe[y]]

http://www.shiftedexposure.com/anti_spam.zip - hope that helps... courtesy of Rob!

it's an addon for removing referer spam.

[posefius]

Thank you, I installed the addon.

Its great to clean up the unwanted visitor stats. But I see it doesnt prevent the counting of new unwanted visits. Is there a way to prevent the counting? Or is there a way to protect me against those unwanted visitors?

[Wadem]

In the last week I've been getting some HEAVY refer spam traffic.

None of the guys have done any e-mail/spam commenting yet. I don't think the bots have found the comments section on my website yet.

# 1770 ://xbuy.bigsitecity.com/
# 211 ://fioricet.1.zs1.biz
# 197 ://tramadol.1.zs1.biz
# 172 ://viagra.1.zs1.biz
# 12 ://www.izhuqiu.com
# 10 //cialis-levitra-viagra.com.cn/
# 04 ://www.plasticmachinery.cn
# 03 ://www.czcn.cn
# 03 ://www.czpcsj.com
# 02 ://www.first-drugstore.com
# 02 ://www.my-online-pharmacy.net
# 02 ://www.buy-online-prescription-drugs.net
# 02 ://www.euro-drugs.com
# 02 ://www.my-drugs.com
# 02 ://www.first-pharmacy.net

posefius, it's hard to 'pre-empt' new spammers. It's always going to be an after the event action to remove them from your DB. You can build smarts into comment systems, but it's impossible to build a reffer system that can stop visitors counting for all spam domains. You could stop any reffers from any of the above being added to the database, but you're going 2 have to update it every time you get a new domain.

--Wadem

[Connie]

Wadem and Posefius,

I took off the "http" in your postings or modified it, I do not want these URLS sit in the forum as active urls...

I hope you understand!

Connie

[Wadem]

Believe me, you've done me a favour :) Sorry for leaving it there!!!!

I would of thought this forum would of had rel='nofollow' set on links, which is why I posted them @ all.

You're too good to us Connie!

:oops:

Thanks,
--Wade

[posefius]

I understand. It seems to be a big problem preventing yourself against spammers. Most of the traffic is spam traffic.
I hope some day we get rid of it.

Piet Osefius

[Wadem]

The stats say that if 1 person in 10,000 buys something, the spammers are able to make money. The whole world needs to become savvy to the ways of the scammer. Mind you, people still follow real life pyramid schemes, and snail-mail scams.

I think there will always be scammer/spammers online, it's just a game of cat and mouse. E-mail spam is on the way down (or meant to be in the next year or so due to the large IT players all developing authentication methods), so something else has to take it's place.

That said, the search engines did all meet up for a spam conference a few months ago, and the rel='nofollow' has come about. The balls is now rolling on the web front, but it's still got a while to go.

(can you tell I work in IT? :p)

Cheers,
--Wade

PS if you're happy updating with each new domain as they come along, Rob's solution will do the trick just nicely! It allows new inputs to be added on the fly. It's not as good as them never occuring, but better then doing it manually!

[posefius]

Thats right. I understand you cant protect yourself from being attacked by those spam bots. Someday you are being attacked.

But I still have a question about the addon. It removes the spam url's after they have been recorded into the database. But is there a way to make an addon or modification of Pixelpost that prevents recording the information into the database by using the same banlist. At what moment does the recording take place?

Piet Osefius

[Connie]

@wadem,

pixelpost.org had 21.000 spam referrers last night, nightmare!

you mention rel='nofollow', this we have integrated already

[raminia]

I got rid of these referer spams.

1) I took the link to referers page on my photoblog.
2) I changed the code in index.php and replcaded the "ref" with "refz" and "referer" with "refererz".
3) I renamed my referer page of the template to refererz_template.html

now If i want to see my referers I use index.php?x=refz in the browser. Soon, in the next version, referers page will be inside admin panel not the photoblog main page. This way the evil spams do not follow it.

[nadams]

I counted over 1600 referer spams in about 12 hours--even with the addon it is too much, so I got fed up and took the referer links out of the referer page html (see :nickadams.com/gumbo). I see that the spambots are still adding hits to my visitor counter, as I would expect, but since their links are not being displayed, am I defeating their intended purpose?

I will try raminia's solution when I get time--thanks for posting that. It is a real nuisance.

Nick

[blinking8s]

We will try to release somekind of update very soon to help users deal with this spam issue. It seems in the last two weeks pixelpost has come under heavy heavy attack from this plague. Its sad, but every application has its time...

[raminia]

I also used a banned list in .htaccess + Access Forbidden error for index.php?x=ref !

[Spudooli]

Wadem, I have exactly the same list of referrer spams as you, with very similar page counts as you for them.

They first started appearing yesterday.

I wonder if PP has got so popular that it's been targeted by those referrer log spammers?

At least I'm not getting the same as pixelpost.org, so I guess I'm lucky?

[Wadem]

I'm not the world's strongest coder here, so here's my attempt @ 2 solutions in plain text english.

1
------------------
Look @ the $referer_code.

If $referer_code == $last_referer_code X 10 in the last $x minutes block any more attempts, mask in DB, e-mail admin new referer blocked, else publish

This to me seems like an automated solution that would block domains, and easily be able to put back legit domains. $x would be the only value that would have to be set for high or low volume domains.

The DB Mask could be an extra field in the reffer log (Binary would do fine). If set to 1 don't display, if else, display. This way if something gets caught that wasn't meant to it could be set back to 0 by the user.

The other thing with having a '1' is that you could remove any items perminantley from the DB that have been flaged from the admin page when it's convinient for the end user.

2 (maybe a combination of both?)
------------

The Referer list, I would like to se be something like MT(either MT-Blacklist or Spamlookup) uses. SpamLookup lists the steps it uses to target spam @ (http://bradchoate.com/projects/spamlookup/wiki/SpamIdentification).

These ideas/method (to me), looks to be EXACTLY what we are all after.

---------

Spudooli: I've googled the domain, there's quite a lot of people bitching that they spam a lot.

Cheers,
--Wade

[er16004]

Count me in for that same referer list. Jerks. Came in this morning from a 3-day weekend and xbuy.bigsitecity.com (64.215.179.48) has had over 3000+ hits alone on my site. I've got Shifted Exposures' Anti-Spam addon installed and I love it. I just don't like the fact I have to go run it (or rather, click on it) every 30 minutes. Is there any way to setup something automated (via a Chron job?) to run at specified intervals to do this for me?

In the meantime, I'm implementing this .htaccess file to see if it will help:
<Files 403.shtml>
order allow,deny
allow from all
</Files>

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://yourdomain.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.yourdomain.com/.*$ [NC]
RewriteRule .*\.()$ - [F,NC]

SetEnvIfNoCase Referer ".*(casino|gambling|poker|porn|sex|nude|xxx|hilton| pics|video).*" BadReferrer order deny,allow deny from env=BadReferrer

deny from 217.26.151.9
deny from 81.3.150.1
deny from 69.57.130.140
deny from 62.146.13.150
deny from 66.235.180.55
deny from 81.3.150.6
deny from 200.189.226.83
deny from 8.4.112.98
deny from 195.2.72.155
deny from 64.136.24.160
deny from 64.136.24.165
deny from 209.63.57.10
deny from 69.50.175.187
deny from 64.191.102.235
deny from 64.215.179.48
deny from 66.79.179.151
deny from 198.63.210.127
deny from 64.136.24.162
deny from 64.237.48.241


Note: the IP's listed above should be a lot of the same ones listed in an earlier post here.

Anyone have any thoughts about this .htaccess file? I know it won't stop the spam, but I hope it reduces it. BTW, I'm not getting ANY more comment spam thanks to Connie's addon.

[Wadem]

I guess the bots got a list of sites running PP from the PP main page where it had all our sites randomly displayed. That's the only thing that connects all of us together. (Unless it used a bot on google looking for "index.php?x=ref". This shows 334 results).

Either way PP is being targeted :(

I don't actually have any comment spam yet, just referer spam.

The more popular PP gets, the more spam/bots it will attract (logically). The whole PP team are great coders and designer, I'm really interested in how they plan on tackling this problem.

The thing I don't like about .htaccess is that it's always a manual update, a manual FTP upload to the server. It's a manual process, it has no smarts, and no throttling by itself. It's a stopgap solution in my mind( my thoughts/ideas above).

--Wadem

[er16004]

(Unless it used a bot on google looking for "index.php?x=ref". This shows 334 results).

Yes, it looks like this is how they are coming into my site. I just looked at my logs a little closer and this is what they are pointing to. I also noticed that hits weren't coming just from the IP's I listed. There's all sorts of IP's that were referering from the one IP I banned, but they were still getting in.

This sucks. I'm leaning more towards the 'get even' route now.

raminia, any chance you could give us the code to replace our index.php file like you have?

[er16004]

Ok, I've finally found something that WORKS for STOPPING REFERER SPAM (at least, it works for me)!!! I found http://planetOzh.com and Ozh wrote a PHP script to refer spammers back to themselves. In other words, he wrote a little bit of genius.

Here's what I did to "install" it:

Copy the code below, make a new file called no-refer-spam.php and paste this code into that file.

<?php
/*
Script Name: No Refer Spam
Version: 1.00
Hack URI: http://frenchfragfactory.net/ozh/archives/2005/02/05/no-refer-spam/
Description: Send refer spammers back to their own sites
Author: Ozh
Author URI: http://planetOzh.com
*/

$spams = array (
"terashells.com", "chat-nett.com", "exitq.com", "cxa.de", "sysrem03.com",
"pharmacy.info", "guide.info", "drugstore.info",
"coresat.com", "psxtreme.com", "freakycheats.com", "cool-extreme.com",
"pervertedtaboo.com", "crescentarian.net", "texas-holdem", "fuck-fest", "yelucie.com",
"poker-online", "findwebhostingnow.com", "smsportali.net", "6q.org", "flowersdeliveredquick.com",
"ronnieazza", "lemonrider", "future-2000", "trackerom.com", "andrewsaluk.com", "4u.net", "4u.com", "doobu.com",
"nutzu", "italiancharms", "likejazz", "kloony", "isacommie.com", "musicbox1.com", "tigerspice", "roody.com",
"bigsitecity", "zs1.biz", "yesno.spb.ru", "newru.net", "9k.com", "cialis", "levitra", "viagra", "tramadol",
"phentermine", "7h.com", "hydrocodone"
);


$ref = $_SERVER["HTTP_REFERER"];

if ($ref) {
foreach ($spams as $site) {
$pattern = "/$site/i";
if (preg_match ($pattern, $ref)) {
header("Location: $ref"); exit();
}
}
}
?>

I saved this file into my Pixelpost root directory (which is also my severs' root).

In line 44 of the Pixelpost index.php, I added this line:

require("no-refer-spam.php");

I uploaded both files to my server and *poof*, no more referer spam!!!

I really hope this works for each of you. Please note, I didn't write this code, as you will find the author's website listed in the PHP code. There's more instructions listed on his website, but I covered the basics. You can add your own sites/keywords to block to the list, but this is also self-explanitory of you look at the code.

[doffer]

Wow :D That seems nice... But what it the idea behind it? Send them back? Could you explain it?

[er16004]

Wow :D That seems nice... But what it the idea behind it? Send them back? Could you explain it?

From my understanding, rather than let the spammer pass through to your website, this script matches the spammers URL with your spam-block list and refers the spammer back to themselves. So, they never really get to your site and your site won't log them. In turn, they are just using up their own bandwidth, not yours. Brilliant.

Note: I haven't had a spammer hit my website since I installed this. Yes, you do have to update it manually, but at least it WORKS!!

[blinking8s]

im testing it now...

[er16004]

Again, I really hope it works for you guys. As of 10pm CST (9 hours after my post), I have had *ZERO* referer spam.

Here's my updated spam-block list along with the code from above one more time:
see PHP code posted by me above and substitute the updated spam-list below (also in my post)

If this code works for everyone, maybe we can start a spam blacklist that everyone could use to keep their site up-to-date. Another option would be to use this code as an addon that would allow for adding to the list via an admin tool, much in the same manner Shifted Exposure's addon does.

[blinking8s]

the list is oh so short

;)

http://www.jayallen.org/comment_spam/blacklist.txt

we are in the processes of prepping 1.4.2 which has moved referrers to the admin and off the main blog, only admin can view...still get spam, but i'm seeing about 90% less of it...

[Wadem]

That list will be outdated in a few days as well. That style of system is only good as long as the file is updated, which is a manual process :(

1.4.2, now that sounds interest!

--WAdem

[er16004]

the list is oh so short

Well, the good thing about this PHP script is that it uses the list as keywords. In other words, you should only need to list "XXX" to remove all of the sites like xxx.somename.com, xxx.anothername.net, mysite.xxx.bix, etc. Again, this acts very similar to Shifted Exposures addon, so building on his original list and this one wouldn't be a bad idea, and hopefully wouldn't be too large.

UPDATED LIST: 2005-06-02
$spams = array (
"terashells.com", "chat-nett.com", "exitq.com", "cxa.de", "sysrem03.com",
"pharmacy.info", "guide.info", "drugstore.info", "coresat.com",
"psxtreme.com", "freakycheats.com", "cool-extreme.com",
"pervertedtaboo.com", "crescentarian.net", "holdem", "fuck-fest",
"yelucie.com", "-online", "findwebhostingnow.com", "smsportali.net",
"6q.org", "flowersdeliveredquick.com", "ronnieazza", "lemonrider",
"future-2000", "trackerom.com", "andrewsaluk.com", "4u.net", "4u.com",
"doobu.com", "nutzu", "italiancharms", "likejazz", "kloony",
"isacommie.com", "musicbox1.com", "tigerspice", "roody.com",
"bigsitecity", "zs1.biz", "spb.ru", "newru", "9k.com", "cialis",
"levitra", "viagra", "tramadol", "phentermine", "7h.com",
"hydrocodone", "propecia", ".com.cn", "izhuqiu", "sphosting.com",
"glwb.info", "phentemine", "findmore.org", "-sex", "paris-hilton",
"bizhat", "XXX", "macvillage.net", "ambien", "3d.net", "iqwide",
"sina.com", "vicodin", "jewelrycity", "adipex", "advicer", "ambien",
"bllogspot", "carisoprodol", "casino", "baccarrat", "cwas", "cyclen",
"cyclobenzaprine", "day-trading", "discreetordering", "dutyfree",
"duty-free", "fioricet", "freenet-shopping", "incest", "macinstruct",
"meridia", "-gambling", "paxil", "platinum-celebs", "poker-chip",
"poze", "prescription", "soma", "slot-machine", "taboo", "teen",
"trim-spa", "ultram", "xanax", "booker", "zolus", "chatroom", "poker",
"valium", "celebrex", "chinamoulds", "plasticmachinery", "offshore",
"czcn", "czpcsj", "freemovie", "xbuy"
);

[er16004]

I'm going to keep posting my spam-list update here.

Has anyone else used this code yet? Have you found it working well for you?

Should this post (or a smaller/shorter version) be made sticky?

[blinking8s]

Im testing this code and robs anti-ref-spam addon side by side at the moment

http://www.shiftedexposure.com/anti_spam.zip

the keywords do now work in robs script, they do, but it seems you have to clear out the tables manually from the addon rather than blocking them, also after adding this much new spam hit...

The script posted here seems to prevent it, but is hard to update for many users...

merge the two for a super anti-ref-spam addon?

[er16004]

merge the two for a super anti-ref-spam addon?

That's essentially what I've been getting at. I posted questions before (not really in this thread) about automating SExposures's addon. Before I was able to stop the spam, I had to keep clicking on his clean referer button every 10 minutes. His works great for removing the visitors links AFTER they visited. The code I posted is great for keeping offending spammers from accessing your site, but both rely on updating a spam-list. I tried to take the initial entries from SE's addon and put them in my spam-list.

It sure would be great to have them combined. My coding isn't 100%, but I'll do what I can to help.

[blinking8s]

I thought about some kind of master list addon, a user selects which are spam new, it updates a central DB here, their list can be updated daily via download or something...i donno, take a lot of effort, but that would be a good ultimate goal. Or a simpler solution would be a spam-link submition here, the db regenerates a file ever half hour or so...users can download it for their updates anytime...

Hopefully 1.4.2 will result in new users having much less spam to start off with...as my blog has never had referrer spam to this date, but that data has always been in my admin

the fight against spam could have its own development team. Sure wouldnt hurt to find a few more solid coders to help out...

[Wadem]

I'm glad you're wanting to take this to the same area I want to go. Have to update files and ftp them manually isn't exactly a spam solution :P

Over the weekend I might look @ adding both of hte current spam solutions listed here. It's a decent stop gap 1.4.2!

--djway

[blinking8s]

Over the weekend I might look @ adding both of the current spam solutions listed here. It's a decent stop gap 1.4.2!


they need to be compbined to work together, the script works more efficiently, the addon is more usabile ect...

you wont be able to avoid a download-->ftp route, its a pain...but its the root of the process until we can find a solution later down the road.

[Wadem]

Yeah I will be joining them together. If no one else gets to this before me, I'll publish the code I used if any1 wants?

--Wadem

[blinking8s]

in needs to be packaged as an addon using the script posted here, so that we can update the blocklist via the pixelpost admin. Pixelpost.org's demo recieves so much spam traffic that it is a great testing ground, sadly...

after testing (pending successful) we will start a master list for users to submit to and we will keep it a txt file to copy/paste from for updating the list.

[Wadem]

b8s, I don't understand what you mean when you say "using the script posted here". If I'm doing the work, I may as well do it in a way that benefits PP, not just myself.

--Wade

[Peloria]

Just tested the script - it seems to be working like a charm! Yay!

[blinking8s]

b8s, I don't understand what you mean when you say "using the script posted here". If I'm doing the work, I may as well do it in a way that benefits PP, not just myself.

--Wade

and I have no clue what you are talking about...haha

[blinking8s]

slight update of my list from pixelpost.org's ref spam

$spams = array (
".com.cn",
"3d.net",
"4u.com",
"4u.net",
"6q.org",
"7h.com",
"9k.com",
"adipex",
"advicer",
"ambien",
"ambien",
"andrewsaluk.com",
"baccarrat",
"bigsitecity",
"bizhat",
"bllogspot",
"booker",
"carisoprodol",
"casino",
"celebrex",
"chat-nett.com",
"chatroom",
"chinamoulds",
"cialis",
"cool-extreme.com",
"coresat.com",
"crescentarian.net",
"cwas",
"cyclen",
"cyclobenzaprine",
"czcn",
"czpcsj",
"day-trading",
"discreetordering",
"doobu.com",
"drugstore.info",
"dutyfree",
"duty-free",
"exitq.com","cxa.de",
"findmore.org",
"findwebhostingnow.com",
"fioricet",
"flowersdeliveredquick.com",
"freakycheats.com",
"freemovie",
"freenet-shopping",
"fuck-fest",
"future-2000",
"-gambling",
"glwb.info",
"guide.info",
"holdem",
"hydrocodone",
"incest",
"iqwide",
"isacommie.com",
"italiancharms",
"izhuqiu",
"jewelrycity",
"jvl.com",
"kloony",
"lemonrider",
"levitra",
"likejazz",
"macinstruct",
"macvillage.net",
"meridia",
"musicbox1.com",
"newru",
"nutzu",
"offshore",
"-online",
"paris-hilton",
"paxil",
"pervertedtaboo.com",
"pharmacy.info",
"phentemine",
"phentermine",
"plasticmachinery",
"platinum-celebs",
"poker",
"poker-chip",
"poze",
"prescription",
"propecia",
"psxtreme.com",
"ronnieazza",
"roody.com",
"-sex",
"sina.com",
"slot-machine",
"smsportali.net",
"soma",
"spb.ru",
"sphosting.com",
"sysrem03.com",
"taboo",
"teen",
"terashells.com",
"tigerspice",
"trackerom.com",
"tramadol",
"trim-spa",
"ultram",
"valium",
"viagra",
"vicodin",
"xanax",
"xbuy",
"XXX",
"yelucie.com",
"zolus",
"zs1.biz",
"wowgeil.com",
);

[Joe[y]]

a master list hosted at pixelpost.org and then grabbed by an addon and stuck into that script wadem posted would be ideal...

the master list could be updated by users - ideally through their own admin panel but we have to be realistic and assume that may just be too tricky and too much effort for what result we might get. perhaps just a link in the admin panel to take users to a page to add a spam link to the master list.

[er16004]

]perhaps just a link in the admin panel to take users to a page to add a spam link to the master list.

That would be ideal, but letting users update the master list automatically could present some problems. Say a user (or even a spammer) gets upset or makes a mistake and adds a URL that shouldn't on the list (e.g. Pixelpost.org), that would create problems. The list should at least have a moderator that monitors entries. If someone adds com, net, org, edu, etc. to that list, then it would shutout nearly everyone. Just some thoughts.

[blinking8s]

the list now crashes the current 1.4.2 realease we are working on. humph...

[nadams]

I have implemented the anti-spam addon combined with the no-referer-spam solution and it works pretty darn well. I usually get one or two new referer spams each day that require manually updating, which is inconvenient, but at least it cut it down from hundreds. Much appreciated! But I'm wondering if something is still tripping the counter -- it seems abnormally high for me to get 100-200 hits a day.

--Nick

[Beilby]

Another victim here. Same spammers, same result. (running 1.3)

I don't really see the point in having a publicly visible ref list anyway, so I am quite happy to implement the "refz" solution posted earlier as a long term solution (obviously made obsolete by 1.4.2).

I worry that such broad, keyword based ban/bounce lists are going to start getting a swag of unwelcome false positives.

[blinking8s]

Another victim here. Same spammers, same result. (running 1.3)

I don't really see the point in having a publicly visible ref list anyway, so I am quite happy to implement the "refz" solution posted earlier as a long term solution (obviously made obsolete by 1.4.2).

I worry that such broad, keyword based ban/bounce lists are going to start getting a swag of unwelcome false positives.

yes, its going to suck for those who have fallen victim already. Sadly there is nothing we can really do.

[Beilby]

Hey, but having this active community here is the best thing to deal with the problem.

[blinking8s]

we'll all do our best...and i assure you 1.4.2 will SLOW down the ref spam for those already victim. We just need to get the block script working.

I see the "keywords" causing issues down the road. I believe in future releases that refs will be totally removed though. With common stats engines available with hosting, it just isnt that big a deal anymore.

[shahram]

I have PP1.4 and/have done what Eric mentioned (based on http://planetOzh.com) to remove spammer. but there was no success. I also could not find the Rob's Antispammer. Is there anybody can help me with Spam filtering.
Shahram

[raminia]

dload it for temporary from:
www.raminia.com/temp/anti_spam.zip

[netwalker]

Connie made a modified version of index.php where the referers aren't counted.

The original thread it's here (http://www.pixelpost.org/forum/viewtopic.php?t=1354&postdays=0&postorder=asc&star t=0)

You can find the instructions to installing this modified version.

I'm still having a little bit of trouble with referer spam (they appear in statistics), because the page it's still appearing in search engines, but it's slowly going down.

Hope it helps.

[reallybites]

Connie made a modified version of index.php where the referers aren't counted.


Isn't that for version 1.3?

Anyways,the spam have been getting worst for me lately.Seems like there are hundrerds of visits every minute.I have a referer script for the whole domain,so i know.But the weird thing is,it dosent appear in my photoblog's referer page.

I have added the spam addon.

I changed the referer template file name yesterday but i think i'm still getting the spams.

Is there any other way?

[shahram]

Thanks Ramin. I used Robs and help alot for removing the spams and also using what Eric mentioned (based on http://planetOzh.com) to returne the spams to themselves was lot of helps.

[netwalker]

Connie made a modified version of index.php where the referers aren't counted.


Isn't that for version 1.3?

In the thread you can find one for 1.3, but I told Connie I was using 1.4 (the latest version) and she posted a new file for 1.4

[danimal]

I got rid of these referer spams.

1) I took the link to referers page on my photoblog.
2) I changed the code in index.php and replcaded the "ref" with "refz" and "referer" with "refererz".
3) I renamed my referer page of the template to refererz_template.html

now If i want to see my referers I use index.php?x=refz in the browser. Soon, in the next version, referers page will be inside admin panel not the photoblog main page. This way the evil spams do not follow it.

With "1)" are saying you removed the referrer link from the page altogether?

So are you saying that I can do a simple find & replace in, say, Dreamweaver and still get an accurate referer list with no spammers?

How does peppering my site with their addresses help them out? Are they hoping to make their Google page rank go up in the assumption that I am actually going to click on the address? I assume that the link on my referrer page indicates that "someone" got to my page through a link on some crazy t&a site. Yeah, I'm confused. :shock:

Anyone know how I can start from scratch and remove all the links from the referrer page after I implement ramina's fix?

[blinking8s]

you can always enter the database and sort through them manually

[Wadem]

For those just reading here, I'll sum up the current solution for (4.0.1).

(Note, I've also updated the no-refer-spam list for old users)

1)Install Rob's anti-spam addon from http://www.shiftedexposure.com/anti_spam.zip.

2)Install this by place the unzipped file in your addons directory

3) Per er16004

Copy the code below, make a new file called no-refer-spam.php and paste this code into that file.

<?php
/*
Script Name: No Refer Spam
Version: 1.00
Hack URI: http://frenchfragfactory.net/ozh/archives/2005/02/05/no-refer-spam/
Description: Send refer spammers back to their own sites
Author: Ozh
Author URI: http://planetOzh.com
*/

$spams = array (
"-buy",
"-home",
".com.cn",
"3d.net",
"4u.com",
"4u.net",
"6q.org",
"7h.com",
"9k.com",
"adipex",
"advicer",
"alleghanyeda",
"ambien",
"ambien",
"andrewsaluk.com",
"baccarrat",
"bigsitecity",
"bizhat",
"bllogspot",
"booker",
"broadphase",
"buy-",
"carisoprodol",
"casino",
"celebrex",
"chat-nett.com",
"chatroom",
"chinamoulds",
"cialis",
"cool-extreme.com",
"coolscott",
"coresat.com",
"crescentarian.net",
"cwas",
"cyclen",
"cyclobenzaprine",
"czcn",
"czpcsj",
"day-trading",
"diet",
"discreetordering",
"doobu.com",
"drugs",
"dutyfree",
"duty-free",
"exitq.com","cxa.de",
"findmore.org",
"findwebhostingnow.com",
"fioricet",
"flowersdeliveredquick.com",
"freakycheats.com",
"freemovie",
"freenet-shopping",
"fuck-fest",
"future-2000",
"-gambling",
"glwb.info",
"guide.info",
"holdem",
"home-",
"host-b",
"hrenax",
"hydrocodone",
"incest",
"iqwide",
"isacommie.com",
"italiancharms",
"izhuqiu",
"jewelrycity",
"jvl.com",
"kloony",
"lemonrider",
"levitra",
"likejazz",
"macinstruct",
"macvillage.net",
"meridia",
"mortgage",
"musicbox1.com",
"neotwin",
"newru",
"nutzu",
"offshore",
"online",
"paris-hilton",
"paxil",
"pervertedtaboo.com",
"pharmacy",
"phentemine",
"phentermine",
"plasticmachinery",
"platinum-celebs",
"poker",
"poker-chip",
"poze",
"prescription",
"propecia",
"psxtreme.com",
"ringtone",
"ronnieazza",
"roody.com",
"-sex",
"sina.com",
"slot-machine",
"smsportali.net",
"soma",
"spb.ru",
"sphosting.com",
"sysrem03.com",
"taboo",
"teen",
"terashells.com",
"tigerspice",
"trackerom.com",
"tramadol",
"trim-spa",
"ultram",
"valium",
"viagra",
"vicodin",
"wowgeil",
"xanax",
"xbuy",
"XXX",
"yelucie.com",
"zakona",
"zolus",
"zs1.biz",
);


$ref = $_SERVER["HTTP_REFERER"];

if ($ref) {
foreach ($spams as $site) {
$pattern = "/$site/i";
if (preg_match ($pattern, $ref)) {
header("Location: $ref"); exit();
}
}
}
?>

I saved this file into my Pixelpost root directory (which is also my severs' root).

In line 44 of the Pixelpost index.php, I added this line:

require("no-refer-spam.php");

I uploaded both files to my server and *poof*, no more referer spam!!!

I really hope this works for each of you. Please note, I didn't write this code, as you will find the author's website listed in the PHP code. There's more instructions listed on his website, but I covered the basics. You can add your own sites/keywords to block to the list, but this is also self-explanitory of you look at the code.

4) updated the anti-spam plugin with the following referer lists. To update the list, login to your PP backend. In here click on Addon, you will see Banned Words

-buy
-home
.com.cn
3d.net
4u.com
4u.net
6q.org
7h.com
9k.com
adipex
advicer
alleghanyeda
ambien
ambien
andrewsaluk.com
baccarrat
bigsitecity
bizhat
bllogspot
booker
broadphase
buy-
carisoprodol
casino
celebrex
chat-nett.com
chatroom
chinamoulds
cialis
cool-extreme.com
coolscott
coresat.com
crescentarian.net
cwas
cyclen
cyclobenzaprine
czcn
czpcsj
day-trading
diet
discreetordering
doobu.com
drugs
dutyfree
duty-free
exitq.comcxa.de
findmore.org
findwebhostingnow.com
fioricet
flowersdeliveredquick.com
freakycheats.com
freemovie
freenet-shopping
fuck-fest
future-2000
-gambling
glwb.info
guide.info
holdem
home-
host-b
hrenax
hydrocodone
incest
iqwide
isacommie.com
italiancharms
izhuqiu
jewelrycity
jvl.com
kloony
lemonrider
levitra
likejazz
macinstruct
macvillage.net
meridia
mortgage
musicbox1.com
neotwin
newru
nutzu
offshore
online
paris-hilton
paxil
pervertedtaboo.com
pharmacy
phentemine
phentermine
plasticmachinery
platinum-celebs
poker
poker-chip
poze
prescription
propecia
psxtreme.com
ringtone
ronnieazza
roody.com
-sex
sina.com
slot-machine
smsportali.net
soma
spb.ru
sphosting.com
sysrem03.com
taboo
teen
terashells.com
tigerspice
trackerom.com
tramadol
trim-spa
ultram
valium
viagra
vicodin
wowgeil
xanax
xbuy
XXX
yelucie.com
zakona
zolus
zs1.biz


Updated Spammers list from my blog, as of 10/Jul/05 for no-refer-spam

$spams = array (
"-buy",
"-home",
".com.cn",
"3d.net",
"4u.com",
"4u.net",
"6q.org",
"7h.com",
"9k.com",
"adipex",
"advicer",
"alleghanyeda",
"ambien",
"ambien",
"andrewsaluk.com",
"baccarrat",
"bigsitecity",
"bizhat",
"bllogspot",
"booker",
"broadphase",
"buy-",
"carisoprodol",
"casino",
"celebrex",
"chat-nett.com",
"chatroom",
"chinamoulds",
"cialis",
"cool-extreme.com",
"coolscott",
"coresat.com",
"crescentarian.net",
"cwas",
"cyclen",
"cyclobenzaprine",
"czcn",
"czpcsj",
"day-trading",
"diet",
"discreetordering",
"doobu.com",
"drugs",
"dutyfree",
"duty-free",
"exitq.com","cxa.de",
"findmore.org",
"findwebhostingnow.com",
"fioricet",
"flowersdeliveredquick.com",
"freakycheats.com",
"freemovie",
"freenet-shopping",
"fuck-fest",
"future-2000",
"-gambling",
"glwb.info",
"guide.info",
"holdem",
"home-",
"host-b",
"hrenax",
"hydrocodone",
"incest",
"iqwide",
"isacommie.com",
"italiancharms",
"izhuqiu",
"jewelrycity",
"jvl.com",
"kloony",
"lemonrider",
"levitra",
"likejazz",
"macinstruct",
"macvillage.net",
"meridia",
"mortgage",
"musicbox1.com",
"neotwin",
"newru",
"nutzu",
"offshore",
"online",
"paris-hilton",
"paxil",
"pervertedtaboo.com",
"pharmacy",
"phentemine",
"phentermine",
"plasticmachinery",
"platinum-celebs",
"poker",
"poker-chip",
"poze",
"prescription",
"propecia",
"psxtreme.com",
"ringtone",
"ronnieazza",
"roody.com",
"-sex",
"sina.com",
"slot-machine",
"smsportali.net",
"soma",
"spb.ru",
"sphosting.com",
"sysrem03.com",
"taboo",
"teen",
"terashells.com",
"tigerspice",
"trackerom.com",
"tramadol",
"trim-spa",
"ultram",
"valium",
"viagra",
"vicodin",
"wowgeil",
"xanax",
"xbuy",
"XXX",
"yelucie.com",
"zakona",
"zolus",
"zs1.biz",
);

I hope this is of some help for someone. The above list removed over 40,000 spam entries from my site.

Thanks,
--Wadem

[riken]

Why maintain two lists of spammers when you can use one twice.

Change no-refer-spam.php to

<?php
/*
Script Name: No Refer Spam
Version: 1.01
Hack URI: http://www.pixelpost.org/forum/viewtopic.php?t=1252
Description: Send refer spammers back to their own sites
Author: Ozh
Author URI: http://planetOzh.com
Modified from original by: Riken
*/

$query = "SELECT banlist FROM {$pixelpost_db_prefix}ban LIMIT 1";
$result = mysql_query($query) or die( mysql_error() );
if( $row = mysql_fetch_row($result) ) {
$banlist = $row[0];
$banlistarray = explode( "\n", $banlist );
} else {
$banlist = '';
$banlistarray = array();
}

$ref = $_SERVER["HTTP_REFERER"];

if ($ref) {
foreach ($banlistarray as $site) {
$pattern = "/$site/i";
if (preg_match ($pattern, $ref)) {
header("Location: $ref"); exit();
}
}
}
?>
and move the require("no-refer-spam.php"); to between start_mysql and book_visitor like so

start_mysql();
book_visitor($pixelpost_db_prefix."visitors");

// becomes

start_mysql();
require("no-refer-spam.php");
book_visitor($pixelpost_db_prefix."visitors");
Now it uses the same list the anti-spam addon uses. You can maintain your list of spammers in the addon page.

[Wadem]

riken: I had that same thought in my sleep last night after writing my post here. lol.

--Wade

[riken]

riken: I had that same thought in my sleep last night after writing my post here. lol.
8)
Great minds think alike.

or

Fools seldom differ. :)

[Box Brownie]

Any update on when the amendments will be made that allows more control of this pest problem?

Oops! I see now that 1.4.2 will be the next release and that will be addressing this problem.

:)

[Wadem]

Box: The current solution that was just listed by riken, is pretty easy.

Update the files as he's listed once, then you can add new domains (as need be) via the admin backend.

I'm updating the list again now too...

--Wadem

[Wadem]

Updated Referer Spammers as @ 15/Jul/05

Please note, I've cut out all .cn domains.

--Wade


-buy
-home
.cn
3d.net
4u
6q.org
7h.com
9k.com
adipex
advicer
alleghanyeda
amateurvoetbal
ambien
andrewsaluk.com
atspace
baccarrat
bigsitecity
bizhat
bllogspot
booker
broadphase
buy-
carisoprodol
casino
celebrex
chat-nett.com
chatroom
chinamoulds
cialis
cool-extreme.com
coolscott
coresat.com
crescentarian.net
cwas
cyclen
cyclobenzaprine
czcn
czpcsj
day-trading
diet
discreetordering
doobu.com
drugs
dutyfree
duty-free
exitq.comcxa.de
exproweb
findmore.org
findwebhostingnow.com
fioricet
flowersdeliveredquick.com
freakycheats.com
freemovie
freenet-shopping
fuck-fest
future-2000
-gambling
glwb.info
guide.info
holdem
home-
host-b
hrenax
hydrocodone
incest
iqwide
isacommie.com
italiancharms
izhuqiu
jewelrycity
jvl.com
kloony
lemonrider
levitra
likejazz
macinstruct
macvillage.net
meridia
mortgage
musicbox1.com
neotwin
newru
nutzu
offshore
orgfree.com
online
paris-hilton
paxil
pervertedtaboo.com
pharmacy
phentemine
phentermine
plasticmachinery
platinum-celebs
polarhome
poker
poker-chip
poze
prescription
propecia
protandin.supportwest.com
psxtreme.com
ringtone
ronnieazza
roody.com
-sex
sina.com
search500.biz
slot-machine
smsportali.net
soma
spb.ru
sphosting.com
sysrem03.com
taboo
teen
teeeeeen
terashells.com
tigerspice
trackerom.com
tramadol
trim-spa
trondalh.com
ultram
valium
viagra
vicodin
wowgeil
xadulthosting
xanax
xbuy
XXX
yelucie.com
zakona
zapto
zolus
zs1.biz

[dawn]

Just for the record, Wadem, my site is legit (when it's up and running) and it uses a "-". http://www.girl-inchoate.com :-)

[Wadem]

Damn you Dawn! And I mean that in a VERY good way ;)

I've taken - out of the list above. That adds a lot more work to the list :(

Thanks for chirping up tho, that's why I posted what I did.

--Wade

[dawn]

Thank you for updating the list, though. I just added it to my other pixelpost site and cleaned out 19,000 referrers.

[Wadem]

No problems Dawn. I'll have to work out another method to exclude the dodgy - sites.

It's going to take another few days before my site gets enough spam again to update the list. I'll post it when I next update mine.

I find it easier using a non-spam referer list of PP's then using my host's referer list, as it contains all the spammers.

--Wade

[Wadem]

Upto Referers list dated as @ 19/Jul/05

I also did a google referer search on my domain, and my god these guys have gotten a free ride off me!!! I've removed them ASAP so they don't get anything.

--Wade

-buy
-home
.cn
3d.net
4u
50webs.com
6q.org
7h.com
8m.com
9k.com
adipex
adult
advicer
alleghanyeda
amateurvoetbal
ambien
amiben
andrewsaluk.com
atspace
baccarrat
bestfreedirectory.com
bigsitecity
bizhat
bllogspot
booker
boxmail.biz
bravehost.com
broadphase
buy-
carisoprodol
casino
celebrex
chat-nett.com
chatroom
cheap
chinamoulds
cialis
cool-extreme.com
coolscott
coresat.com
credit
crescentarian.net
cwas
cyclen
cyclobenzaprine
czcn
czpcsj
dating-s.net
day-trading
diet
discreetordering
doobu.com
drugs
dutyfree
duty-free
exitq.comcxa.de
exproweb
findhere.org
findmore.org
findwebhostingnow.com
fioricet
flowersdeliveredquick.com
freakycheats.com
free
fuck-fest
funpic.org
future-2000
-gambling
glwb.info
guide.info
holdem
home-
host-b
host-c
hrenax
hydrocodone
incest
iqwide
isacommie.com
italiancharms
itrama.notlong.com
izhuqiu
jewelrycity
jvl.com
kloony
lemonrider
levitra
likejazz
macinstruct
macvillage.net
meridia
mortgage
mtsby.com
musicbox1.com
neotwin
newru
nutzu
offshore
oilmed
orgfree.com
online
parbox.com
paris-hilton
paxil
penis
pervertedtaboo.com
pharmacy
phent
phentemine
phentermine
plasticmachinery
platinum-celebs
polarhome
poker
poker-chip
poze
prescription
promisance.biz
propecia
protandin.supportwest.com
psxtreme.com
q.fm
ringtone
ronnieazza
roody.com
-sex
sina.com
search500.biz
slot-machine
smsportali.net
soma
spb.ru
sphosting.com
sysrem03.com
taboo
teen
teeeeeen
terashells.com
ticketsbot.com
tigerspice
trackerom.com
tramadol
trim-spa
trondalh.com
twinlook.com
ultram
valium
viagra
vicodin
wowgeil
xadulthosting
xanax
xbuy
XXX
yelucie.com
zakona
zapto
zolus
zs1.biz

[Wadem]

Up to date as @ 30/07/05
-buy
-home
.cn
100webcustomers.com
3d.net
4u
50webs.com
6q.org
7h.com
8m.com
9k.com
adipex
adult
adygeya.su
advicer
alleghanyeda
amateurvoetbal
ambien
amiben
andrewsaluk.com
atspace
axionfootwear.com
baccarrat
bestfreedirectory.com
bigsitecity
bizhat
bllogspot
booker
boxmail.biz
bravehost.com
broadphase
buy-
candiria.com
carisoprodol
casino
celebrex
chat-nett.com
chatroom
cheap
chinamoulds
cialis
conecrusher.org
cool-extreme.com
coolscott
coresat.com
credit
crescentarian.net
cwas
cyclen
cyclobenzaprine
czcn
czpcsj
dating-s.net
day-trading
diet
discreetordering
dnip.net
doobu.com
drug
dutyfree
duty-free
edthompson.com
exitq.comcxa.de
exproweb
findhere.org
findmore.org
findwebhostingnow.com
fioricet
flowersdeliveredquick.com
freakycheats.com
free
fuck-fest
funpic.org
future-2000
-gambling
glwb.info
guide.info
hardlink.info
holdem
home-
host-
hrenax
huytebe
hydrocodone
incest
iqwide
isacommie.com
italiancharms
itrama.notlong.com
izhuqiu
jewelrycity
jvl.com
kapsi.fi
kloony
lemonrider
levitra
likejazz
macinstruct
macvillage.net
meridia
mortgage
mtsby.com
musicbox1.com
neotwin
newru
nutzu
offshore
oilmed
orgfree.com
online
ownsthis.com
parbox.com
paris-hilton
paxil
penis
pervertedtaboo.com
pharmacy
phent
phentemine
phentermine
pizdeckakojto
plasticmachinery
platinum-celebs
polarhome
poker
poker-chip
poze
prescription
promisance.biz
propecia
protandin.supportwest.com
protzonbeer.com
psxtreme.com
pulsar.net
q.fm
ringtone
ronnieazza
roody.com
sbj-broadcasting.com
sex
sina.com
search500.biz
slot-machine
smsportali.net
soma
spb.ru
sphosting.com
sysrem03.com
taboo
teen
teeeeeen
terashells.com
ticketsbot.com
tigerspice
trackerom.com
tramadol
trim-spa
trondalh.com
twinlook.com
ultram
valium
viagra
vicodin
wowgeil
xadulthosting
xanax
xbuy
XXX
yelucie.com
zakona
zapto
zolus
zs1.biz
zu5.net

Cheers,
--Wade

[steff]

I also did a google referer search on my domain, and my god these guys have gotten a free ride off me!!! I've removed them ASAP so they don't get anything.


how do you perform a google referer search? :)

[Wadem]

I have NO idea anymore steff!!!! I think I just searched for my site name TBH and as you can see from here http://www.trendmapper.com/charts/pages/-EyeofWade-_id31032005-23.html Google picked up all my spam one day. Then I removed all the spam and it was gone. I don't think I was specifically doing a referer search.

sorry,
--Wade

[Wadem]

Also, for those who aren't reading the other thread on 1.4.2...

Please see Here (http://forum.pixelpost.org/showpost.php?p=12103&postcount=66) for the updated referer's list.

In short I'm hosting the update file on my server for you all to download whenever. And I will update it @ least once a week, more or less depeding on how much spam i see and/or people PM me URL's/IP's for.

Cheers,
--Wadem