- MDDHosting Forums
- → Mike W's Content
Mike W's Content
There have been 8 items by Mike W (Search limited from 23-January 20)
Why would you not recommend blocking more than 50 - 100 ip's? I've read that when people ban bots, they sometimes have to ban a whole range of ip's.
Banning this bot would justify spending good money at odesk or something. It literally cut my Adsense earnings in half. And I'm willing to spend some money to get it to disappear for a period of time.
I'm willing to spend a $1,000 or more to get rid of this bot. Do you think this would be enough $$$ to hire an American from odesk to ban the bot by name? I have 4 websites.
Wow, what a nightmare...But I try to take it in stride
Don't forget Amazon has a huge cloud hosting service so it's not actually Amazon generating that clicks it's something someone is running on one of their servers.
If I were you I wouldn't worry much about it. Adsense will automatically filter out 'bad' clicks so long as they aren't coming from you IP address Google isn't likely to penalize you.
Block the IP range via htaccess (or cPanel's deny access feature) to prevent that bot from sucking down your bandwidth.
It's not just about not getting banned, my $1 to $4 clicks have vanished since this started. I've been smartpriced and it's affecting my whole account.
How can I ban these bots that show up in Awestats? Here's an example of one: Unknown robot (identified by 'spider') Hits 641+58
I don't know how to find the ip range for this bot. Can I ban it by what it identifies itself as ('spider')? And what does the "+" mean under 'hits'? Either way, it appears as if it has visited my site more than 600 times this month. The bot that is causing the invalid clicks is surely visiting my sites quite a bit. Maybe that would be a good place for me to start.
I think the stats programs list bots that "self identify", so they won't list ones that don't identify themselves.
You can limit bots using a robots.txt file in your site root, but ill behaved bots will simply ignore robots.txt. One way to catch them is to create a "bad bots trap" ... there are several different variations out there, but most create a folder or link specifically denied to all bots in robots.txt and then track and see which bots ignore your directives.
The problem with banning IP addresses of bots is that they are often shared IPs, and you are also banning any legitimate users with that IP address. You can get lists of bad bots by Googling "bad bots list", but they list from 120 to 4,500 IP addresses.
Thanks for the response fshagan, hopefully I can work this out somehow. It's wreaking havoc on my Adsense earnings.
I wonder why the bot only clicks "animated image" ads. And why it keeps coming back. Even though I've switched to text only ads, 7 or so "animated image" ads still are displayed everyday. Apparently, this bot still finds them and clicks them.
This makes it appear as if it's crawling my sites constantly. Doesn't it have something else to do?
About a week ago, I noticed a huge increase in the CTR for my Adsense ads. After looking into it, I found out that only one type of add was getting increased clicks. It was the "animated image" ad type. It went from a CTR of under 5% to 35%+. The CTR on my other ad types are all normal. Since I changed the ads back to "text" only, everything has gone back to normal (though I now seem to be smart priced)
I'm almost sure this is some misguided bot. If it were malicious, they would be clicking more than just "animated image" ads (which are by far in the minority)
I've been monitoring with Statcounter, but it doesn't appear to even show all bot activity. In Awestats, it shows a list of 10 or so bots. I want to ban all the ones that aren't Google, Yahoo etc...But I don't see the ip for these bots listed anywhere. I looked through some of the other options in cpanel and didn't see anything useful except for the raw access logs. But it takes a huge amount of time to go through these, and I'm not even sure what I'm looking for.
Also, does Awestats show all bot activity. I located a Sitelock bot in my raw access logs and banned it. But this bot was not listed in Awestats.
Sorry this is so long, I'm cpanel noob. I tried getting in contact with Adsense through their invalid click activity link. No luck so far.
Yes, I did report it to Adsense twice through their invalid click activity link. They never responded. I posted the same info as above.
I asked for the ip of the bot that was clicking the ad.
Adsense is discounting the clicks on that ad type. Clicks on that ad type range between 5 cents and 0. My average CPC is around 50 cents.
But this seems to be hurting my CPC as a whole. My average CPC used to be over 80 cents.
And as I said above, I switched all my ad units over to "text only," but for some reason, the "animated image" ad still appears. For about every 1000 impressions, the offending ad unit will appear about 5 times. And it gets clicked a lot. How could this be? I don't know much about bots, but I don't understand how this could be possible. If a bot was constantly visiting my site, wouldn't my page impression stats be super inflated?
Here's an idea of some stats from individual ad units.
Text only ad CTR-about 2% CPC- about 50 cents
Rich media ad CTR-about 2% CPC- almost $2
Image ad CTR-under 2% CPC- over $1
Flash ad CTR-about 2% CPC- over $1
Animated Image ad CTR-about 15% CPC-5 cents The stats for this ad unit used to be about the same as the flash ad unit, I've been using Adsense for almost a year.
Weird, maybe Adsense is giving me bogus stats.
I'll try to contact them again.
Thanks for the responses.
In this thread, a bot was clicking flash ads only. Here's another one involving the Amazon bot. But I haven't seen any Amazon bots.
Hmmm...just had a thought. @fshagan I mentioned in the OP that I banned a Sitelock bot, but I just banned the ip I found for it. You said these bots used rotating ips. So it could be coming back.
Sitelock is an upsell that my old host (iPage) uses. Apparently, it's still crawling around my sites looking for 'vulnerabilities.' Why does it not appear in Awestats under spiders, but it shows up in my raw log files? It clearly said 'Sitelock Bot'
I've read you can ban a bot by it's name or where it originates from. Can I do this with info from my raw access logs?
I know I'm kinds clueless and am asking a lot of vague questions. I will study up on this immediately. But wow! Where to start...
What would you do first if you were me?
You can ban bots that don't adhere to the robots.txt file by having a "secret" folder and file that is disallowed in robots.txt and not visible to normal web surfers (no links to it, etc.) Typically people put a no-index, no-follow header in it so it doesn't show up in search engines.
You then scan your logs for visits to that page, and ban the IPs that find it. Some programs have been written that provide the list of IPs ... see http://www.kloth.net...net/bottrap.php and http://danielwebb.us...tware/bot-trap/
You could hire someone to implement that kind of thing for you.
Thanks for the tip. Much appreciated. I'm waiting for a reply from a few people right now. Wish me luck.