You must be logged in to post Login


Lost Your Password?

Search Forums:


 






Minimum search word length is 4 characters – Maximum search word length is 84 characters
Wildcard Usage:
*  matches any number of characters    %  matches exactly one character

Blocking Bad Bots

No Tags
UserPost

7:56 am
May 3, 2013


Eric – PersonalProfitability.com

Portland, OR

Member

posts 2120

Hey all, I just had an interesting experience that required an interesting solution that I want to share with you.

A long time ago, I worked somewhere that blocked many websites from the office. To get around it, I used PHProxy installed on my server (a sub-directory on my personal blog). Just this week, it appears that Yandex, a major Russian search engine, began to scan that sub-directory.

Being a PHP tool, I started getting all sorts of warnings from Wordpress Firewall, and it appeared that Yandex was scanning my site constantly including that directory, which has since been deleted.

It turns out Yandex has bots that are quite relentless and use a lot of bandwidth. A little searching showed that I could add this to my .htaccess file and block them:

SetEnvIfNoCase User-Agent "^Yandex*" bad_bot
Order Deny,Allow
Deny from env=bad_bot
Anywho, if you see tons of bandwidth and resources being stolen by bots you don't really care about, you can use code like this to block them.

1:27 pm
May 3, 2013


Glen Craig

Member

posts 1087

Thanks for passing this along.  I'll have to look at my server and see what my usage is.

No Tags

About the Yakezie.com Forum

Forum Timezone: America/Los_Angeles

Forum Stats:

Groups: 2
Forums: 9
Topics: 6383
Posts: 84794

Membership:

There are 13651 Members
There have been 20 Guests

There are 9 Admins
There are 8 Moderators

Top Posters:

My Personal Finance Journey – 3159
Khaleef @ KNS Financial – 3149
Budgeting in the Fun Stuff – 3048
Sustainable PF – 2759
Miss T @ Prairie Eco-Thrifter – 2213
Eric – PersonalProfitability.com – 2120

Administrators: The College Investor (1935 Posts), Financial Samurai (1803 Posts), LaTisha @YoungFinances (1715 Posts), Forest Parks (1337 Posts), 20s Finances (1147 Posts), Money Reasons (697 Posts), Chris Johnson (78 Posts), Sydney at Untemplater (0 Posts), Suba (0 Posts)

Moderators: Suba @ Wealth Informatics (1876 Posts), sooverthis (1041 Posts), PK @ DQYDJ (361 Posts), jmichelsen (208 Posts), Ramona (13 Posts), JeremyNJohnson (4 Posts), Moderator (0 Posts), rackgeek (0 Posts)