You might want to block this fucker

dr rockerdr rocker Regular
edited September 2011 in Help and Suggestions
As there were 209 users online, I thought I would have a look at 'users online'. Baidu Spider (a chinese indexing bot) appeared numerouse times. Now, bots indexing things is no bad things, however, being that:

1. Totse is probably banned in China
2. Its probably ripping images and content that will bring in low to 0 traffic - being as totse is probably banned in PRC, why let it index shit, let alone content rob (which it is known to do)?
3. Its probably responsible for eating bandwidth - a shit load of bandwidth. Some web site operators have said in some months is has eaten all BW allowence very quickly

Now, that is three probably's, but I am probably correct.

On top of that, letting it do such shit is helping China to dominate and so potential subvert the internets.


Or modify robots.txt or block its IP ranges.


  • edited September 2011
    That's funny, I was just looking at our online users as well and I noticed that Baidu Spider crawling around. Maybe we should look into restricting it or something - have you got any links to other places which are commenting on this particular spider?
  • dr rockerdr rocker Regular
    edited September 2011
    Just google Baidu Spider - lots of shit on how to block it comes up and descriptions of what it does.
  • edited September 2011
    I think we should add something into robots.txt - it's a good idea. Having a look through Google results is proving to be very interesting and it's obvious that Baidu is a problem to a lot of website administrators.
Sign In or Register to comment.