Chinese search engine

**ippacegail** · 06-04-2009, 10:45 AM

I have a pernicious chinese search engine bot that has crawled my whole website 90+ times in the last 3 days. Being as it has banned my site from China (religious content) is there a good reason to let it continue to crawl, or is it just as good to ban it. It is skuing my numbers something fearce.

**Ron** · 06-04-2009, 10:55 AM

I don't know how much traffic you're getting from China (presumably close to 0) but are you getting any traffic from that search engine? Is it obeying robots.txt? If so, does it obey any particular crawl limiting parameter that won't affect Google?

Personally, when I find a search engine or bot crawling around on my site that I don't want on my site, I will just stick its IP addy into .htaccess and block them thataway.

Other thoughts:
Other than skewing your numbers, is it causing a problem?
You could put in a java-based tracker like the one from sitemeter.com whose numbers generally won't be skewed by most search engines.

Tough decisions sometimes.

**ippacegail** · 06-04-2009, 11:02 AM

We won't be view in China anytime soon.... and I don't see it messing anything up... comes in as an unknown robot, but with numerous IPs. Just aggrevating to see a big jump in bandwidth and it just be them for the 15th time that day.

**Ron** · 06-04-2009, 11:17 AM

I say if they are stressing you out, ban'em!

**homoludens** · 06-04-2009, 01:09 PM

I believe Vinnie just bans China. The amount of people with chinese IP addresses I get trying to relay through my mail server, I'm inclined to agree with him.

Originally Posted by Ron

I say if they are stressing you out, ban'em!

I couldn't agree more.

**Frank Broughton** · 06-06-2009, 08:05 AM

China yes and Russia too....

**VinDSL** · 06-07-2009, 02:29 AM

Originally Posted by ippacegail

I have a pernicious chinese search engine bot that has crawled my whole website 90+ times in the last 3 days. Being as it has banned my site from China (religious content) is there a good reason to let it continue to crawl, or is it just as good to ban it. It is skuing my numbers something fearce.

News flash!

China is THE new superpower.

Resistance is futile...

Yes, I've banned the entire country, but they still sneak through.

Iran is the #1 visiting country on my web site - still haven't figured that out.

**ippacegail** · 06-10-2009, 06:13 PM

Haven't had any hits from Iran. But good to know. I blocked China, and it is giving me almost a perverse pleasure to look at my error logs now. lol.....

Also getting hits (not visits) from Pakistan, India, and the Cocos (keeling) Islands.