Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

Bot traffic / Site scraping. Anyone have any good solutions? 
 
jasonf_maxtool
Jr. Member
 
Total Posts:  2
Joined:  2013-02-11
 

Some of our stores get hammered with hundreds of searches an hour for skus that we dont even carry. This directly impacts our traffic.

Anyone have any good tips on how to stop this kind of traffic?

Thanks in advance!

 
Magento Community Magento Community
Magento Community
Magento Community
 
serpyre
Enthusiast
 
Avatar
Total Posts:  771
Joined:  2013-05-20
 

That is the cost of doing business, you can restrict at the web server level but probably some badly behaved bots. Are these quick searches or page requests, very different scenarios.

 
Magento Community Magento Community
Magento Community
Magento Community
 
MagenX
Enthusiast
 
Total Posts:  791
Joined:  2008-05-26
Dublin
 

you have few options, rewrite them using a user agent string, or filtering your traffic by limiting requests from the same source IP / Host

google for : “blocking bad bots”

cheers

 
Magento Community Magento Community
Magento Community
Magento Community
 
gapsupport
Jr. Member
 
Total Posts:  4
Joined:  2013-11-27
 

What form of ddos protection does magento offer?
This type of attack is NOT the price of doing buisness. There should be availble services listed fir security and at the very least Cloudflare.

Are you hosted with a private server or directly with Magento.

 
Magento Community Magento Community
Magento Community
Magento Community
 
serpyre
Enthusiast
 
Avatar
Total Posts:  771
Joined:  2013-05-20
 

Of coure it is which is why companies spend a lot of money on solutions, CE/EE are self-hosted so if you are targeted then you will need a CloudFlare or equivalent (which is what we use). It is at the hosting level, unless you have specific solutions implemented you are always at risk, that is the cost.

 
Magento Community Magento Community
Magento Community
Magento Community
 
gapsupport
Jr. Member
 
Total Posts:  4
Joined:  2013-11-27
 

ok, look I am new to the Magento product but have 10 years of Ecommerce expierence.  I am looking at setting a Magento cart so I am in the Forum trying to understand the best way to protect a Magento cart from:

Scrappers, DDOS attacks and hackers.

#1 I do not know what CE/EE or that it can be self-hosted
#2 CloudFlare protects against a range of threats: cross site scripting, SQL injection, comment spam, excessive bot crawling, email harvesters, and more. BUT at a significant monthly price!

If you are already paying for hosting or a server, adding the cheapest Cloudflare protection will add at least another $200/mo in cost and that does not give you any real support for the product except by email and the reports are not detailed enough to really get a handle on future protection you could implement.

Is there a hosting provider out there that host Magento AND provides Cloudflare and/or other types of protection that is affordable?

 
Magento Community Magento Community
Magento Community
Magento Community
 
gapsupport
Jr. Member
 
Total Posts:  4
Joined:  2013-11-27
 

ok, look I am new to the Magento product but have 10 years of Ecommerce expierence.  I am looking at setting a Magento cart so I am in the Forum trying to understand the best way to protect a Magento cart from:  Scrapers, DDOS attacks and hackers.

#1 I do not know what CE/EE is or that it can be self-hosted

#2 I know CloudFlare protects against a range of threats: cross site scripting, SQL injection, comment spam, excessive bot crawling, email harvesters, and more. BUT at a significant monthly price of at least $200!

If you are already paying for hosting or a server, adding the cheapest Cloudflare protection will add at least another $200/mo in cost and that does not give you any real support for the product except by email and the reports are not detailed enough to really get a handle on future protection you could implement.  Then what is the best combination solution to allow one to do business online?

Is there a hosting provider out there that host Magento AND provides IP blocking, Cloudflare and/or other types of protection that is affordable?

 
Magento Community Magento Community
Magento Community
Magento Community
 
serpyre
Enthusiast
 
Avatar
Total Posts:  771
Joined:  2013-05-20
 

CE is the open source version, EE is enterprise version at $15,000/yr and for companies with revenue of $2-10million per year, both are self-hosted. Magento Go is the online version of Magento to compete against Big Commerce/Volusion so for small stores.

DDOS mitigation is an enterprise level issue, you will not find many solutions which are affordable if any, we have heard hosts will just shut you down if you are targeted. We know of one company who has hosting and DDOS but that\’s about it.

 
Magento Community Magento Community
Magento Community
Magento Community
 
gapsupport
Jr. Member
 
Total Posts:  4
Joined:  2013-11-27
 

I know first hand with a DDOS level 7 attack that it is not an Enterprise issue and that an unethical competitor can pay someone $25/hr to launch an attack on your site. 

My concern is that I can not find any information for the Magento Go hosting regarding offering DDOS or even kiddie scripter, hacker protection. 

We can not afford an Enterprise level product but need Enterprise level protection in order to keep our site up and grow our business.

I am open to suggestions.

 
Magento Community Magento Community
Magento Community
Magento Community
 
serpyre
Enthusiast
 
Avatar
Total Posts:  771
Joined:  2013-05-20
 

We can not afford an Enterprise level product but need Enterprise level protection in order to keep our site up and grow our business.

Welcome to the new world of eCommerce, you will either have to do it manually or pay for the software, there are no shortcuts. A DDOS can be launched on any site, however the automated anti-DDOS tools are not cheap as they are classed as enterprise. For Go you would have to put CloudFlare or equivalent in front of it, for CE ior EE you can do the same, use an enterprise class tool, or do it manually/custom scripts.

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top