Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

A strategy for large inventory import and update
 
Bonaventure
Jr. Member
 
Avatar
Total Posts:  29
Joined:  2007-09-26
 

I have 101000 products. My data is coming from a wholesaler. I’ve managed to import data at about one second per product (about 28 hours).
I can’t close my store for two days every time I need to recharge my inventory, so I’m brainstorming ways to keep everything running.

Would the following strategy work?
I would start by making two Magento installations on two different servers (one localhost and the other live). Then I would import products into one installation (28 hours), then copy this database into the other (this should be relatively fast). I can perform daily updates of stock and new products on the fly by mapping specific fields (I would be dealing with a much smaller amount of records). When I need to import my entire inventory again (about once a month), I could load it into my localhost installation while the live store continues to capture sales. Once localhost was updated, I could quickly copy the database over to the live installation. I don’t want to change my customer data and sales history, so I’ll need a way to move product data only.

Is this strategy typical of large e-commerce deployments? Can you think of pitfalls and/or advantages of using this strategy with Magento?

 
Magento Community Magento Community
Magento Community
Magento Community
 
jeff.d
Sr. Member
 
Total Posts:  103
Joined:  2008-05-26
 

I was planning on just updating changes on a daily basis. Hopefully DataFlow will get faster in the future. I have roughly 40k sku items to import so I am interested in any options. Please keep me posted on any progress you make.

 
Magento Community Magento Community
Magento Community
Magento Community
 
corbykissler
Member
 
Total Posts:  39
Joined:  2008-05-14
 

That sounds like a decent strategy.

I would say that this is not typical of a large ecommerce store ..  using other packages I have run across (for much more money) the process is a bit more streamlined.

I’m wondering why you can’t just update the product tables directly . .and may have to write something to do that - 1 product per second is completely unacceptable.

Will let you know what I find out - there doesn’t seem to be much activity other than users helping users.

Best of luck!

Corby

 
Magento Community Magento Community
Magento Community
Magento Community
 
nate001
Jr. Member
 
Total Posts:  14
Joined:  2008-08-15
 

I played with both adding products via a script and doing the product upload with the csv in the backend.  A script that uses Magento core can do this much faster then the stock backed system.

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top