Magento Forum

DataFlow is too slow for my needs. 
 
rivertonstudios
Jr. Member
 
Total Posts:  1
Joined:  2009-01-16
 

Our store has hundreds of thousands of items.  The datalow that is provided with Magento is just too slow for me.  I did a test of importing 1000 items.  I did the math and it would take several days for me to get these items into my store.  That just isn’t gonna work.

I am looking for a script that imports the products faster.  If anyone knows anyone that has made a script that works faster, please let me know.  That is it. 

I am also wondering if I can create an osCommerce store and then do a migration using the Migration Tool.  Will this be any faster, or is it still really slow? 

I know this issue has been brought up in the past.  In my opinion, it has yet to be properly addressed.  I am also aware of someone in the process of creating a script who ended up stopping.

It doesn’t even have to be a script; it can be an extension.  Basically any way I can get products into the system faster. 
Thank you in advance. 

P.S. I can go into more detail about my situation if need be.  I didn’t put too much detail because I didn’t want people to have to read through all of it.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Damian Culotta
Enthusiast
 
Total Posts:  878
Joined:  2008-12-10
Argentina
 

Did you try with the Magento API?

 
Magento Community Magento Community
Magento Community
Magento Community
 
resonance
Member
 
Avatar
Total Posts:  62
Joined:  2008-09-24
 

From what I know, using any of the “official” ways to import products will be slow to do. I wrote a script (it’s a hack) that automaticaly imports configurable products with options, but since it ties into Magento and uses calls to save the object, it is not super fast. It’s here if it helps you: http://www.magentocommerce.com/boards/viewthread/30394/

Otherwise, for that may products, you may have to write something that directly inserts SQL into the database. It will take a little figuring to put things in the right places, but once done it’ll go super fast and you’ll be done. Be mindful that the schema changes regularly as they are building Magento out.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Mirasvit
Guru
 
Avatar
Total Posts:  639
Joined:  2009-08-22
 

Indeed when a large number of goods or attributes need to be imported, backend works very slow, as each time you save data the system re-indexes the goods through the entire catalog.

Asynchronous Re-indexing is a solution of this problem. When a product or category is saved it is not immediately re-indexed, but put into a queue. The queue is re-indexed in the background. This greatly speeds up the backend. This mechanism is implemented by using an extension of http://mirasvit.com/magento-extensions/magento-asynchronous-reindex.html.

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top