Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

CLI product update script memory usage increased very quickly
 
gunardiwu
Jr. Member
 
Total Posts:  13
Joined:  2008-03-14
 

Hi,

We created a script to read external files created by our ERP system and use it to update product info (such as expiry date, quantity, etc).

We currently have only 500+ number of products live, and this already consumed almost 300Mb of memory space. Here’s the sample output from the script:

555056836 2009-04-04 03:41:19 [DEBUG] Starting to import file /vol/hw/ftp/product_update/200943162930.xml
555056836 2009
-04-04 03:41:19 [DEBUG] 0 Memory usage 4424656 bytes
555056836 2009
-04-04 03:41:20 [DEBUG] product_id800011 UPDATED.
555056836 2009-04-04 03:41:20 [DEBUG] 1 Memory usage 12886032 bytes
555056836 2009
-04-04 03:41:22 [DEBUG] product_id801827 UPDATED.
555056836 2009-04-04 03:41:22 [DEBUG] 2 Memory usage 13381964 bytes
555056836 2009
-04-04 03:41:23 [DEBUG] product_id801881 UPDATED.
555056836 2009-04-04 03:41:23 [DEBUG] 3 Memory usage 13845044 bytes
555056836 2009
-04-04 03:41:25 [DEBUG] product_id801888 UPDATED.
555056836 2009-04-04 03:41:25 [DEBUG] 4 Memory usage 14304132 bytes
555056836 2009
-04-04 03:41:26 [DEBUG] product_id810267 UPDATED.
555056836 2009-04-04 03:41:26 [DEBUG] 5 Memory usage 14772048 bytes
555056836 2009
-04-04 03:41:28 [DEBUG] product_id879551 UPDATED.
555056836 2009-04-04 03:41:28 [DEBUG] 6 Memory usage 15278016 bytes

As you can see, the first time it encountered an existing product the memory usage increased by 8Mb and 450Kb for each following products.

With this rate the memory usage will climb to 1Gb for 2000+ products and surely will break if we set the allowed memory to lower than that.

Question:
Once we updated the product, it will no longer needed, but the script seems to cache-ing the object in the memory, but we can’t be sure if it’s really the cache-ing or something else. Is there a way to use the memory more efficiently without the cache-ing or some sort of it?

We run this script on a machine with less than 2Gb machine, and the number of products is growing steadily, which mean we are going to run into problem with exhausted memory allocation soon.

Please help

Thanks

Gwu

 
Magento Community Magento Community
Magento Community
Magento Community
 
Periculi
Sr. Member
 
Total Posts:  249
Joined:  2009-02-03
 

Fooman reported finding a similar issue, I think, and posted a few fixes.  Magento apparently has some memory leaks. 
http://www.magentocommerce.com/boards/viewthread/36312/

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top