Magento Forum

Fatal Memory Error
 
seldon
Sr. Member
 
Total Posts:  92
Joined:  2007-11-08
 

Hi, using the product parser/adapter as an example, I created a category mysql import profile (as soon as it fully works ill post the steps/code). It works very well and imports about 1/2 of my categories (48 in total). The problem is that halfway in the collection saving process, i get this error:

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 196606 bytes) in /var/www/html/melrowdirect/magento/lib/Zend/Db/Statement/Mysqli.php on line 227

And I’m totally unsure what to do now and how to fix it. Does it really need more than 64mb of memory? Why on earth does it need that for only 48 categories?

I hope anyone could help me out here,

Seldon

PS. I added my category parser/convert/entity/adapter class for further info

File Attachments
Category.php  (File Size: 5KB - Downloads: 97)
Category.php  (File Size: 2KB - Downloads: 92)
Convert.php  (File Size: 1KB - Downloads: 97)
Category.php  (File Size: 4KB - Downloads: 133)
 
Magento Community Magento Community
Magento Community
Magento Community
 
seldon
Sr. Member
 
Total Posts:  92
Joined:  2007-11-08
 

Ok I traced the memory problem and it turns out that upon saving a category it also loads/saves all parents. This means that, depending on the depth of the category tree, the number of categories that get loaded/saved can be a lot larger than the number of categories in the tree. This is causing the massive memory usage. Also per category it uses about 240 kb of memory, i guess this is acceptable? Still seems to be quite a bit though.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Moshe
Magento Team
 
Avatar
Total Posts:  1770
Joined:  2007-08-07
Los Angeles
 

Thanks, great find, definitely an aspect for optimization.

As a temporary solution, try adding ->setOrigData() right before ->save() for category model.

 
Magento Community Magento Community
Magento Community
Magento Community
 
seldon
Sr. Member
 
Total Posts:  92
Joined:  2007-11-08
 

You mean at this spot (in the parser)?:

foreach ($collection->getIterator() as $model{
                         $model
->saveAsImport true;
                        
$model->setOrigData();                          
                        
$model->save();
                        
$i++;
                    
}

Cause that doesnt help :S

PS. where does it tell to also load/save the parent? Cause I cant seem to find that part yet.

 
Magento Community Magento Community
Magento Community
Magento Community
 
seldon
Sr. Member
 
Total Posts:  92
Joined:  2007-11-08
 

Ok, the problem is due to the _afterSave method in Mage_Catalog_Model_Entity_Category, there you have

$this->_saveCategoryProducts($object)
            ->
_updateCategoryPath($object$this->_getTree()->getPath($object->getId()));

The _updateCategoryPath is causing all parent categories to load & save. I solved my problem by checking for my custom import flag:

if (!$object->saveAsImport()) $this->_updateCategoryPath($object$this->_getTree()->getPath($object->getId()));

I would advise to solve the problem by updating the path using a very leight version of the objects (where only certain necessary properties are loaded), or writing a specific method that updates the path.

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top