Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

Reindexing Huge Product
 
amsoft2001
Jr. Member
 
Total Posts:  6
Joined:  2011-06-27
 

Hi,

In my magento installation, I have around 50K products. I cannot perform redindexing. I have allowed memory to 256M, still it goes out of memory when I try to reindex from shell (shell/reindex.php)

Is there any way to optimize tables before reindexing or delete some unnecessary data from any table? It’s really a pain :(

 
Magento Community Magento Community
Magento Community
Magento Community
 
chiefair
Mentor
 
Avatar
Total Posts:  1848
Joined:  2009-06-04
 

With 50k products, a realistic memory_limit is in the 768M range.

Since this is being run from the command line, you are being restricted by the command line memory limit (separate from the GUI memory_limit) which needs to increased enough to let anything run from the command line.

If it’s not possible to get your hosting provider to set the CLI memory_limit, you might try editing the indexer.php file by including

ini_set('memory_limit''768M');
on a line previous to
require_once 'abstract.php';
at the top of the file.

If you cannot increase your memory_limit to anything more than 256M, it’s time to move to a new hosting provider that will allow you the resources so that Magento will run.

 
Magento Community Magento Community
Magento Community
Magento Community
 
amsoft2001
Jr. Member
 
Total Posts:  6
Joined:  2011-06-27
 

Hi,

Thanks for the help. I have tried doing that already. I mean setting memory limit at the top of indexer.php, but no matter what memory limit I set, I always get that the memory is limited to 256M

PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 87 bytes) in /var/www/vhosts/educatorsden.com/httpdocs/lib/Varien/Data/Collection.php on line 562

What shall I do now?

Regards
Rakesh

 
Magento Community Magento Community
Magento Community
Magento Community
 
serpyre
Enthusiast
 
Avatar
Total Posts:  771
Joined:  2013-05-20
 

Try php -q -d memory_limit=768M shell/indexer.php reindexall

 
Magento Community Magento Community
Magento Community
Magento Community
 
amsoft2001
Jr. Member
 
Total Posts:  6
Joined:  2011-06-27
 

Nope :( Still exhausted at 256M. It seems a hard limit is applied. But I am not sure how to remove or increase a hard limit :(

 
Magento Community Magento Community
Magento Community
Magento Community
 
chiefair
Mentor
 
Avatar
Total Posts:  1848
Joined:  2009-06-04
 

You contact your hosting provider and tell them the CLI limit needs to be increased.

They refuse, you find a new hosting provider interested in allowing you to run Magento.

It’s that simple.

All too many Magento problems are actually server issues where neither Apache, MySQL or PHP are being given the resources to run Magento or are configured to be fair to all the other users on a shared host. If you’re not on a shared host, then you’re dealing with stock, default settings which usually is a bad sign since a LAMP stack with default settings is only capable of running small, boutiquey websites with 500 or less product.

 
Magento Community Magento Community
Magento Community
Magento Community
 
amsoft2001
Jr. Member
 
Total Posts:  6
Joined:  2011-06-27
 

HI chiefair,

I found a strange thing. How come the indexer reads the .htaccess when running from command line? I increased the memory limit value in htaccess to 768M, and it did the job!! I did not realize command line could read .htaccess

Though my problem is resolved, but just for curiosity - Any idea?

Kind Regards
Rakesh

 
Magento Community Magento Community
Magento Community
Magento Community
 
chiefair
Mentor
 
Avatar
Total Posts:  1848
Joined:  2009-06-04
 

Wow, you’re in luck!

Usually there is a php.ini for each way PHP can be run, so you have one for CLI, one for fCGI, one for Apache DSO module. And Apache DSO module is usually the only one that reads from .htaccess

Looks like your server admins were looking out for you and configured PHP to pull the memory_limit setting out of .htaccess for any way PHP may be run.

It is not a usual setup, but one I wish I had on our original server as we quickly overran the 128M limit and started losing data soon thereafter.

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top