Magento

Moshe ...I tested this and have logged the results on my linux box ... am I missing something ...have listed the detailed steps im my previous update ..please review..

Hi I’ve also ran the php -f replace_recursive.php command and restarted the apache but no improvement of the performance at all. Still far to slow to go in production.

Any advice is more than welcome.

Thanks,
Geert

Hi, me again,

Even with the advised settings:
realpath_cache_size=1M (careful, this is per apache process)
realpath_cache_ttl=86400 (ok for production site)

it is still very slow and no real improvement

@srinigenie: i can’t test it right now, let’s wait for salsasepp, i think he has more insight on this issue than me

@graes0: is APC or xcache installed?

Moshe - 27 May 2008 09:57 AM

@redpen: any difference if you add this to your xampp/apache/bin/php.ini as salsasepp suggested:

realpath_cache_size=1M

And restarting apache?

Hi Moshe.  No difference, sorry.

However, my main problem was the Search function, which could easily take over a minute to return simple results (we have over 2000 products).  BUT I’ve fixed it - I simply went into PHPMyAdmin and optimised all the Magento tables, some had slight overheads and presumably needed re-indexing.

20 seconds later and I’m now getting search results that only take 3 or 4 seconds!  Fantastic!

Hi Moshe,

No I don’t have APC and xcache installed on virtual private server. But according the Stefan the perfance should already be better even without APC and xcache

@redpen: glad to hear about the improvement

@graes0: bytecode cache will improve the performance and reduce memory consumption greatly, you should try it.

Attempted again (this time with xcache ON) and of no help still. It still seems to be looking abt for the files in various directories!!!
-------------------------------------------------------------------------------------------------------------------------------------------------------------------

7355 stat64("/home/xy/xyz.com/html/app/code/local/Mage/Sales/Helper/Reorder.php", 0xbfa8fd70) = -1 ENOENT (No such file or directory)
7355 stat64("/home/xy/xyz.com/html/app/code/community/Mage/Sales/Helper/Reorder.php", 0xbfa8fd70) = -1 ENOENT (No such file or directory)
7355 stat64("/home/xy/xyz.com/html/app/code/core/Mage/Sales/Helper/Reorder.php", {st_mode=S_IFREG|0777, st_size=1237, ...}) = 0
7355 fcntl64(18, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=1}) = 0
7355 fcntl64(18, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=1}) = 0
7355 time(NULL) = 1212031909
7355 lstat64("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
7355 lstat64("/home/xy", {st_mode=S_IFDIR|S_ISGID|0751, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app/code", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app/code/local", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app/code/local/Mage", 0xbfa89ab0) = -1 ENOENT (No such file or directory)
7355 open("/home/xy/xyz.com/html/app/code/local/Mage/Sales/Helper/Reorder.php", O_RDONLY) = -1 ENOENT (No such file or directory)
7355 time(NULL) = 1212031909
7355 lstat64("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
7355 lstat64("/home/xy", {st_mode=S_IFDIR|S_ISGID|0751, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app/code", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app/code/community", 0xbfa89ab0) = -1 ENOENT (No such file or directory)
7355 open("/home/xy/xyz.com/html/app/code/community/Mage/Sales/Helper/Reorder.php", O_RDONLY) = -1 ENOENT (No such file or directory)
7355 time(NULL) = 1212031909
7355 lstat64("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
7355 lstat64("/home/xy", {st_mode=S_IFDIR|S_ISGID|0751, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app/code", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app/code/core", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app/code/core/Mage", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
7355 lstat64("/home/xy/xyz.com/html/app/code/core/Mage/Sales", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0

@srinigenie: This is a normal PHP behaviour when include statement invoked, it just looks in all include_path parts.

Try increasing your realpath_cache_size

Hi Moshe,

As recommended by you I’ve installed xcache but I’m not impressed by the performance improvement. The slowest part of all is the adding to cart and completing the order and here I can’t notice any performance improvement. At the moment it takes too long and will scare my customers. Any new advise?
Thanks,

Geert

Short update from my side:
On my system, the optimum realpath_cache_size is 96k. See attached diagram.

I have done experiments with the original Zend framework as included in magento (no Moshe script run), APC off and varying the realpath_cache_size parameter from 16k to 1M. I see the value dropping until I hit 96k, then a further increase has no effect.

Note: As of PHP 5.2.0, the realpath cache is turned off when safe_mode or open_basedir are used, see http://www.php.net/releases/5_2_0.php . I believe this is still the case in current PHP versions, so watch out for these options when playing with your realpath cache.

The “red dot” in the diagram at 128k represents “22 lstat calls” at that cache size. This is what I had measured earlier (see my post above) and exactly once today, but I have not been able to reproduce it. Repeated attempts under varying circumstances always resulted in >10,000 lstat64 calls. So, right now, I am not trusting my procedure and I’ll have to call the “22” bogus.

Apart from that, re-ordering your include paths may help, see http://www.magentocommerce.com/bug-tracking/issue?issue=2100 .

I still get the best results with Moshe’s script (commenting out require_once’s in the Zend framework), and I still need to compare that against the realpath cache, and I still need to drill down to the real cause of all this. Hoping to find something that will benefit somebody else beside myself. Stay tuned. This is fun!

Click thumbnail to see full-size image

@moshe and all thread posters:
I had committed to providing more detailed results on performance with the Zend Framework patched with Moshe’s script. Sorry guys - can’t hold that up. Business decision, our shop project is on hold right now in favor of a different project. You’ll probably see me back some time after the 1.1 release. Looking forward to contribute and continue learning later. Keep up your excellent work!

All I have right now:
Indeed with Moshe’s script I can re-confirm the dramatic drop in lstat64 calls. Sorted out my procedure, I confused something before, but I can now positively reproduce my results. I have not tested that with realpath cache or APC, however.

I have googled dozens of articles about PHP and it’s require_once performance, all the way to this directive making the top-10 list of PHP DON’Ts. I wonder what the Zend people think about that, but I trust Varien has connections there. So long I’ll happily use Moshe’s script and I’m getting a decent performance improvement from that. Depends on your filesystem stat performance and the depth of your path to lib/Zend. Moving my shop from /var/www/magento to /magento alone saved almost 20% stat calls!
The main open questions still remains: How much actual performance boost does that get you? Some other posters reported no noticeable improvement. So where’s the trick? Will have to wait.

Magento is fun!
Stefan.