Magento

As many others I am a bit concerned about the performance of Magento.

I have noticed that when it is a while since I have visited my Magento store, it takes an awful long time to load the front page (12-15 seconds). However, on subsequent accesses it loads much faster (2-3 seconds). I first thought it had to do with the built-in caching feature, but after disabling all caching in the admin, it doesn’t seem to change much. Also the /var/cache/ directory is still populated with data.

It is not caused by local caching by the browser either, as I have tried using various “website speed test” sites and if I first access the site using one of these tools, it still loads much quicker on the second access from my own computer.

Does anyone know exactly what is causing this? And why it only happens on the initial page load? Also, it does not seem to matter if I access the front page or a subpage (such as /customer-service/) on the first access. It takes way too long to load the page in either way.

The overall performance is acceptable except for the initial page load, which of course also is the most important. A customer simply won’t wait for >10 seconds for the page to load the first time. I don’t see why hardware should be an issue here either, as it performs quite well after the initial load. It just seems like it is “halting” for about 10 seconds without doing nothing, then suddenly the page loads.

Is it the intricate templating system that is causing additional processing, database queries, or what? Other, similarly advanced and large PHP/MySQL-based applications runs much, much better than Magento on the same hardware.

I really hope someone from the community (or preferably the Magento team) can comment on this. I doubt I am the only one wondering about this?!

I think Magento is great and has the potential to become a market leader in its segment, I just hope the performance issue will get enough attention.

Thanks,
Sindre

the default UI provided by Magento is quite heavy in the sense that it loads around 200 KB of Javascript ( this is gzipped to 56KB though if deflate is enabled in apache), a hoard of CSS files requested individually all turn it a pretty slow page to render. The reason the page loads much faster the second time on is that these are cached by then and hence accessed quicker!!

Am myself looking at avoiding the multiple CSS requests and on removing the heavy JS libraries on the home page ...

srinigenie - 12 May 2008 10:12 AM

the default UI provided by Magento is quite heavy in the sense that it loads around 200 KB of Javascript ( this is gzipped to 56KB though if deflate is enabled in apache), a hoard of CSS files requested individually all turn it a pretty slow page to render. The reason the page loads much faster the second time on is that these are cached by then and hence accessed quicker!!

Am myself looking at avoiding the multiple CSS requests and on removing the heavy JS libraries on the home page ...

Thank you for your reply.

Although 250-500KB is quite a bit for a web page, it should not take close to 15 seconds to download. I am downloading at about 500 KB/s from the same server.

When you say the JS and CSS are cached, do you mean by the web server or locally (by browser)? I have all caching disabled in Magento, and use “forced” refresh when reloading the page (CTRL+F5 in IE).

I mean caching at the client side -that is at the browser - run the web page, all the JS and CSS get cached and the page loads fast. Now clear the browser cache and re-run and the performance is again poor

srinigenie - 12 May 2008 10:31 AM

I mean caching at the client side -that is at the browser - run the web page, all the JS and CSS get cached and the page loads fast. Now clear the browser cache and re-run and the performance is again poor

Well, actually it is not. I have both cleared the browser cache and forced a refresh using CTRL+F5 but the page still loads reasonable fast on the second try. The small slowdown caused by the client cache is understandable and expected, but it does not explain the large initial load / second load time difference.

Magento must be doing some extra processing during the initial load (aside from the built-in caching) that is causing the hang-up.

Hosting plays a part in speed, I’m using http://www.helpinghandhost.com/semi-dedicated.html and my site is flying.

-Lee

leeaston - 12 May 2008 12:25 PM

Hosting plays a part in speed, I’m using http://www.helpinghandhost.com/semi-dedicated.html and my site is flying.

-Lee

Well, yes, throwing more hardware at it is the easy solution, but it still does not answer why the store loads so considerable slower at the first access. The server I use for my Magento testing is otherwise fast enough.

Someone from the Magento team willing to comment on this?

I have the same problem, but worst. My initail page load takes over 1 full MINUTE. It’s awfull. And it’s not because of CSS and javascript. That’s only like 800K of data and I’m downloading from my server at 250KB/s. It shouldn’t take that long. Anyone found a solution to this? http://epinvest.t-e-a.ro

This just started for me too. Although I just noticed it after I added code to pull the newest products up on the homepage with this code:
{{block type="catalog/product_new" template="catalog/product/new.phtml"}}

Any way to fix this? My homepage load time is now over a minute!

I found out that the problem resides in the New Products block being loaded on the homepage. It’s all about the the MySQL querry being too intricate. Replicating the same query but using one temporary table to filter results (current date being between new_from_date and new_to_date) BEFORE applying the INNER and LEFT joins for selecting images, product status, product visibility and special prices seems to fix the problem (query takes 2seconds as oposed to like 60+ seconds). What I don’t know is how to do that using the new products block model without writing a custom MySQL query to fetch the results. Any ideas ?

Maybe it helps. Here it is the actual query being executed at load time:

core7.hostbase.net
[192.168.0.7]
# Query_time: 98 Lock_time: 0 Rows_sent: 5 Rows_examined: 1440
SELECT `e`.*, IFNULL(_table_news_from_date.value,
_table_news_from_date_default.value) AS `news_from_date`,
IFNULL(_table_news_to_date.value, _table_news_to_date_default.value) AS
`news_to_date`, IFNULL(_table_name.value, _table_name_default.value) AS
`name`, `_table_price`.`value` AS `price`,
IFNULL(_table_small_image.value, _table_small_image_default.value) AS
`small_image`, `_table_special_price`.`value` AS `special_price`,
IFNULL(_table_special_from_date.value,
_table_special_from_date_default.value) AS `special_from_date`,
IFNULL(_table_special_to_date.value,
_table_special_to_date_default.value) AS `special_to_date`,
IFNULL(_table_status.value, _table_status_default.value) AS `status`,
IFNULL(_table_visibility.value, _table_visibility_default.value) AS
`visibility` FROM `catalog_product_entity` AS `e`
INNER JOIN `catalog_product_entity_datetime` AS
`_table_news_from_date_default` ON
(_table_news_from_date_default.entity_id = e.entity_id) AND
(_table_news_from_date_default.attribute_id=’66’) AND
_table_news_from_date_default.store_id=0
LEFT JOIN `catalog_product_entity_datetime` AS `_table_news_from_date`
ON (_table_news_from_date.entity_id = e.entity_id) AND
(_table_news_from_date.attribute_id=’66’) AND
(_table_news_from_date.store_id=’1’)
LEFT JOIN `catalog_product_entity_datetime` AS
`_table_news_to_date_default` ON (_table_news_to_date_default.entity_id
= e.entity_id) AND (_table_news_to_date_default.attribute_id=’67’) AND
_table_news_to_date_default.store_id=0
LEFT JOIN `catalog_product_entity_datetime` AS `_table_news_to_date` ON
(_table_news_to_date.entity_id = e.entity_id) AND
(_table_news_to_date.attribute_id=’67’) AND
(_table_news_to_date.store_id=’1’)
INNER JOIN `catalog_product_entity_varchar` AS `_table_name_default` ON
(_table_name_default.entity_id = e.entity_id) AND
(_table_name_default.attribute_id=’45’) AND _table_name_default.store_id=0
LEFT JOIN `catalog_product_entity_varchar` AS `_table_name` ON
(_table_name.entity_id = e.entity_id) AND
(_table_name.attribute_id=’45’) AND (_table_name.store_id=’1’)
INNER JOIN `catalog_product_entity_decimal` AS `_table_price` ON
(_table_price.entity_id = e.entity_id) AND
(_table_price.attribute_id=’49’) AND (_table_price.store_id=0)
INNER JOIN `catalog_product_entity_varchar` AS
`_table_small_image_default` ON (_table_small_image_default.entity_id =
e.entity_id) AND (_table_small_image_default.attribute_id=’60’) AND
_table_small_image_default.store_id=0
LEFT JOIN `catalog_product_entity_varchar` AS `_table_small_image` ON
(_table_small_image.entity_id = e.entity_id) AND
(_table_small_image.attribute_id=’60’) AND (_table_small_image.store_id=’1’)
LEFT JOIN `catalog_product_entity_decimal` AS `_table_special_price` ON
(_table_special_price.entity_id = e.entity_id) AND
(_table_special_price.attribute_id=’50’) AND
(_table_special_price.store_id=0)
LEFT JOIN `catalog_product_entity_datetime` AS
`_table_special_from_date_default` ON
(_table_special_from_date_default.entity_id = e.entity_id) AND
(_table_special_from_date_default.attribute_id=’51’) AND
_table_special_from_date_default.store_id=0
LEFT JOIN `catalog_product_entity_datetime` AS
`_table_special_from_date` ON (_table_special_from_date.entity_id =
e.entity_id) AND (_table_special_from_date.attribute_id=’51’) AND
(_table_special_from_date.store_id=’1’)
LEFT JOIN `catalog_product_entity_datetime` AS
`_table_special_to_date_default` ON
(_table_special_to_date_default.entity_id = e.entity_id) AND
(_table_special_to_date_default.attribute_id=’52’) AND
_table_special_to_date_default.store_id=0
LEFT JOIN `catalog_product_entity_datetime` AS `_table_special_to_date`
ON (_table_special_to_date.entity_id = e.entity_id) AND
(_table_special_to_date.attribute_id=’52’) AND
(_table_special_to_date.store_id=’1’)
INNER JOIN `catalog_product_entity_int` AS `_table_status_default` ON
(_table_status_default.entity_id = e.entity_id) AND
(_table_status_default.attribute_id=’69’) AND
_table_status_default.store_id=0
LEFT JOIN `catalog_product_entity_int` AS `_table_status` ON
(_table_status.entity_id = e.entity_id) AND
(_table_status.attribute_id=’69’) AND (_table_status.store_id=’1’)
INNER JOIN `catalog_product_entity_int` AS `_table_visibility_default`
ON (_table_visibility_default.entity_id = e.entity_id) AND
(_table_visibility_default.attribute_id=’74’) AND
_table_visibility_default.store_id=0
LEFT JOIN `catalog_product_entity_int` AS `_table_visibility` ON
(_table_visibility.entity_id = e.entity_id) AND
(_table_visibility.attribute_id=’74’) AND
(_table_visibility.store_id=’1’) WHERE (e.entity_type_id = ‘4’) AND
(IFNULL(_table_news_from_date.value,
_table_news_from_date_default.value) <= ‘2008-06-03’) AND
((IFNULL(_table_news_to_date.value, _table_news_to_date_default.value)
>= ‘2008-06-03’) OR (_table_news_to_date.value is null)) AND
(IFNULL(_table_status.value, _table_status_default.value) in (1)) AND
(IFNULL(_table_visibility.value, _table_visibility_default.value) in (2,
4)) ORDER BY `_table_news_from_date`.`value` desc,
`_table_news_from_date`.`value` desc LIMIT 5;

And here the teaked query that did better in terms of timing::

select ee.*, IFNULL(_table_name.value, _table_name_default.value) AS
`name`, `_table_price`.`value` AS `price`,
IFNULL(_table_small_image.value, _table_small_image_default.value) AS
`small_image`, `_table_special_price`.`value` AS `special_price`,
IFNULL(_table_special_from_date.value,
_table_special_from_date_default.value) AS `special_from_date`,
IFNULL(_table_special_to_date.value,
_table_special_to_date_default.value) AS `special_to_date`,
IFNULL(_table_status.value, _table_status_default.value) AS `status`,
IFNULL(_table_visibility.value, _table_visibility_default.value) AS
`visibility`
FROM
(SELECT `e`.*, IFNULL(_table_news_from_date.value,
_table_news_from_date_default.value) AS `news_from_date`,
IFNULL(_table_news_to_date.value, _table_news_to_date_default.value) AS
`news_to_date`
FROM `catalog_product_entity` AS `e`

INNER JOIN `catalog_product_entity_datetime` AS
`_table_news_from_date_default` ON
(_table_news_from_date_default.entity_id = e.entity_id) AND
(_table_news_from_date_default.attribute_id=’66’) AND
_table_news_from_date_default.store_id=0

LEFT JOIN `catalog_product_entity_datetime` AS `_table_news_from_date`
ON (_table_news_from_date.entity_id = e.entity_id) AND
(_table_news_from_date.attribute_id=’66’) AND
(_table_news_from_date.store_id=’1’)

LEFT JOIN `catalog_product_entity_datetime` As
`_table_news_to_date_default` ON (_table_news_to_date_default.entity_id
= e.entity_id) AND (_table_news_to_date_default.attribute_id=’67’) AND
_table_news_to_date_default.store_id=0

LEFT JOIN `catalog_product_entity_datetime` AS `_table_news_to_date` ON
(_table_news_to_date.entity_id = e.entity_id) AND
(_table_news_to_date.attribute_id=’67’) AND
(_table_news_to_date.store_id=’1’)

WHERE (e.entity_type_id = ‘4’) AND
(IFNULL(_table_news_from_date.value,
_table_news_from_date_default.value) <= ‘2008-06-03’) AND
((IFNULL(_table_news_to_date.value, _table_news_to_date_default.value)
>= ‘2008-06-03’) OR (_table_news_to_date.value is null)) ORDER BY `_table_news_from_date`.`value` desc,
`_table_news_from_date`.`value` desc LIMIT 5) as ee

INNER JOIN `catalog_product_entity_varchar` AS `_table_name_default` ON
(_table_name_default.entity_id = ee.entity_id) AND
(_table_name_default.attribute_id=’45’) AND _table_name_default.store_id=0

LEFT JOIN `catalog_product_entity_varchar` AS `_table_name` ON
(_table_name.entity_id = ee.entity_id) AND
(_table_name.attribute_id=’45’) AND (_table_name.store_id=’1’)

INNER JOIN `catalog_product_entity_decimal` AS `_table_price` ON
(_table_price.entity_id = ee.entity_id) AND
(_table_price.attribute_id=’49’) AND (_table_price.store_id=0)

INNER JOIN `catalog_product_entity_varchar` AS
`_table_small_image_default` ON (_table_small_image_default.entity_id =
ee.entity_id) AND (_table_small_image_default.attribute_id=’60’) AND
_table_small_image_default.store_id=0

LEFT JOIN `catalog_product_entity_varchar` AS `_table_small_image` ON
(_table_small_image.entity_id = ee.entity_id) AND
(_table_small_image.attribute_id=’60’) AND (_table_small_image.store_id=’1’)

LEFT JOIN `catalog_product_entity_decimal` AS `_table_special_price` ON
(_table_special_price.entity_id = ee.entity_id) AND
(_table_special_price.attribute_id=’50’) AND
(_table_special_price.store_id=0)

LEFT JOIN `catalog_product_entity_datetime` AS
`_table_special_from_date_default` ON
(_table_special_from_date_default.entity_id = ee.entity_id) AND
(_table_special_from_date_default.attribute_id=’51’) AND
_table_special_from_date_default.store_id=0

LEFT JOIN `catalog_product_entity_datetime` AS
`_table_special_from_date` ON (_table_special_from_date.entity_id =
ee.entity_id) AND (_table_special_from_date.attribute_id=’51’) AND
(_table_special_from_date.store_id=’1’)

LEFT JOIN `catalog_product_entity_datetime` AS
`_table_special_to_date_default` ON
(_table_special_to_date_default.entity_id = ee.entity_id) AND
(_table_special_to_date_default.attribute_id=’52’) AND
_table_special_to_date_default.store_id=0

LEFT JOIN `catalog_product_entity_datetime` AS `_table_special_to_date`
ON (_table_special_to_date.entity_id = ee.entity_id) AND
(_table_special_to_date.attribute_id=’52’) AND
(_table_special_to_date.store_id=’1’)

INNER JOIN `catalog_product_entity_int` AS `_table_status_default` ON
(_table_status_default.entity_id = ee.entity_id) AND
(_table_status_default.attribute_id=’69’) AND
_table_status_default.store_id=0

LEFT JOIN `catalog_product_entity_int` AS `_table_status` ON
(_table_status.entity_id = ee.entity_id) AND
(_table_status.attribute_id=’69’) AND (_table_status.store_id=’1’)

INNER JOIN `catalog_product_entity_int` AS `_table_visibility_default`
ON (_table_visibility_default.entity_id = ee.entity_id) AND
(_table_visibility_default.attribute_id=’74’) AND
_table_visibility_default.store_id=0

LEFT JOIN `catalog_product_entity_int` AS `_table_visibility` ON
(_table_visibility.entity_id = ee.entity_id) AND
(_table_visibility.attribute_id=’74’) AND
(_table_visibility.store_id=’1’);

What I did eventually was creating a new custom attribute named “is_new” of type Yes/No and change the Mage_Catalog_Block_Product_New class to (I left the changed lines as comments):

class Mage_Catalog_Block_Product_New extends Mage_Catalog_Block_Product_Abstract
{
public function __construct()
{
parent::__construct();

$storeId = Mage::app()->getStore()->getId();

$product = Mage::getModel(’catalog/product’);
/* @var $product Mage_Catalog_Model_Product */
$todayDate = $product->getResource()->formatDate(time());
$products = $product->setStoreId($storeId)->getCollection()
->addAttributeToFilter(’is_new’, 1)
->addAttributeToSelect(array(’name’, ‘price’, ‘small_image’), ‘inner’)
->addAttributeToSelect(array(’special_price’, ‘special_from_date’, ‘special_to_date’), ‘left’)
;
//->addAttributeToSort(’news_from_date’,’desc’)
//->addAttributeToFilter(’news_from_date’, array(’date’=>true, ‘to’=> $todayDate))
//->addAttributeToFilter(array(array(’attribute’=>’news_to_date’, ‘date’=>true, ‘from’=>$todayDate), array(’attribute’=>’news_to_date’, ‘is’ => new Zend_Db_Expr(’null’))),’’,’left’)

/* @var $products Mage_Catalog_Model_Resource_Eav_Mysql4_Product_Collection */

Mage::getSingleton(’catalog/product_status’)->addVisibleFilterToCollection($products);
Mage::getSingleton(’catalog/product_visibility’)->addVisibleInCatalogFilterToCollection($products);

//$products->setOrder(’news_from_date’)->setPageSize(6)->setCurPage(1);

$this->setProductCollection($products);
}
}

The result is a dramatical improvement to like 4 seconds initial load time for Homepage as oposed to +60.

CalinT: great work on finding these “bugs”. Hopefully this is something that will be fixed in the v1.1 performance upgrades.

I am having the same problem. The front page loads very slow, not only the first time but subsequent times as well. It might take anywhere from 10 seconds to 4-5 seconds at best. And the problem seems to be in the front page only.

In comparison, the platform I am currently using to run my store, loads in a split second. I have tried accessing from different computers and different connections, both my existing store as well as the Magento installation where I am slowly migrating to, and the difference in load times is significant enough to wonder whether I should continue with the migration.

I hope there will be a solution soon.
Otherwise it is a fantastic product.