Over the past few months, we've dedicated significant resources to diagnose and optimize Magento's performance. Now that Magento's 1.0 feature set is mostly complete, we are glad to share some of our work and the effort that has gone towards the goal of making Magento fast and furious. We are extremely happy with the results of our testing and benchmarks.

With such extensive features, functionality and flexibility performance may sometimes take a hit, and Magento is no exception to this rule. But there are performance gains in every incremental improvement, and we're starting to see significant results in the latest release. Below are a few of the performance improvements that appear in Magento's latest release.

• Top menu join queries rewritten, top menu block is cached now on per store view level for every active category
• Added catalog indexes for layered navigation and search
• Added dynamic merge and whitespace remove during JavaScript load
• Footer block is cached
• Implemented <remove name=""> directive in layout xml to avoid instantiating of unwanted block objects, skipped left/right column blocks generation for pages with 1-column template</remove>
• Most of sales order attributes became static and were moved to one raw table
• Quote, quote address and quote items entities are stored separately
• Greatly improved popular tags block generation time
• Quote optimization for using in top links block and in the sidebar
• Implemented caching API for using in Mage_Core_Model_Abstract descendant models
• Removed unnecessary queries from catalog pages and homepage (e.g. reduced total number of database queries from 700 to 60 for homepage, and from 500 to 100 on product listing page on a database of 1000 products and 300 categories).

From our performance and load tests we have found that the latest Magento release outperforms version 0.8.17240 by almost double. Below we show a graph of one of the load tests we ran using Apache Benchmark (ab) comparing the two Magento versions. We experienced similar results throughout our tests and comparisons of the two versions.

DYI (Do it Yourself) Performance Enhancements

While we continue to improve Magento's core performance, there are additional improvements that Magento users can implement to yield performance gains in a single-server environment:

• Install a PHP opcode cache such as APC or XCache. These can deliver significant improvements to PHP's responsiveness by caching PHP code in an intermediate bytecode format, which saves the interpreter from recompiling the PHP code for each and every request.
• Make sure your Apache configuration has KeepAlives enabled. KeepAlives are a trick where multiple HTTP requests can be funneled through a single TCP connection. Since the setup of each TCP connection incurs additional time, this can significantly reduce the time it takes to download all the files (HTML, JavaScript, images) for a website.
• Use a memory-based filesystem for Magento's var directory. Magento makes extensive use of file-based storage for caching and session storage. The slowest component in a server is the hard drive, so if you use a memory-based filesystem such as tmpfs, you can save all those extra disk IO cycles by storing these temporary files in memory instead of storing them on your slow hard drive.
• Modify the configuration for your MySQL server to take better advantage of your server's RAM. Most Linux distributions provide a conservative MySQL package out of the box to ensure it will run on a wide array of hardware configurations. If you have ample RAM (eg, 1gb or more), then you may want to try tweaking the configuration. An example my.cnf is below, though you will want to consult the MySQL documentation for a complete list of configuration directives and recommended settings.

Example my.cnf:

Time to Cluster?

As any successful online retailer knows, a growing business means growing pains. If your store is a huge success, then you'll undoubtedly require more horsepower to keep your customers happy. This means leaving the world of single-server websites and moving to a clustered environment with two or more servers.

There are varying ways to accomplish this, but a typical first step is to add more web servers and place them all behind a load balancer, which will distribute the load fairly among all web servers. This method is popular because (a) web servers are cheaper than database servers; and (b) there's really no limit to the amount of web servers you can add.

Note: One current limitation in Magento is that there is no official method to propagate catalog images to each web node. If you're not comfortable setting up some sort of facility to do this, then you may want to hire a professional or try a different method of scaling.

A second method involves a performance tactic called “database replication.” When using replication, you will have two or more database servers in your cluster. One will be the “master” and the rest will be called “slaves.” The master is the only database server that accepts any sort of write-based queries. These write queries are then replicated by each of the slaves in real-time (or very close to it). The advantage of this setup is that Magento can issue read queries to any of the slave servers, saving all the write queries for the master database. A database is traditionally a difficult component to scale horizontally, so replication is a happy compromise.

The initial setup and configuration for MySQL replication is out of the scope of this document, but we can refer you to the experts. Here is a tutorial that explains all the details.

Once you have MySQL replication functioning, configuring Magento to use multiple database servers is a relatively simple task. The only file you will have to edit is app/etc/local.xml. An example is shown below. As you can see, there are two database connections configured, one called default_setup and one called default_read. The default_setup connection will be used for all write-based queries, and the default_read connection will be used for all read-based queries.

Example app/etc/local.xml:

This article has only covered a few of the well-known tactics for improving site performance. However, every website's scenario is slightly different, so it's often worth hiring a professional to appraise the needs of your site and provide you with possible upgrade options. But if you're comfortable with Linux and confident when working with configuration files, try out a few of our suggestions above. You may be pleasantly surprised.

Have any ideas of your own? We'd love to hear your thoughts. Leave a comment or join the Magento Performance and Optimization Group.]]> 2008-03-20T07:11:00-08:00 {url_as_title} sometimes, choosing the right web host makes all the difference.
take a look at http://www.magentify.com for instance, speed is really not an issue anymore.

Don’t know if this makes sense, but as far as I know, the system loads every time the etc/ folders and parses the XML files.

This is necessary for process new modules.

Here is the idea. Why not cache all XML files for 30 days, and when we want to add a new module we go to the admin section and we empty the cache, forcing Magento to process the XML files again.

Regards

admin panel is slow to load due to the dashboard. We are improving the dashboard and will make it available soon.

There is also a fix that was published on the boards here: http://www.magentocommerce.com/boards/viewthread/5366/P15/#t19853

This fix is helps with the caching of the JavaScript.

Thanks

yoav

Logging in to Admin takes about 6 seconds, loading the shop home page with no sample products takes about 5 seconds.

I believe my vps has minimum 256 mb ram and software such as Expression Engine and LiteCommerce run well.

Wat about caching templates (at least the catalog)?

will certainly increase disk space, but i think every template could cached for about 3 minutes?

Don’t know if is a viable alternative, but…

Regards

Thanks

yoav

I was just reading over on Yahoo Developer Network about some new ideas they have come up with for front-end performance improvements, which is also a good read (for theme developers).
http://developer.yahoo.net/blog/archives/2008/03/yahoos_latest_p.html

Maybe once you folk at Varien have finished optimising the back-end, you could also look at some front-end optimisation with the default theme?  (Also, it would be great to have a minimalist core ‘interface’ as a base for skinning/theming - I find the default one quite heavy)

Hopefully page caching will help reduce this, but it would be cool if some of those select statements got cached themselves to avoid this many trips to the db.

I’m glad to see Varien has an interest in performance, keep up the good work guys, and thanks for keeping us informed.

I am now wondering about the viability of a single box solution.  Magento make quite a number of round trips to the database per page.  It’s 60 on the home page and 100 on the product listing.  The contention between the webserver and database would lead one to speculate that there exists a good possibility of creating a bottleneck.