Magento

I’ve been working with the Magento system for about a week now and the one things I’ve noticed is that it is just flat out slow on both the front and backend of the system.  One of the biggest problems with OScommerce has always been the speed of those systems and I would have thought loading speeds limiting database queries would have been a major focus of this new system.  I have to say while it looks beautiful if we can’t increase user speed on both the front and backend I can’t see how it will ever be usable.  Are there any tricks to increasing speeds… I’m running my site on a dedicated rackspace server so I know that the server isn’t the problem.. (at least nothing obvious).

Thanks

Further optimization is one of our next steps, which will include increase of performance and decrease of memory consumption.

As PHP does not have application wide memory storage, it looks like some optional but significant optimization steps will require installation of additional extensions/services such as memcached, and/or running background scripts, that would serve data on demand instead of loading it with every instance, for example cached configuration or layouts.

I don’t know anything about the internal design of Magento, but from a very general point of view there’s not a lot it can do that takes any noticeable time, unless you have hundreds of simultaneous users. Interpreting the scripts should be almost instantaneous on just about any modern hardware. The only thing that really takes time on a server is database and file access, plus of course any communication delays.

Is your database tuned beyond the default settings? Basically, if the database manager has memory enough it will have all the data it needs in the cache, and so database access will also be just the processing time.

I’m just about to install Magento right now and speed is a concern for me. There is a thread about SEO that has quite a few people excited, but honestly, the speed is a concern for me. In my experience such SEO issues can be resolved more easily compared to efficiency issues. So speed is more important to me.

We do happen to have hundreds of simultaneous users (customers)...and isn’t that the goal? And then we also have managers CSRs and warehouses staff. Oh, and don’t forget the crons that run for various reasons.

Database tuning is nice, but you will run into a wall if you have lots of traffic and bad query logic. When you do, you’ll find that now you need to scale in ways that WILL require redesign of the application. That costs money and you just shake your head saying, why wasn’t this done right the first time? These problem are age old and there are solutions available if people would just sit down, think and then design first.

Too many PHP-based content system do database base queries for all content. Why? Isn’t most content static? Pull it out of memcache or from a file on your web server(s). If the data or template changes, update your cached version.

It’s simple, use the database for synchronous permanent storage and computational tasks it can do more efficiently (and effectively). Personally, I do my best to keep the load OFF of the database because it WILL become an unnecessary bottleneck. Fancy and expensive hardware arrays should always come last.

There’s obviously much more that we can get into, but it will be nice to see at least some basic optimization at the Magento level (if it exists). I’ll try to help where my time allows because this is something I’d like to see...Evolved (once and for all).

Speed! Speed! Speed! Nobody likes waiting.

I agree, even the beta testing can be slow and just grinds on me.  I love a lot of what is trying to be accomplished here and I would start putting in my own speed optimizations but I don’t want the next core upgrade to come along and break everything.

Yeah, this is also a main concern for me at the moment as well.  A lot of developers are putting time into working out the software and how to customise it for their purposes, unfortunately at the moment it is borderline unusable because of speed issues. Can anyone at Varien give us an idea of when we can expect an update in terms of performance?

There is no set time table for this. Performance will be greatly improved in the weeks ahead as we move forward with development and I do expect future version to reflect this.

Thanks for the reply Roy. I hope we aren’t coming across as ungrateful =)

I think the amount of passion people have for your product says a lot about what you guys are trying to achieve with Magento. We all just want to get our hands on it and make it work well for our clients. The open source commerce scene at the moment is breathtakingly bad, but with Magento developers are finally seeing some kind of lighthouse beacon to lead us away from the rocky coastline of OSCommerce =)

Ok, perhaps a bit dramatic, but you get the point =)

We absolutely get it and respect all feedback. I appreciate the comments and keep ‘em coming (we don’t take anything too hard)

Excellent. If it is possible, an update when you guys get the speed issues sorted would most likely rank up there as #1 wishlist item with all developers working with Magento at the moment. All the other features we realise are also being currently developed, but i think most of us can live without those in the meantime =)

As i replied on German forums , Magento’s performance is nearly as good as in highly modified osCommerce. I’ve made 2 tests using pingback

That shows us:

Magento ( http://demo.lento.pl/magento ) - my version of M for testing purposes
Total loading time:
9.1 seconds
Total objects:
25 (486.4 KB)
External objects: 0

Netcart (http://demo.netcart.eu) - this is my actual version of modified osC but with optimised DB structure and queries.

Total loading time:
8.3 seconds
Total objects:
108 (348.8 KB)
External objects: 1 (20.6 KB)

I’m using a high-end server for my development purposes so that may be different on many other servers .

Processor Intel 2 x Xeon 3.0 GHz
4 GB RAM
HD: 320GB (RAID1) x 2
Linux Debian

Now M isn’t quicker than my osC but we can espect that the time for optimising M will come .

TDub - 10 September 2007 09:48 AM

Fancy and expensive hardware arrays should always come last.

I totally agree. When we talk about scaling a CMS solution, architecture and design is number one and two.

On the other hand, every day I see a lot of people trying to access tiny, underspecified web servers from remarkably overblown desktops. We all seem to have agreed that it takes a new generation of hardware to run every new generation of desktop software. I just want to make sure that everybody understands that it’s the same with server software - moving to a CMS will consume a lot more server resources that a bespoke hand-coded solution ever did.

I frequently see people expecting to run Magento-class software in shared hosting environments, or in 128MB VPS solutions. That’s just ignorance. Whenever you want to run anything CMS-like on MySQL, get at least a gig of server storage, half of which you give to MySQL. You know when you’ve done this correctly, because you will have changed at least two settings in MySQL.

I share your worries that Magento may suffer from bad query logic. In that case it will certainly scale poorly. But the complaints I’m hearing here are also related to one-user development performance. I just want to make sure that anyone complaining about poor single-user performance has at least a 90% database cache hit ratio. If your figure is worse, or if you didn’t know you had one, you now know why Magento is slow on your server, and what to do about it. Stop cheating.

I finally got it installed. With images turned off page loads are about 1 second on my VPS:
Product detail page

However, there are A LOT of queries required to generate that page:
Query log for product detail page

So, I wondered what the response would be on my VPS test server under a small load. I ran ab and I can post specs if necessary, but it looked “ok”.

Magnus Wester - 11 September 2007 12:02 AM

I frequently see people expecting to run Magento-class software in shared hosting environments, or in 128MB VPS solutions. That’s just ignorance.

*TDub blends in with the crowd to avoid the lynch mob*

Magnus Wester - 11 September 2007 12:02 AM

I share your worries that Magento may suffer from bad query logic. In that case it will certainly scale poorly. But the complaints I’m hearing here are also related to one-user development performance. I just want to make sure that anyone complaining about poor single-user performance has at least a 90% database cache hit ratio. If your figure is worse, or if you didn’t know you had one, you now know why Magento is slow on your server, and what to do about it. Stop cheating.

I think that’s a good point. For me however, our server’s query cache fills up quickly simply because some of the dataset results are HUGE (the result of custom searches). So within about a minute, the query cache has garbage collected any usable catalog/product data. For me dataset caching is stopgap. It’s just too volatile to be effective for us, but for the mom and pops, sure, it’ll probably help 40-50% of the time.

I glanced at the log I posted with SELECT queries for the product detail page and it looked very clean. I also glanced at the db schema and right off the bat I noticed that there are 157 tables and the normalization seems very decent. OSC was not nearly as flexible and one look at the entity diagram reveals it.

So, generally, I think that Magento can run pretty well in VPS/shared environments for the average small business...I’m just concerned for the rest of us.

Magnus Wester - 11 September 2007 12:02 AM

I frequently see people expecting to run Magento-class software in shared hosting environments, or in 128MB VPS solutions. That’s just ignorance.

True “ignorance” is really believing that everyone that wants to use Magento will be willing to pay for a dedicated box. The vast majority of users down the road will most definitely be using it in a shared hosting environment.

What always blows me away is the early adopters (as in all the techno-geeks here - myself included) honestly believing that they’re going to be the “average” user. Nothing could be further from the truth. The average user is going to be a small time operation that just wants to sell their “stuff” on the web and likes that fact that Magento is everything they could ever need. They’ll be the ones asking “How do I change the logo?” for the thousandth time in the forums (a quick peek around the OSC or Zen forums will confirm that they’re there by the thousands). Get ready for it....they’ll be here when Magento is ready for production use. They will NEVER be interested in their own server (except in the off chance that their business explodes and warrants it). Most never will.

Put it this way....OSC runs fine in a shared environment. Zen Cart runs fine in a shared environment (well, as “fine” as either of those hunks of spaghetti code can anyways). Most others likely do as well. If Magento doesn’t it will be missing the boat big time.

Of course, I’m not in any way suggesting that Varien isn’t going to be 100% sure that shared hosting will work just fine. I know they’ll be sure that it does. 

/rant

I certainly hope that the average user hoping to utilize Magento is well versed in XHTML, CSS, some PHP, Linux system administration and has the ability to understand why Magento’s layout and templating system is the way it is, the feature which has the highest learning curve in my opinion.

I think the “small timers” who “just want to sell their stuff” should probably stick to something remotely hosted and FAR MORE simple like Shopify.  If not, Varien is going to have to hire a full time forum manager to constantly re-categorize posts into the newbie category and we’re all screwed!

Overall, I see Magento as a tool for developers rather than a software for end users.

However, I think it’s inevitable that we will see a large number of sites with the exact same default layout with a couple colors changed, a few image replacements (logo), and terrible product images.

Time will tell.