Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

javascript and css optimization
 
lisali
Enthusiast
 
Avatar
Total Posts:  889
Joined:  2008-04-28
London, UK
 

Hi Mencargo,

That’s a good idea - but Magento needs more than just those two. No less that 12 JS files are called initially:

prototype/prototype.js
prototype/validation.js
scriptaculous/builder.js
scriptaculous/effects.js
scriptaculous/dragdrop.js
scriptaculous/controls.js
scriptaculous/slider.js
varien/js.js
varien/form.js
varien/menu.js
mage/translate.js
mage/cookies.js
lightbox/lightbox.js (if using lightbox - that’s 13)

Minify works - but the page generation time still takes way too long and that is due to unoptimised Magento caching and SQL queries overload.

 
Magento Community Magento Community
Magento Community
Magento Community
 
rrochelle
Jr. Member
 
Total Posts:  11
Joined:  2008-08-15
 

Hi lisali,

sounds interesting to minify the css. I have my store currently hosted on a 1und1 Business pro account and at least got the js to compress to around 100KB using a php.ini file in the js directory to enable zlib output compression.

Unfortunately, this did not do the trick for css. So my question is: How did you enable this by google minify? Which steps does one have to take?

Cheers,
rrochelle

 
Magento Community Magento Community
Magento Community
Magento Community
 
i960
Guru
 
Avatar
Total Posts:  633
Joined:  2007-10-01
Bakersfield, CA
 

@lisali - The following files are automatically included when you load scriptaculous.js:

scriptaculous/builder.js
scriptaculous
/effects.js
scriptaculous
/dragdrop.js
scriptaculous
/controls.js
scriptaculous
/slider.js

And Google is hosting those files.  So using Google’s service would take care of 6 of the files.  However, I just noticed something.  If those extra scriptaculous files are dynamically loaded by scriptaculous.js, then why is Magento loading them individually on each page?  I also noticed when going through the xml files that scriptaculous.js is only being called on the compare products page.  So I am wondering if scriptaculous is only needed on the compare products page, but is being loaded everywhere. Or is it needed everywhere, but it is being loaded twice on the compare products page.  Either way, that’s not good.  I can only guess that the reason for loading the files individually is so they will end up being concatenated by the index.php script.  I wish I knew exactly what parts of Magento are using each of these javascript libraries so I can see if it would be possible to just eliminate them altogether.

 
Magento Community Magento Community
Magento Community
Magento Community
 
lisali
Enthusiast
 
Avatar
Total Posts:  889
Joined:  2008-04-28
London, UK
 

@ rrochelle - I’m sorry to hear you’re with 1and1. I am too. They are a complete bunch of incompetent drones. What you need to do is download the Minify script from
http://code.google.com/p/minify/

Then you need to upload it to your server as per the instuctions enclosed there. Then just go to http://www.mysite.com/min for a GUI solution. The easiest way to catch all the links is to use Firefox and the scriptlet link provided there.

Two notes: do set up a temp directory so Minify can cache the compressed files. Check config.php and uncomment as appropriate. Also, for some reason, the link builder will report that your server does not support rewrites. If it says that, you can add this line to the.htaccess file inside the min directory:
RewriteBase /min/

Hope this helps and good luck!

@i960 - That is very interesting, did not know that about Scriptaculous. Wonder what Magento would neet Scriptaculous for to compare products, as that page looks very static to me. Any way - with Minify, I have been able to compress all JS files down to about 50k. Almost bearable now, although the initial page generation time can stil take 1-2sec - and that’s before anything can start happening and any css/js and images can start downloading.

 
Magento Community Magento Community
Magento Community
Magento Community
 
i960
Guru
 
Avatar
Total Posts:  633
Joined:  2007-10-01
Bakersfield, CA
 

I’ve done some testing using Google’s hosted API service, and I don’t think it’s a very good solution for Magento.  If we were in a situation where a large number of websites were using it, then there would be a good chance that a customer visiting your site would already have the files cached.  But that isn’t the case.  Here are the results I came up with:

Default Magento:

3.7K    1    HTML/Text
87.5K    1    JavaScript File
15.6K    5    Stylesheet Files
15.3K    9    CSS Images
18.3K    7    Images
140.7K    Total size
23    HTTP requests

Loading prototype/scriptaculous via Google:

3.7K    1    HTML/Text
104.5K    10    JavaScript Files
15.6K    5    Stylesheet Files
15.3K    9    CSS Images
18.3K    7    Images
157.7K    Total size
32    HTTP requests

Obviously that’s not an improvement at all.  The only benefit I can see is if the visitor already has the files in cache.  Plus, Google does not host minified versions of prototype/scriptaculous.  So the difference will be larger if you pre-compress the files or use the minify app.  BTW, the way I made this work was to first comment out all of the prototype/scriptaculous files in layout/page.xml, then I added the following to template/page/html/head.phtml

<script src="https://www.google.com/jsapi"></script>
<script>
    
google.load("prototype""1.6.0.3");
    
google.load("scriptaculous""1.8.1");
</script>

You can also load the files directly which would save you one HTTP request and about 4 KB taking into account gzip compression.  Now if we were to dump prototype in favor of jquery, then using Google’s service might be ok.

I think for me the best solution is going to be to just pre-compress all of the js files that are being loaded on every page using YUI compressor and make sure it’s gzipped and the appropriate headers are being set.  Magento’s index.php file in the js directory already handles the last two, so all I would need to do is combine and pre-compress the files, and then load that one file via page.xml.  And just an FYI, these are the headers being sent from my server using the index.php script:

HTTP/1.1 200 OK
Date
Fri03 Oct 2008 17:29:30 GMT
Server
Apache/2.2.9 (Win32mod_ssl/2.2.9 OpenSSL/0.9.8h PHP/5.2.6
X
-Powered-ByPHP/5.2.6
Last
-modifiedMon15 Sep 2008 19:03:00 +0000
Expires
Sat03 Oct 2009 17:29:31 +0000
Content
-Encodinggzip
Vary
Accept-Encoding
Transfer
-Encodingchunked
Content
-Typetext/javascript
 
Magento Community Magento Community
Magento Community
Magento Community
 
i960
Guru
 
Avatar
Total Posts:  633
Joined:  2007-10-01
Bakersfield, CA
 

Well, I just noticed something after playing around with the minify app.  My only complaint with it is that it uses JSMin, which chokes on validation.js.  I thought the only way I could use YUI Compressor is to do each file by hand.  Well, I just noticed the following file included with the minify app: /min/lib/Minify/YUICompressor.php.  If I can figure out how to use that instead of JSMin, then I think that’s going to be the ideal solution.  Especially if it will do CSS as well.

 
Magento Community Magento Community
Magento Community
Magento Community
 
lisali
Enthusiast
 
Avatar
Total Posts:  889
Joined:  2008-04-28
London, UK
 

Hi i960,

validation.js is a problem - as Varien have modified the original file and there’s an error in the script now. The original script compresses fine but does not work with Magento.

Minify does CSS too, but the problem is that I could not get it to work properly with relative links, so had to set them all to absolute. The compression gained more than makes up for that.

smile

 
Magento Community Magento Community
Magento Community
Magento Community
 
i960
Guru
 
Avatar
Total Posts:  633
Joined:  2007-10-01
Bakersfield, CA
 

I haven’t been able to get this to work.  There doesn’t seem to be any documentation showing how to use any of the alternative javascript compressors.  I did manage to get it to use YUI instead, but I can’t set the necessary options for it, such as the path to the yui compressor .jar file.  When I set the options, I get errors like this:

PHP Fatal error:  Class 'Minify_YUICompressor' not found

If I don’t try to set any options, I get errors like this:

PHP Fatal error:  Uncaught exception 'Exception' with message 'Minify_YUICompressor : $jarFile and $tempDir must be set.'

I know I’m just doing it wrong, but without any documentation I’m left in the dark on how to use this.

lisali, I’m curious how you are handling the validation.js issue.  Since JSMin chokes on it, I’m guessing you are loading that one with it’s own script tag and compressing it manually (or using the one I provided).  If I were to go through the trouble of loading that one separately, then I might as well just compress all of the files manually and not use Minify.  But I’d much rather have an all in one solution like Minify and let it handle everything.  Just getting it to use YUI is going to be the solution for me if I can figure it out.  It won’t be a universal solution though because YUI requires java on the server.

I also noticed that the latest version of Minify is supposed to rewrite relative links in the CSS for you.  Is it not doing that?

 
Magento Community Magento Community
Magento Community
Magento Community
 
lisali
Enthusiast
 
Avatar
Total Posts:  889
Joined:  2008-04-28
London, UK
 

Hi i960,

Regarding validation.js - the problem seems to be with a ‘ symbol in a regex which is used to check for valid email addresses. After DAYS of figuring out, I found that a tiny little symbol was the culprit. I just removed it - it will now not give an error if a user types an email with a ‘ in it - but I can live with that. Around line 368, after the second a-z0-9,!#\$%&’

Remove the last symbol. Try that! Maybe a javascript wizard can help us with a proper regex that will work.

CSS and links - am using the latest version of Minify but it still did not work for me. There are some additional settings in Minify, I tinkered with those for a day or two, but in the end thought it was not worth it. So I have just used find&replace;to convert all the links in the CSS files to absolute. It may just be my server configuration.

Good luck!

 
Magento Community Magento Community
Magento Community
Magento Community
 
FOOMAN
Guru
 
Avatar
Total Posts:  669
Joined:  2007-12-13
Auckland, New Zealand
 

Thanks lisali for your detective work. I have updated our Speedster extension to now include a validation file which can be minified.

We are fairly happy with how the extension currently performs, but while working on it we have run into a few issues:

Magento is too flexible for its own good
The main problem for caching lies in the flexibility of Magento. If you wanted to, you could load different CSS and JS for every page. While this flexibility is nice to have it causes major headaches. How do you know what to cache if you don’t even know what is going to be required for this page?

An idea would be to include everything that is specified in page.xml on all pages. This could then be packaged in one file and wouldn’t need to be loaded on subsequent pages. A typical page might then look like this:

Sidewide.js (combination of all js files which are needed across the store)
Page.js (combination of all js files needed only for this page)

Sidewide.css (combination of all css files which are needed across the store)
Page.css (combination of all css files needed only for this page)

Unfortunately at the stage where the actual Javascript and CSS is written we don’t know how to find out where it was requested from. Anyone got any ideas?

URLs need to change for updates
My understanding is that if a browser has downloaded something and it hasn’t expired it will not ask the server if there has been an update for it. This leads to the problem that you can’t force the browser to load a new version of an updated file simply because it doesn’t ask for it. The only way around this is to change the url of the requested file. Magento doesn’t handle this properly either at the moment. We are trying to address this in the extension by adding a version number into the url.

Feel free to give our extension a shot and let us know if it works for you. I am happy to help with any issues.

ps: on our host we had some problems with the extension in conjunction with zlib.output_compression on. We had to move m.php to a different folder to make it work. If others run into the same problem we might make this the default in future versions.

 
Magento Community Magento Community
Magento Community
Magento Community
 
lisali
Enthusiast
 
Avatar
Total Posts:  889
Joined:  2008-04-28
London, UK
 

Hi Fooman,

Many thanks for your work on this extension - glad I could help out.

Could you tell me what changes you made to validation.js - did you also remove the same symbol, or did you modify the regex to make it work?

In Minify, one can force the script to check for updated files as often as one wants to, in config.php. Further to that, it can check the timestamp of the file. Not sure if that can help with your extension, but thought I’d tell you just in case.

smile

 
Magento Community Magento Community
Magento Community
Magento Community
 
FOOMAN
Guru
 
Avatar
Total Posts:  669
Joined:  2007-12-13
Auckland, New Zealand
 

Hi Lisali,

I only removed the offending ’. I had to save it as a new file (validation-4min.js) so it can be distributed in the extension (I was unable to overwrite the original file).

Thanks for the tips. Unfortunately I haven’t been able to find the fix-all-solution, yet. The method to include the timestamp in the url works great if you have fixed groups of files which you know upfront. You then can simply have the files as static groups in groupsConfig.php and call them in your HTML generating bit with a call to Minify_Build. Since in Magento the JS/CSS can potentially change on every page this currently doesn’t play together too nicely.

And you are correct that Minify is fairly clever in knowing when a file has been updated. But the problem is that the browser reads the old file from its own cache and never asks Minify if there is a new version, unless there is a changed url.

 
Magento Community Magento Community
Magento Community
Magento Community
 
FOOMAN
Guru
 
Avatar
Total Posts:  669
Joined:  2007-12-13
Auckland, New Zealand
 

Hi all,

I am happy to announce the release of a further update to our Speedster extension

+ The inclusion of a newer minify library now allows the CSS to automatically adjusts to the current Magento skin - no complicated configuration any more.
+ The Url generated includes a timestamp of the last modified file

Let me know how it works for you.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Sherif
Jr. Member
 
Total Posts:  2
Joined:  2008-09-20
 

hi

i tried the fooman module and i went through the steps you declared , i have magento in /magento folder i edited my .htaccess files to be like this
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteRule ^minify/([^/]+)/(.*\.js) magento/lib/minify/m.php?f=$2&$1
RewriteRule ^index.php/minify/([^/]+)/(.*\.js) magento/lib/minify/m.php?f=$2&$1

RewriteRule ^minify/([^/]+)/(.*\.css) magento/lib/minify/m.php?f=$2&$1
RewriteRule ^index.php/minify/([^/]+)/(.*\.css) magento/lib/minify/m.php?f=$2&$1

when i try to view source of my magento frontend , the scripts links are like this

/magento/default/minify/1223300150/js/prototype/prototype.js,js/prototype/validation-4min.js,js/scriptaculous/builder.js,js/scriptaculous/effects.js,js/scriptaculous/dragdrop.js,js/scriptaculous/controls.js,js/scriptaculous/slider.js,js/varien/js.js,js/varien/form.js,js/varien/menu.js,js/mage/translate.js,js/mage/cookies.js,js/lightbox/lightbox.js,js/sb/src/js/lib/yui-utilities.js,js/sb/src/js/adapter/shadowbox-prototype.js

I assume /default/ is a wrong directory but i don’t know why its there

please advice

 
Magento Community Magento Community
Magento Community
Magento Community
 
rrochelle
Jr. Member
 
Total Posts:  11
Joined:  2008-08-15
 

@lisali: Thanks for your input, it worked!

@fooman: I tested your extension on a 1und1 account. It works fine there when you change the relative urls for images to absolute ones in the css files like lisali pointed out. Thanks for your great effort!

Cheers, rrochelle

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top