Magento Forum

   
Page 1 of 2
Redis vs Memcache for Cache/Session Storage? 
 
fr0x
Member
 
Total Posts:  59
Joined:  2009-05-20
 

Always see a lot of back and forth between Memcache and APC but havent seen much talk about Redis.

Noticed some talk about Magento performance and cache issues which brought up comparisons between Redis and File cache:
http://magebase.com/magento-tutorials/improving-the-file-cache-backend/

Im not sure how Redis stacks against Memcache or APC (and how those behave differently than the file cache) but since I went through the work of installing it on a test server and got it to work, figured I would lay out some steps on how to do it here.

Most of the instructions were followed from the github installation here:
https://github.com/colinmollenhour/Cm_Cache_Backend_Redis

Download and make Redis:

wget http://redis.googlecode.com/files/redis-2.4.11.tar.gz
tar xzf redis-2.4.11.tar.gz
cd redis-2.4.11
make

Move our main two redis files into the /usr/local/bin/

cd src
cp redis-server /usr/local/bin/
cp redis-cli /usr/local/bin/

Make folders to hold our our config/data info (note: 6379 is the port I am using redis on in this example)

mkdir /etc/redis
mkdir /var/redis
mkdir /var/redis/6379

Move the default init script and the default conf file

cp utils/redis_init_script /etc/init.d/redis_6379
cp redis.conf /etc/redis/6379.conf

Edit the config file

nano /etc/redis/6379.conf
set daemonize to yes
set pidfile to /var/run/redis_6379.pid
set logfile to /var/log/redis_6379.log
set dir to /var/redis/6379
set maxmemory to some value appropriate for your system (otherwise it will gobble it all up)

Install and Make phpRedis

git clone git://github.com/owlient/phpredis
cd phpredis
phpize
$ ./configure
make && make install

Make/Edit your php redis.ini file

nano /etc/php.d/redis.ini
extension
=redis.so

Get Colins Magento Redis code (this got a little messy since I didnt use his module symlinking git)

git clone git://github.com/colinmollenhour/Cm_Cache_Backend_redis.git
mv Cm_Cache_Backend_Redis/Cm/Cache/Backend/* /var/www/html/magento/app/code/community/Cm/Cache/Backend/

That unfortunately was missing the /lib/Credis/Client.php You can get the code from here:
https://raw.github.com/colinmollenhour/credis/master/Client.php
That code should go into a new folder/file called /var/www/html/magento/lib/Credis/Client.php file

With that all set, we now need to add it to our local.xml file

nano /var/www/html/magento/app/etc/local.xml
<!-- this is a child node of config/global -->
<
cache>
  <
backend>Cm_Cache_Backend_Redis</backend>
  <
backend_options>
    <
server>127.0.0.1</server> <!-- or absolute path to unix socket for better performance -->
    <
port>6379</port>
    <
database>0</database>
    <
force_standalone>0</force_standalone>  <!-- for phpredisfor standalone PHP -->
    <
automatic_cleaning_factor>0</automatic_cleaning_factor> <!-- Disabled by default -->
    <
compress_data>1</compress_data>  <!-- 0-for compression levelrecommendedor -->
    <
compress_tags>1</compress_tags>  <!-- 0-for compression levelrecommendedor -->
    <
compress_threshold>20480</compress_threshold>  <!-- Strings below this size will not be compressed -->
    <
compression_lib>gzip</compression_lib> <!-- Supports gziplzf and snappy -->
  </
backend_options>
</
cache>

Clear out your cache/session folders in /var/www/html/magento/var/cache /var/www/html/magento/var/session

Restart Apache

service httpd restart

Start up Redis

$ /etc/init.d/redis_6379 start

 
Magento Community Magento Community
Magento Community
Magento Community
 
fr0x
Member
 
Total Posts:  59
Joined:  2009-05-20
 

After doing that I did find Colin had done a module for session storage in Redis as well (no core rewrites):
https://github.com/colinmollenhour/Cm_RedisSession

git clone git://github.com/colinmollenhour/Cm_RedisSession.git
cd Cm_RedisSession
mkdir /var/www/html/magento/app/code/community/Cm/RedisSession
mv code/* /var/www/html/magento/app/code/community/Cm/RedisSession
$ cp Cm_RedisSession.xml /var/www/html/magento/app/etc/modules/

Edit the /var/www/html/magento/app/etc/local.xml and add/edit:

<config>
    <global>
        ...
        <
session_save>db</session_save>
        <
redis_session>                   <!-- All options seen here are the defaults -->
            <
host>127.0.0.1</host>            <!-- Specify an absolute path if using a unix socket -->
            <
port>6379</port>
            <
timeout>2.5</timeout>            <!-- This is the Redis connection timeoutnot the locking timeout -->
            <
db>0</db>
            <
compression_threshold>2048</compression_threshold>  <!-- Set to 0 to disable compression -->
            <
compression_lib>gzip</compression_lib>              <!-- gziplzf or snappy -->
        </
redis_session>
        ...
    </global>
    ...
</
config>

Clear out your cache

If you are using redis for your cache and need to flush it, you can do it this way:

redis-cli
redis 127.0.0.1
:6379FLUSHALL
OK

If you just want to view or flush a specific database (if you have one for cache and one for sessions), use the select and the database number.

redis-cli
redis 127.0.0.1
:6379select 0
OK
redis 127.0.0.1
:6379FLUSHDB
OK
redis 127.0.0.1
:6379keys *
(empty list or 
set)
redis 127.0.0.1:6379>

According to Colin, both the session module and the cache module can be used together.  He does suggest using different ‘db’ numbers (both the cache and the session settings in local.xml have a <db></db> setting).  The reason for this is so if you flush the cache you don’t also flush the sessions.  You could also run two instances of Redis (different ports I assume).

 
Magento Community Magento Community
Magento Community
Magento Community
 
fr0x
Member
 
Total Posts:  59
Joined:  2009-05-20
 

Making a second running redis server

Make a duplicate of your config file and change references from your 6379 port to 6380

cp /etc/redis/6379.conf /etc/redis/6380.conf
nano /etc/redis/6380.conf

Make a dump directory for your second instance

mkdir /var/redis/6380

Copy and edit the startup file and change REDISPORT=6379 to

cp /etc/init.d/redis_6379 /etc/init.d/redis_6380
nano /etc/init.d/redis_6380

Edit the /var/www/html/magento/app/etc/local.xml and add/edit the port to 6380 (assuming your cache port is something different like 6379):

<redis_session>                   <!-- All options seen here are the defaults -->
            <
host>127.0.0.1</host>            <!-- Specify an absolute path if using a unix socket -->
            <
port>6380</port>
            <
timeout>2.5</timeout>            <!-- This is the Redis connection timeoutnot the locking timeout -->
            <
db>0</db>
            <
compression_threshold>2048</compression_threshold>  <!-- Set to 0 to disable compression -->
            <
compression_lib>gzip</compression_lib>              <!-- gziplzf or snappy -->
        </
redis_session>

Startup the new redis instance

$ /etc/init.d/redis_6780 start

And clear you caches

 
Magento Community Magento Community
Magento Community
Magento Community
 
fr0x
Member
 
Total Posts:  59
Joined:  2009-05-20
 

The author of the Redis extensions, Colin Mollenhour, is giving a presentation today at Imagine about caching and speed:
“The Fast, the Slow and the Ugly: a Magento Cache Showdown”

Should be interesting to see those results.  Hopefully the full thing will be available to us non-Imagineers.

 
Magento Community Magento Community
Magento Community
Magento Community
 
tininho
Jr. Member
 
Total Posts:  14
Joined:  2009-08-12
 

Thanks for this guide! I am getting the following error though, when trying to access my site:

Fatal error: Class ‘Credis_Client’ not found in *******/app/code/community/Cm/RedisSession/Model/Session.php on line 57

What could be wrong?

 
Magento Community Magento Community
Magento Community
Magento Community
 
fr0x
Member
 
Total Posts:  59
Joined:  2009-05-20
 

That is due to this part here:

That unfortunately was missing the /lib/Credis/Client.php You can get the code from here:
https://raw.github.com/colinmollenhour/credis/master/Client.php
That code should go into a new folder/file called /var/www/html/magento/lib/Credis/Client.php file

I’m guessing if you look in your magento install under /lib/ that it is missing the Credis folder (and the Client.php file that should be in it.

For some reason the git package seemed to be missing it.  You can get it though from here:
https://raw.github.com/colinmollenhour/credis/master/Client.php

Just make a new Credis folder under /lib and then a new file in /lib/Credis called Client.php and copy that code in that URL and paste it in.

(also make sure that permissions/ownership is set properly...they should mirror other lib directory/files)

 
Magento Community Magento Community
Magento Community
Magento Community
 
agencesoon
Jr. Member
 
Avatar
Total Posts:  11
Joined:  2010-09-16
Paris, France
 

I’ve just take the time to publish a full howto about the Redis installation, configuration/tuning for both single and multi server…
Moreover the installation and configuration on Magento of the excellent work of Colin Mollenhour is also explainded here : http://blog.agence-soon.fr/magento-redis-data-cache-server/

Don’t hesitate to give me your comments and feedback !
Have a good day / regards,
Aurélien

 
Magento Community Magento Community
Magento Community
Magento Community
 
fr0x
Member
 
Total Posts:  59
Joined:  2009-05-20
 

Nice writeup.

I would suggest if you do decide to do session storage in Redis, to run a seperate instance instead of running a second db on the same redis instance as your cache.  The reason for that being, if your cache is filling and refreshing quickly, it is conceivable that it could start kicking out sessions from the Redis instance when it hits its maxmemory limit.  By seperating them, only new sessions have the possibility of kicking out old sessions and new cache will only kick out old cache.

 
Magento Community Magento Community
Magento Community
Magento Community
 
agencesoon
Jr. Member
 
Avatar
Total Posts:  11
Joined:  2010-09-16
Paris, France
 

Thanks and for your advice too. I’ll write a warning about that on the blogpost for someone who haven’t set the memorylimit to 0 on small server memory amount a:d/or large catalogue/store views.
See you soon !

 
Magento Community Magento Community
Magento Community
Magento Community
 
tininho
Jr. Member
 
Total Posts:  14
Joined:  2009-08-12
 

Thanks for help! Now the error is as following:

Fatal error: session_start(): Failed to initialize storage module: user (path: /var/www/XXXXX/www/var/session) in /var/www/XXXX/www/app/code/core/Mage/Core/Model/Session/Abstract/Varien.php on line 123

I use Litespeed as web server instead of Apache, would this have any effect on installation method?

 
Magento Community Magento Community
Magento Community
Magento Community
 
fr0x
Member
 
Total Posts:  59
Joined:  2009-05-20
 

Hmmm....I hadn’t ever run into that error before.

Is your Redis instance successfully running?  If you run ‘redis-cli’ does it connect?

[root@localhost bin]# redis-cli
redis 127.0.0.1:6379keys *
1"zc:ti:7a6_Zend_Locale"
2"zc:ti:7a6_DB_PDO_MYSQL_DDL"
3"zc:ti:7a6_MAGE"
4"zc:k:7a6_CONFIGURATION_FILES_ACCESS_LEVEL_VERIFICATION"
5"zc:k:7a6_LPASLO_STORE_1_ENTITY_16400"
6"zc:tags"
7"zc:k:7a6_CORE_CACHE_OPTIONS"
8"zc:ti:7a6_ADMINHTML"
redis 127.0.0.1:6379>

I’m assuming your keys * command will be empty but at least we’ll know redis is running.  Maybe double check your local.xml config to make sure everything is correct in there as well.

Also, are you running Redis to do your sessions?  By that error I assume so.

Are you doing it with different redis instances or the same instance with two dbs?

If you are doing it with two instances, make sure both instances are running.  You can test your other one by specifying the port.

[root@localhost bin]# redis-cli -p 6380
redis 127.0.0.1:6380keys *
 
1"sess_sge77k9i96bfpj80pc6ai3m4r2"
 
2"sess_4kf869jr2acl34hd08v5ul8j67"
 
3"sess_im70bn3p4thkk2bq3g90is1ac1"
 
4"sess_5alkvdqp8mcbslcd1cnntcft16"
 
5"sess_mr6eujq8cadv0pa73jelu5dfs4"
 
6"sess_df8dvpekktu5col4vebues3ue3"
 
7"sess_2bulge7imv8nptubb7mdkauhs7"
redis 127.0.0.1:6380>

 
Magento Community Magento Community
Magento Community
Magento Community
 
fr0x
Member
 
Total Posts:  59
Joined:  2009-05-20
 

Also, I’m not sure which versions this will work in

I had it running in 1.6.2 without issue.

I tried it on a 1.3.x install and it would not work.  Basically the getCache() function in /app/code/core/Mage/Core/Model/App.php has changed quite a bit inbetween those versions.  In the 1.3.x build, that function has an if/else statement that will only accept ‘apc’, ‘eaccelerator’, or ‘memcached’.  If your cache doesn’t match one of those three, it defaults to file cache.  So in 1.3 with Redis installed, it basically ignored my Redis statements in local.xml and defaulted to file cache.

Not sure how 1.4 / 1.5 handles.

 
Magento Community Magento Community
Magento Community
Magento Community
 
tininho
Jr. Member
 
Total Posts:  14
Joined:  2009-08-12
 

Despite your much appreciated efforts to try to help me I cannot get this to work. Magento error report:

a:5:{i:0;s:127:"Cannot send headers; headers already sent in /var/www/******/www/app/code/community/Cm/Cache/Backend/Redis.php, line 1461";i:1;s:942:"#0 /var/www/******/www/lib/Zend/Controller/Response/Abstract.php(115): Zend_Controller_Response_Abstract->canSendHeaders(true)
#1 /var/www/******/www/app/code/core/Mage/Core/Model/App.php(1246): Zend_Controller_Response_Abstract->setHeader(’Content-Type’, ‘text/html; char...’)
#2 /var/www/******/www/app/code/core/Mage/Core/Controller/Varien/Front.php(80): Mage_Core_Model_App->getResponse()
#3 /var/www/******/www/app/code/core/Mage/Core/Model/Url/Rewrite.php(204): Mage_Core_Controller_Varien_Front->getResponse()
#4 /var/www/******/www/app/code/core/Mage/Core/Controller/Varien/Front.php(166): Mage_Core_Model_Url_Rewrite->rewrite()
#5 /var/www/******/www/app/code/core/Mage/Core/Model/App.php(354): Mage_Core_Controller_Varien_Front->dispatch()
#6 /var/www/******/www/app/Mage.php(683): Mage_Core_Model_App->run(Array)
#7 /var/www/******/www/index.php(87): Mage::run(’’, ‘store’)
#8 {main}”;s:3:"url”;s:15:"/opensearch.xml”;s:11:"script_name”;s:10:"/index.php”;s:4:"skin”;s:7:"english";}

Phpinfo reports:
redis Redis Support enabled
Redis Version 2.1.0

virtual@virtual273:~/sources$ redis-cli
redis 127.0.0.1:6379> keys *
(empty list or set)

local.xml is edited as instructed on your first message (direct copy paste). What do you think is causing this, as there is clearly no connection to redis even though the folders and all has been set..

 
Magento Community Magento Community
Magento Community
Magento Community
 
fr0x
Member
 
Total Posts:  59
Joined:  2009-05-20
 

There is quite a bit out there regarding that error (Google: “Cannot send headers; headers already sent “).

The most common thing seems to be some “extra” characters beyond the closing brackets or at the top of the file.  Maybe open up Redis.php, make sure there is nothing past the closing bracket and no extra space at the top and resave out the file.  Make sure to also clear out whatever you current cache is (shutdown your memcache instance or clear out /var/www/html/magento/var/cache).

Also, if you cut and pasted in the code for lib/Credis/Client.php, definitely double check that file for any extra white space at top or past the final closing tags.

It still could have something to do with lightspeed instead of apache but every issue I’m reading about “Cannot send headers; headers already sent” seem to almost always point to some corrupt data in a file that needs to be removed and re-saved back out.

This person had a similar issue (although not Redis specific):

Thank you Julien and Mathew. I feel embarrassing. It is clear that the white space is included before the PHP tag. It is my bad habit of cut and paste. Thank you guys again for speedy reply.

And for someone may encouter this, please notice the white space before the PHP tag

----- begin code --------
<----- 
this is where the white space included before the php tag 
<?php

class ServicesController extends Zend_Controller_Action {

public function indexAction() { $this->view->title " | News ::index"}
public function b1Action() { $this->view->title " | News ::b1"$this->render('index'); }
public function b2Action() { $this->view->title " | News ::b2"$this->render('index'); }
// end class

------ end code ----

 
Magento Community Magento Community
Magento Community
Magento Community
 
fr0x
Member
 
Total Posts:  59
Joined:  2009-05-20
 

Not sure if you are still watching this thread or if you ever got your issue solved but I did run across this page for getting Redis/Magento/Lightspeed combo to work:

http://www.drewgillson.com/blog/customize-tinybricks-lightspeed-for-use-with-redis/

 
Magento Community Magento Community
Magento Community
Magento Community
 
NMediaSystems
Member
 
Avatar
Total Posts:  56
Joined:  2010-02-18
London
 

In addition to caching solutions developed for Magento, we’ve built Magento Cache Warmer, which irregardless of what caching you have in place, will build up missing cache for your pages, more information about the extension http://www.nmediasystems.com/services/magento-ecommerce-software/extensions/magento-cache-warm-up.html

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
    Back to top
Page 1 of 2