nexcess_net

Turpentine - Varnish Cache

  • Popularity Score 4997 Popularity Score

    Popularity ScoreReferrals to Site/Downloads

    0 0 1+ 1+ 250+ 250+ 1000+ 1000+ 10000+ 10000+

Simplify Varnish management and deployment with Magento.
Compatible with:
This extension is currently unavailable on Magento Connect. Please contact the developer.

You will be re-directed to the developer's website to complete your purchase.

You must be registered and logged in to get extension key.
In order to get this extension, you must be logged in to the Magento Community. Click here to login or register.
In order to get this extension, you must have active Magento Go Store. Click here to create Go store.

Magento Connect 2.0

Magento Connect 2.0

FREE
Previous Next
Overview

Turpentine - Varnish Cache

Nexcess.net Turpentine Extension for Magento

Turpentine is a full page cache extension for Magento that works with Varnish, a very fast caching reverse-proxy. By default, Varnish doesn't cache requests with cookies and Magento sends the frontend cookie with every request causing a (near) zero hit-rate for Varnish's cache. Turpentine configures Varnish to work with Magento and modifies Magento's behaviour to significantly improve the cache hit rate.

Note that while this extension is now considered stable, it is strongly recommended that it be tested on a development/staging site before deploying on a production site due to the potential need to add custom ESI policies for blocks added by other extensions.

Features

  • Full Page Caching, with hole-punching via Varnish ESI and/or AJAX, even for logged in visitors
  • Configurable via standard Magento methods (Admin system configuration and layout XML), no manual editing of Varnish config required for most cases
  • Able to generate and apply new Varnish VCLs (configurations) on the fly, without restarting/changing Varnish's config files or flushing the cache
  • Blacklist requests from caching by URL or parameters (SID, store, etc)
  • Configure cache TTL by URL and individual block's TTL
  • Supports multiple Varnish instances for clustered usage
  • Automatic cache clearing on actions (clearing product/catalog/cms page after saving)
  • Supports non-root Magento installs (i.e. putting Magento in /store/ instead of /) and multi-store/multi-site setups
  • Support for site-crawlers for cache warming, and includes a (basic) built-in site-crawler
  • SSL support through Pound or Nginx

Requirements

  • Magento Community Edition 1.6+ or Magento Enterprise Edition 1.11+
  • Varnish 2.1+ (including 3.0+)

Installation & Usage

See the Installation and Usage pages.

Support

If you have an issue, please read the FAQ then if you still need help, open a bug report in GitHub's issue tracker.

Please do not use Magento Connect's Reviews or (especially) the Q&A for support. There isn't a way for me to reply to reviews and the Q&A moderation is very slow.

Contributing

If you have a fix or feature for Turpentine, submit a pull request through GitHub to the devel branch. The master branch is only for stable releases. Please make sure the new code follows the same style and conventions as already written code.

How it works

The extension works in two parts, page caching and block (ESI/AJAX) caching. A simplified look at how they work:

For pages, Varnish first checks whether the visitor sent a frontend cookie. If they didn't, then Varnish will generate a new session token for them. The page is then served from cache (or fetched from the backend if it's not already in the cache), with any blocks with ESI polices filled in via ESI. Note that the cookie checking is bypassed for clients identified as crawlers (see the Crawler IP Addresses and Crawler User Agents settings).

For blocks, the extension listens for the core_block_abstract_to_html_before event in Magento. When this event is triggered, the extension looks at the block attached to it and if an ESI policy has been defined for the block then the block's template is replaced with a simple ESI (or AJAX) template that tells Varnish to pull the block content from a separate URL. Varnish then does another request to that URL to get the content for that block, which can be cached separately from the page and may differ between different visitors/clients.

Notes and Caveats

  • Turpentine will not help (directly) with the speed of "actions" like adding things to the cart or checking out. It only caches, so it can only speed up page load speed for site browsing. It will remove a lot of load on the backend though so for heavily loaded sites it can free up enough backend resources to have a noticeable effect on "actions".
  • There are some technical limitations when using Varnish 2.1.x: External ESI requests are not blocked, and per-block TTLs in ESI policies are not honored (all blocks use the default TTL)
  • The core parts of Turpentine (caching and ESI/AJAX injection) work under Magento CE 1.5, but a significant portion of the auxillary functionality doesn't work due to changes to event names. That said, it would be possible to use Turpentine with Magento CE 1.5 with an understanding that it is not supported and what actions need to be taken manually. Both cache flushing (both automatic an manual) and cache warming (due to missing events that trigger the cache flushing) do not work.
  • Anonymous blocks are not able to be hole-punched. For CMS pages, it is recommended that you include the block in the page's layout updates XML and give it a name, then it can have an ESI policy like normal

Known Issues

  • Logging and statistics will show all requests as coming from the same IP address (usually localhost/127.0.0.1). It should be possible to work around this using Apache's mod_remoteip or mod_rpaf.

Demo

See the Demo Sites wiki page.

If you use Turpentine (on a production site), feel free to add your site to the list!

License

The code is licensed under GPLv2+, much of the ESI-specific code is taken from Hugues Alary's Magento-Varnish extension, which is licensed as GPLv3.

Release Notes

RELEASE-0.5.1

2013-03-06 19:21:54
  • Version number: 0.5.1
  • Stability: Stable
  • Compatibility: 1.6, 1.6.1, 1.6.2.0, 1.7, 1.11, 1.11.1, 1.12
You must be registered and logged in to get extension key.

Magento Connect 2.0

Magento Connect 2.0

RELEASE-0.5.1

This is the first release of Turpentine marked "stable" in Magento Connect

  • Fix PHP error when Block Logging is turned on
  • Saving a child product of a configurable or grouped product will now also
    cause the parent product(s) to be flushed. This makes saving consistent with
    the out-of-stock flushing behaviour
  • Varnish will now properly distinguish between HTTP and HTTPS responses. This
    should fix warnings in some browsers about mixing secure and insecure content
    when using Varnish+Turpentine behind an SSL proxy

RELEASE-0.5.0

2013-03-01 20:13:05
  • Version number: 0.5.0
  • Stability: Beta
  • Compatibility: 1.6, 1.6.1, 1.6.2.0, 1.7, 1.11, 1.11.1, 1.12
You must be registered and logged in to get extension key.

Magento Connect 2.0

Magento Connect 2.0

RELEASE-0.5.0

There are changes to the ESI layout syntax (again) in this release, any
customizations that have been done to the ESI layout or added for custom blocks
will need to be updated and moved to the layout.xml file. Additionally, the
Varnish cache will need to be fully flushed or cached pages that reference
previously cached ESI blocks will not load those blocks

  • Added support for event-based partial ESI cache flushing. ESI blocks will
    now only be flushed on the events specified for them in the layout rather than
    all of a client's ESI blocks being flushed on any event that triggered a flush
  • Removed the purge_events option as it was no longer needed with the new
    partial ESI flushing support
  • Removed the need for initial request pass through, Varnish will now generate
    a session token for new visitors and serve them pages from the cache on their
    first visit
  • Fixed CMS pages not being automatically flushed on update
  • Fixed piped requests in Varnish not having headers handled appropriately
  • Added ability to whitelist IP addresses for debug info from Varnish, even
    if Varnish debugging is disabled via Magento's developer IP setting
  • Added flushing on a product page, product review list, and individual review
    view on review saving (such as changing from 'Pending' to 'Approved')
  • Combined the Varnish and ESI debug options
  • Made the logging done by the extension consistent, all messages logged by
    Turpentine will be prefixed with TURPENTINE:
  • Added HMAC signing to the encrypted ESI data to prevent possible
    tampering with the ESI data in the request
  • The turpentine_esi_custom.xml file that was previously suggested to be
    used for custom ESI policies has been removed in favor of Magento's built-in
    facility for this (local.xml). Any ESI policies defined in turpentine_esi_custom.xml
    should be moved the local.xml file in the theme's layout directory to
    prevent issues with the order of defining blocks and setting ESI policies on
    them.

RELEASE-0.4.0

2013-02-13 18:58:53
  • Version number: 0.4.0
  • Stability: Beta
  • Compatibility: 1.6, 1.6.1, 1.6.2.0, 1.7, 1.11, 1.11.1, 1.12
You must be registered and logged in to get extension key.

Magento Connect 2.0

Magento Connect 2.0

RELEASE-0.4.0

There are changes to the ESI layout syntax in this release, any customizations
that have been done to the ESI layout will need to be updated

  • Improved serialization of ESI/AJAX data in URLs. ESI/AJAX URLs should now be
    50-75% as long as they previously were
  • Fixed rare cases of invalid characters in ESI/AJAX URLs, which would cause
    the block to not load or display an error message
  • Unified AJAX and ESI code paths. ESI and AJAX includes no longer use
    duplicated code which will make it easier to add other other inclusion methods
    in the future and reduces the potential for bugs
  • Selecting between AJAX and ESI inclusion is now done with the method
    parameter to setEsiOptions, setAjaxOptions is no longer supported
  • Added ability to cache AJAX requests
  • The old cacheType parameter for setEsiOptions in the layout has been
    split into two new options: scope and access<ul>
  • scope = global: the cached object valid for the entire site
  • scope = page: the cached object is specific to a single page
  • access = public: the cached object is valid for any visitor
  • access = private: the cached object is valid for a single visitor's session
</li>
  • ESI block default TTL options have been removed. The new default for public
    ESI blocks is the same as the Varnish page TTL and private blocks use the
    session cookie lifetime as the TTL
  • Turpentine will now detect if there is a difference between how a block
    should be loaded vs how it is requested (ex. manually changing the method
    parameter for a AJAX block to ESI). If ESI debugging is turned off an error
    (403) will be returned
  • Client-side caching will now be used for static assets, in addition to
    Varnish's regular caching (via the Cache-Control header)
  • Improved page rendering performance. Pages that include ESI blocks will now
    render significantly faster (up to 50% in testing), dependant on how many
    ESI block includes were on the page
  • Fixed Turpentine ESI request entries in the Magento visitor log. This should
    result in a small speed-up for AJAX requests particularly as most of the time
    token for the request was spent doing the logging
  • Added custom VCL inclusion. Custom logic can be added to the VCL Turpentine
    generates by putting it in
    app/code/community/Nexcessnet/Turpentine/misc/custom_include.vcl. If this
    file is found, Turpentine will automatically include it. Note that it is not
    Varnish version-specific like the other templates, users will need to make sure
    the VCL code used is compatible with the version of Varnish that they use
  • </ul>

    RELEASE-0.3.2

    2013-01-28 16:19:18
    • Version number: 0.3.2
    • Stability: Beta
    • Compatibility: 1.6, 1.6.1, 1.6.2.0, 1.7, 1.11, 1.11.1, 1.12
    You must be registered and logged in to get extension key.

    Magento Connect 2.0

    Magento Connect 2.0

    RELEASE-0.3.2

    • Fixed Turpentine not correctly detecting and disabling ESI/AJAX on requests
      with HTTP Authorization
    • Fixed some requests being passed through Varnish instead of piped resulting
      in incorrect handling of response output
    • Removed cached object TTL extension (added in RELEASE-0.2.0) due to issues
      with cached object contention at high load
    • Improved warm-cache.sh script to handle different platforms better and
      take advantage of multiple processors/cores
    • Added advanced session validation. Varnish will now respect the Magento settings
      for validating the User-Agent, X-Forwarded-For, and Via headers, and the remote
      IP address with respect to sessions (under System &gt; Configuration &gt; General &gt;
      Web &gt; Session Validation Settings).

    RELEASE-0.3.1

    2013-01-18 21:33:25
    • Version number: 0.3.1
    • Stability: Beta
    • Compatibility: 1.6, 1.6.1, 1.6.2.0, 1.7, 1.11, 1.11.1, 1.12
    You must be registered and logged in to get extension key.

    Magento Connect 2.0

    Magento Connect 2.0

    RELEASE-0.3.1

    • Fixed generated VCL to work with multi-store/multi-site setups with different
      URL paths
    Reviews

    About the Developer

    This extension was developed by and is supported by nexcess_net

    Turpentine - Varnish Cache 4.6 5 20 20
    It works perfect in 1.7.0.2 (multistore and multidomain) It works perfect in 1.7.0.2 (multistore and multidomain), after modifying the vcl to avoid the mistake of dirty cookie "double cookie frontend" If you solve the problem of double cookie, without needing to have to call our technicians, would be to give it a 5, for now I leave in 4 * January 31, 2014
    Huge speed improvement but After finally getting it working, I noticed a massive speed increase on the website. But we are using 1.8.1.0 and found it has a problem with the cart. If you are using 1.8.0 or 1.8.1 don't use this yet until it has been updated. Or customers often wont be able to add items to the cart. January 29, 2014
    Great speed improvement I got great speed improvement on my server using Varnish, it was a bit hard to set-up at the beginning but then I made it work. In the configuration backend port needs to be 8080 for Apache and Varnish needs to listen at port 80 in front. This is a sample configuration that works for me. Great piece of software for Magento ! October 28, 2013
    Superb Varnish can be a real pain, particularly with ecom sites, but with a little nous this module works really really well, and we've seen massive page speed increases - particular on HA multi server environments. That said, if doesn't negate the need for some prior knowledge of varnish and your web server of choice. Big thanks to Nexcess for making it freely available! October 11, 2013
    Turpentine - Varnish Cache - mature product. Today this is the only cache you can use to lower your server load to 0%. Easy install, easy to configure and maintain. No errors whatsoever. You need some magento layout knowledge to configure it with highly customized theme, but it works in 95% out of the box. August 12, 2013
    Works great but... I'm not seeing any headers in the pages - does the extension remove them. Also the cart in the header is not updated when something is added - how do I fix this? August 7, 2013
    why it happens? when i install it through magento connect manager, it returns: Checking dependencies of packages CONNECT ERROR: Unsupported resource type July 4, 2013
    Super!!!! Hats off for the people who made this extension possible.... June 12, 2013
    Loading Time I don't know why the loading time need around 5 seconds then can be opening the webpage. May I to know how to fix this problem, Thank you :) June 5, 2013
    Holy cow zoom. zoom. zoom. amazing. we've been using the phoenix varnish extension for ages. this blows it away. April 13, 2013
    2 next>>
    Q&A
    (10 Questions : 5 Answers)

    Product Q&A

    Ask your questions. Share your answers.

     
     
    « Back
     
     
    Sort by
     
    2 answers

    1.8.1.0 Compatibility

    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_27347772
    Customer avatar
    Does this work with CE 1.8.1.0?
    by
    sml12
    -
    9 months ago
    0points
    0out of 0found this question helpful.
    Was this helpful?Yes0No0

    Answers

    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_27454397
    Customer avatar
    Answer: 
    Version 0.6.0 upwards works with 1.8+. This version fixes an incompatibility with the form_key CSRF protection. You'll need to add -p esi_syntax=0x2 to your Varnish startup options.
    by
    multicoptersltd
    -
    3 months, 4 weeks ago
    +1point
    1out of 1found this answer helpful.
    Was this helpful?Yes1No0

    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_404763
    Customer avatar
    Answer: 
    From what I've gathered in my test environment, it is not. Looks like Varnish itself has some compatibility issues with 1.8.X and Turpentine goes right along with it. I can't get this working at all since the upgrade.
    by
    barproducts
    -
    4 months, 4 weeks ago
    +1point
    1out of 1found this answer helpful.
    Was this helpful?Yes1No0
    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_26234457
    Customer avatar
    After I enable Varnish and complete the install I am no longer able to view GET timthumb.php images for my homepage. Timthumb.php brings in several images for my slideshow and product slider. Anyone have any experience with getting timthumb.php images to work after enabling varnish on port 80 and setting up the turpentine plugin?

    Cody
    by
    Betsworth22
    -
    1 year, 6 months ago
    0points
    0out of 0found this question helpful.
    Was this helpful?Yes0No0

    Answers

    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_26268542
    Customer avatar
    Answer: 
    Hi, im no expert but we managed to also install varnish and turpentine, the way i understand how it works is that varnish server is serving cached content. If timthumb.php serves dynamic images then varnish will show only the cached version. You have to create an xml block for that file. Then edit the turpentine_esi.xml and add that block with an esi parameter to tell varnish not to cache that block. When a the page is called, varnish will pull the block from the webserver and combine it with the cached version to serve the complete page. Hope this helps.
    by
    jpadua
    -
    1 year, 6 months ago
    0points
    0out of 0found this answer helpful.
    Was this helpful?Yes0No0
    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_371050
    Customer avatar
    Hi, I don't know where and how to configure Varnish.
    by
    jaautomacao
    -
    1 year, 2 months ago
    0points
    0out of 0found this question helpful.
    Was this helpful?Yes0No0

    Answers

    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_27405268
    Customer avatar
    Answer: 
    Hi,

    After installation go to your Configuration panel.
    ( System > Configuration ).

    In left panel there has to be a "Turpentine" option list. And in that list you can see Varnish and Caching options pages
    by
    semihmasat
    -
    8 months ago
    +1point
    1out of 1found this answer helpful.
    Was this helpful?Yes1No0
    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_105580
    Customer avatar
    Does this extension work with Magento 1.9.0.1?
    by
    Kunzi
    -
    1 week, 5 days ago
    0points
    0out of 0found this question helpful.
    Was this helpful?Yes0No0

    Answers

    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_26873850
    Customer avatar
    Answer: 
    BUMP! - Has this developer Varnished? lol.

    When is an update available for 1.9.1 ?????
    by
    scribblevision
    -
    2 days ago
    0points
    0out of 0found this answer helpful.
    Was this helpful?Yes0No0
     
    0 answers

    Tracking

    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_25884151
    Customer avatar
    Since we installed Turpentine Varnish our E-Commerce Tracking does not work anymore.

    Is this a known problem?

    Thank you for the great work!

    Best regards
    Daniel Meyer
    by
    visuwerkdm
    -
    10 months ago
    0points
    0out of 0found this question helpful.
    Was this helpful?Yes0No0
    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_27401563
    Customer avatar
    How do you integrate turpentine vcl with Unixy?
    by
    jasonnovember
    -
    8 months ago
    0points
    0out of 0found this question helpful.
    Was this helpful?Yes0No0
     
    0 answers

    Success on ajax call

    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_392532
    Customer avatar
    Hey! Great extension! Thank you!
    Is there a way to call a jquery function on an object loaded with turpentine through ajax after it finished loading? ( similar to "success" from jquery ajax)
    by
    ddevil
    -
    6 months ago
    0points
    0out of 0found this question helpful.
    Was this helpful?Yes0No0
    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_26155438
    Customer avatar
    by
    separaatcom
    -
    5 months, 1 week ago
    0points
    0out of 0found this question helpful.
    Was this helpful?Yes0No0
    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_25933365
    Customer avatar
    HI,

    How to solve this problem? On product click

    ESI processing not enabled

    Varnish 3
    Magento 1.7.0.2
    by
    brunomenol
    -
    1 month, 1 week ago
    0points
    0out of 0found this question helpful.
    Was this helpful?Yes0No0
    Avatar Image Url:: http://www.magentocommerce.com/images/avatars/uploads/avatar_25987036
    Customer avatar
    Hi, How can I configure Varnish to see my site from corporates networks (no direct internet connection)?
    Caching behavior is not good, no privacy between sessions.
    by
    PabloCalvo
    -
    2 weeks, 2 days ago
    0points
    0out of 0found this question helpful.
    Was this helpful?Yes0No0
    1 of 1
    1 of 1
    Get Help

    Support for This Extension

    The best place to start if you need help with a specific extension is to contact the developer. All Magento developers have both a contact email and a support email listed.

    Magento Platform Support

    If you need support for a Magento platform, there are different options for support depending on which Magento platform you are using. Below are links for specific platforms.

    In order to upload extension, you must be logged in to the Magento Community. Click here to login or register.

    Close

    Register

    * Required Fields

    Close window

    Forgot Your Password?

    Please enter your email below and we'll send you a new password.

    * Required Fields

    Close window

    Login

    To upload extension you must be logged in.

    * Required Fields

    Close window

    You are using an outdated browser

    We built Magento Connect using the latest techniques and technologies.
    This makes Magento Connect faster and easier to use.
    Unfortunately, your browser doesn't support those technologies.
    Use the links below to download a new browser or upgrade your existing browser.