Magento

I’ve been working with Magento in a live production environment every since it went public.  It’s a fantastic product and I’ve been very happy with it in all aspects, but I have not found a good policy/procedure for deployment and code management.  I thought I would look to the community and/or some of the core developers for insight and advise on this topic.

I use http://www.beanstalkapp.com for svn repository and plan to start using it’s ftp deployment features (currently I use rsync to deploy). We do a lot of custom code and I place all of that custom core code in app/code/local to avoid losing code on any upgrades, and I use three different environments to help with stability - development, test and production environment. 

However, my main concerns and questions are with the following:

1.  When upgrading Magento is it important to go through the same upgrading process for each separate environment - development, test and production environments?  What’s a good policy/procedure for the upgrading process?

I assume that upgrading Magento could possible change both the core code and the database (and also some theme code depending on the upgrade).  If this is the case then I believe the answer to the question above would be “yes”. 

That being said, what part of the app do I need to keep my programmers and deployment tools away from?  Is this just the core code?  Should I place the entire app under my own svn repository and still use the magento upgrade process in each environment to upgrade?

2. I have similar questions and concerns when installing extensions in magento.  Is this process very similar to upgrading Magento?

3. How do I manage the config files - local.xml for each environment? I assume this may also be similar to upgrading Magento in that each environment may need to initially be setup separately and manually and then upgraded from the repository without changing those config files.  Are there any other config files that I need to be careful not to change on a deploy?

The main question at hand here is how do we properly manage code and the deployment process with multiple environments without messing up core code, the upgrading process and extensions.

I’ve had several people ask me if we ever came up with a solid process, so I thought I would share what we do.  It is in no way the right way...in fact, in might be wrong, but it works great for us. It is simple and effective for us.

Our main goal while defining this process was to stay out of Magento core code as much as possible and let it “do it’s thing” on upgrading and management of magento code and extensions.

We ended up going this route:

We use svn checkout to install the magneto code base (whether that is a development, test or production environment).  We use svn (I like to use http://beanstalkapp.com/) to manage our own custom code, and finally we use rsync to install and update our custom code (you can read about rsync here: http://samba.anu.edu.au/rsync/) . Our rsync script ‘syncs’ the directories and files in the appropriate place only if the code as been checked in to our repository. 

Our repository only holds our custom template (skin and template code), our custom code (both Mage core to override and custom to our store), and our custom local.xml (config for database, cron jobs ..etc).  And the rysnc script places everything where in needs to be.

We use magento connect to upgrade and we use the magento upgrade process on all of our environments (dev, test and production).  We treat extensions the same way and test them thoroughly before installing them to production. 

We also make sure to check any changes and merges on the template themes we are using just in case there have been major changes that might break our templates. 

If you all would like more details....just ask.

hope this was helpful.

I am new to svn. I actually made a thread asking for how people use it just last week. Here is the svn structure I am going with. Maybe you will have some pointers for me if you see any errors:

Here are the main folders. A description of the subfolders are given:

branches/[branch_name]/src - where all the source code goes
branches/[branch_name]/sql - the base database structure / sample data needed by someone who checks out to use
branches/[branch_name]/scripts - any scripts that will need to be ran on the main database when this is merged with trunk.

trunk/src - where all the source code goes
trunk/sql - the current database structure / sample data needed by someone who checks out to use
trunk/scripts - any scripts that will be needed to upgrade the live database to look like trunk

tags/[version]/src - where all the source code goes
tags/[version]/sql - the current database structure / sample data needed by someone who checks out to use
tags/[version]/scripts - any scripts that will be needed to upgrade the live database to look like this release version

vendors/magento/trunk
vendors/magento/tags/[version]

Upgrading live server to latest

1) copy the trunk to a branch and version it (2.1 for example)
2) Copy the branch 2.1 and tag it (2.1.1 for example)
3) Checkout the tag version, install the database and sample data to test database
4) Take down live web site
5) Compare database structures and update live database structure
6) On production server export 2.1.1 to folder
7) Run any /scripts/ that need to be ran to complete database upgrade
8) rsync to the main web site ignore cache / media/ config.xml
9) delete cache folder
10) Put live site up

Upgrading Magento

1) Copy the latest changes to folder
2) Run svn_load_dirs.pl on folder and /trunk/
3) Copy trunk to /vendor/magento/tags/[version] (lets say version 1.1.8)
4) Merge the changes between latest tag (1.1.8) and last tag(1.1.7) into trunk
5) Take care of any conflicts

yeah...looks good.  I think I have seen similar strategies in the wiki.  I’ll look over it more in detail later.

Again though, we’ve chosen to take a much simpler route and we have had much success with our upgrades and installations of extensions.

I have just started to use Magento and was contemplating this very issue. I have started to use a product called SnapshotCM (http://www.truebluesoftware.com) for version control. It has many advantages over subversion and is free as long as only one client is using it. Best feature of all is that it uses a ‘tree-like’ graph to represent all of your relationships between trunks (snapshots). Give it a try you might like it better than subversion, I do.

Would be good for me to learn about this. I am not a developer but I like to try to do things myself, even though have no knowledge of php. I look in forums for the code I might need and try to make it work for me. Therefore a good code management habit would save me in case of disaster. I have subscribed to beanstalkapp.com and downloaded version for mac to go with it. Now what’s next? As far as I have understood I don’t need to copy the all magento installation to beanstalkapp.com but only my changes. So the changes I make to the standard magento installation would be contained in the app/code/local folder on my server. There I would copy the part of the magento core code I want to modify. I would then create the same folder (and structure) on beanstalkapp.com, containing the same code as repository. I would download the code locally from beanstalkapp.com (with versions???) and start to make the changes there. I would then test the changes by uploading the new files to the server in the app/core/local directory. If things go well then I upload my revision to beanstalkapp.com. If things go wrong I will get the original code from beanstalkapp.com and replace the mess I made on the server. Is this the basic idea? Not sure if I have get it right in simple words. I am not much technician in this, sorry.

@ jimmypalmer - thanks for this info. We are basically doing the same thing. Just as a sanity check, I’ve attached a screengrab of the folders I’ve got under SVN.

On an equally important note, what are you doing wrt test vs. production data? Are you guys pushing snapshots of production back down into test and integration? If so, what are you using?

Thanks!

Chad

Click thumbnail to see full-size image

@jimmypalmer - I would love to know more about your svn process. I to want to use a service like beanstalk.

This is what I am personally after.

Maintain dev, staging and production environments.

Dev version will be on my local machine, the other two would be online.

Only keep themes under version control.
Right now I have no interest in keeping the core under subversion control, but I might be convinced otherwise if their was a benefit.

Use a service like beanstalk or springloops. I know springloops lets you deploy your code to any server and keeps all the .svn files out of your production envirionment.

Use Versions for the Mac.

Would love any feedback.

I’ve written down our solution of deployment with Magento (in german)
Perhaps it is useful for some of you....
http://blog.tudock.de/startseite/beitrag/2010/09/17/deployment-prozess-eines-magento-shops.html

We\\\’ve developed Magento deployment suite which helps to automate following steps:

branching / tagging code base for next release
code release deployment to application servers
encoding / precompiling certain application components if necessary
change control and diff code review
opcode & objects cache flush

If your organization faces the same challenge, our Magento Deployment Suite is available at http://www.nmediasystems.com/services/magento-ecommerce-software/magento-deployment-suite.html