Wayfair Tech Blog

Wayfair Code Deployment, Part 2

push_screenshot

The architecture of the deployment system

In the last post I explained our deployment system goals and the basic architecture.

Now that stage is set, let's get into the details of the architecture from the deployment server perspective. As we designed the deployment server we wanted to keep the interface and the logic and requirements of the server itself as simple as possible.

The deployment server is running on our standard FLAMP stack. ( FreeBSD, Lighttpd, APC, MSSQL/MySQL/Memcache, PHP). We'll save more details on those decisions for a later post.

On the deployment server there is a configuration file that defines each application and the required details to deploy it. It defines the basic things like friendly name, allowed code reviewers, repository name and repository folders to be included.

To deploy an application you choose it from a drop down list, define an SVN revision to push and hit deploy. The deployment servers only job is to do an SVN export of the defined code, tar it up, gzip it, and set the applications current revision number to the deployed version number.

push_screenshot
Screen Shot of Deployment Tool interface

 

Once that is done it's up to the deployment clients to fetch the code and do anything application specific required for the deployment. Each webserver has a cron job that runs every minute to check for new deployments. It makes an HTTP request to the deployment server with the current revision of its local code. A client must be pre-registered with the deployment system with a deployment stage (Dev/Staging/Prod) and an application.

Once registered the next client request will fetch the current revision number for the deployment stage and application combination.

The client checks to see if the returned number is different from its own current version.

If its not, it has nothing to do. If it is either higher or lower, it checks to see if it has that code revision already on the server. If not it downloads the gzipped tarball and extracts it into a folder named by the revision number. This allows for quick roll backs since we keep the last X revisions per application locally. Where X is a per application setting.

Once the new code is deployed it writes out a text file locally with the currently deployed revision and makes an HTTP request back to the deployment server with the deployment results.

There is a watchdog process on the deployment server that checks to make sure that all registered servers to an application have updated successfully within 2 minutes of a deployment. An email is sent out to the developer only if it meets the SLA , and it sends an email to the developer and systems group if it is out side of the SLA.

OutofCompliance
Example UnHappy Code Push

 

HappyPush
Example Happy Code Push

 

This gives you the foundation for how the Deployment system is architected and how the deployment server and clients interact from the deployment server perspective. In the next part we are going to discuss the client side of the deployment system. We'll discuss our "symlink-less symlink" deployment method and secret sauce that goes with it.

Share