jordillonch/deploy-bundle

A Symfony2 deploy bundle

Symfony2 Deploy Bundle

WIP, _(*1)

, _(*2)

What it is?

This bundle aims to be a deploy system for your projects., _(*3)

It is a Symfony2 Bundle but it can be used to deploy several kind of projects., _(*4)

The bundle provides some commands to automatize deploy process. Here are main commands:, _(*5)

• Initialize: Prepare deployer and remote servers creating a directories structure to host new code.
• Download: Download code from repository, adapt, warn up… and ship it to remote servers in order to put new code to production.
• Code to production. Deploy new code to production atomically and reload web server, app…
• Rollback. Return back to previous deployed version.

Deployer have zones configured to deploy new code., _(*6)

Zones are a project and environment (e.g. prod_api, our project Api for the production environment)., _(*7)

Deployer uses GitHub repository, a configured branch, and HEAD as a target to deploy., _(*8)

You can use this bundle adding it to your projects via composer (see installation section) but my recommendation is that you create a new project to deploy because you may be want to not have your production configurations in your repository project. So it is a good idea to delegate add productions configuration to the deploy system., _(*9)

Furthermore this bundle offers helpers to automatize common operations like install composer dependencies, do a cache warm up for Symfony2 projects, refresh php-fpm after put code to production, get url to GitHub with diffs of new deployed code..., _(*10)

How it works?

There are two basic ideas that allows to deploy several code versions and put one of them to production and rollback between them., _(*11)

When you want to deploy new code you have to do a "download" operation. That operation clones code from your git repository to a new directory (e.g. 20130704_180131_a618a56b08549794ec4c9d5db29058a01a58977f) then do adaptations to the code. Adaptations are the step where configurations are added, app cache is warned up, dependencies are downloaded and installed, symlinks are created to shared directories…, _(*12)

Shared directories are directories where there are data that you want to keep between deploys. (e.g. logs, reports, generated images...), _(*13)

After that, code is copied to configured servers. Ssh authorized keys are used to allow copy and execute commands to remote servers., _(*14)

Then, when you want to use last downloaded code to production you have to execute "code to production" operation. This operation modify a symlink to the directory where last version are downloaded., _(*15)

Usually, after that you should restart webserver or php-fpm., _(*16)

Note: These ideas are taken from Capistrano., _(*17)

Quick start

Create a new Symfony 2.3 project

php composer.phar create-project symfony/framework-standard-edition path/ 2.3.0

Install the bundle

Add following lines to your composer.json file:, _(*18)

"require": {
  "jordillonch/deploy-bundle": "dev-master"
},
"minimum-stability": "dev",

Execute:, _(*19)

php composer.phar update

Add it to the AppKernel.php class:, _(*20)

new JordiLlonch\Bundle\DeployBundle\JordiLlonchDeployBundle(),

Configure general settings and a zone

app/config/parameters.yml, _(*21)

jordi_llonch_deploy:
    config:
        project: MyProject
        vcs: git
        servers_parameter_file: app/config/parameters_deployer_servers.yml
        local_repository_dir: /home/deploy/local_repository
        clean_max_deploys: 7
        ssh:
            user: myuser
            public_key_file: '/home/myuser/.ssh/id_rsa.pub'
            private_key_file: '/home/myuser/.ssh/id_rsa'
            private_key_file_pwd: 'mypassword'
    zones:
        prod_myproj:
            deployer: myproj
            environment: prod
            checkout_url: 'git@github.com:myrepo/myproj.git'
            checkout_branch: master
            repository_dir: /var/www/production/myproj/deploy
            production_dir: /var/www/production/myproj/code

app/config/parameters_deployer_servers.yml, _(*22)

prod_myproj:
    urls:
        - deploy@testserver1:8822
        - deploy@testserver2:8822

• Note: Servers urls are in other file (parameters_deployers_servers.yml) because this file contains a dynamic configuration that would be changed in a scalable environment like AWS.

Create a deploy class to myproj

• First create your own bundle:

php app/console generate:bundle --namespace=MyProj/DeployBundle --dir=src --no-interaction

• Create your own class to deploy:

src/MyProj/DeployBundle/Service/Test.php:, _(*23)

<?php

namespace MyProj/DeployBundle/Service;

use JordiLlonch\Bundle\DeployBundle\Service\BaseDeployer;

class Test extends BaseDeployer
{
    public function downloadCode()
    {
        $this->logger->debug('Downloading code...');
        $this->output->writeln('<info>Downloading code...</info>');
        $this->downloadCodeVcs();

        $this->logger->debug('Adapting code');
        $this->output->writeln('<info>Adapting code...</info>');
        // Here you can download vendors, add productions configuration, 
        // do cache warm up, set shared directories...

        $this->logger->debug('Copying code to zones...');
        $this->output->writeln('<info>Copying code to zones...</info>');
        $this->code2Servers();
    }

    public function downloadCodeRollback()
    {
    }

    protected function runClearCache()
    {
        $this->logger->debug('Clearing cache...');
        $this->output->writeln('<info>Clearing cache...</info>');
        $this->execRemoteServers('sudo pkill -USR2 -o php-fpm');
    }
}

• Add your deploy class as a service with tag: jordi_llonch_deploy

<service id="myproj.deployer.test" class="MyProj/DeployBundle/Service/Test">
    <tag name="jordi_llonch_deploy" deployer="myproj"/>
</service>

• myproj is used in the configuration as deployer value.

Allow ssh access to remote servers

It is necessary to add the public key of the deploy user to .ssh/authorized_keys in remote servers in order to allow access to the deploy system., _(*24)

Initialize

After configure the deployer you have to do the initialization., _(*25)

app/console deployer:initialize --zones=prod_myproj

Download

Now you can download code from your repository and copy to your servers., _(*26)

app/console deployer:download --zones=prod_myproj

Code to production

After download you just need to put code into production., _(*27)

app/console deployer:code2production --zones=prod_myproj

Rollback

If there is any problem you can roll back to a previous version. See rollback command help., _(*28)

Commands

initialize

Prepare deployer and remote servers creating a directories structure to host new code., _(*29)

app/console deployer:initialize --zones=[zone1,zone2...]

download

Download code from repository, adapt, warn up… and ship it to remote servers in order to put new code to production., _(*30)

app/console deployer:download --zones=[zone1,zone2...]

code2production

Deploy new code to production atomically and reload web server, app..., _(*31)

app/console deployer:code2production --zones=[zone1,zone2...]

syncronize

Ensure that all downloaded versions of code are copied to all servers. Useful when you add a new server to a zone. If you not syncronize the new server the rollback operation will break the code in the new server., _(*32)

app/console deployer:syncronize --zones=prod_myproj

rollback

If there is any problem and you need to roll back to a previous version you have two options:, _(*33)

Using number of steps to roll back

app/console deployer:rollback execute [steps_backward] --zones=[zone1]

• If you want to roll back to a previous version steps_backward should be 1.

Referencing to a concrete version

1) Ask deploy for available versions to rollback., _(*34)

app/console deployer:rollback list --zones=[zone1]

2) Execute rollback to specific version, _(*35)

app/console deployer:rollback execute [version] --zones=[zone1]

status

Shows running version and last downloaded version prepared to put to production., _(*36)

app/console deployer:status [--zones=[zone1,zone2...]]

configure

Configure remote servers for zones. Useful for automatize scaling., _(*37)

app/console deployer:configure zone [add, set, rm, list, list_json] [url]

clean

Remove old code. Left clean_max_deploys deploys., _(*38)

app/console deployer:clean

exec2servers

Executes command passed as argument to all configured servers., _(*39)

app/console deployer:exec2servers [command]

Configuration

Deployer configurations are set in parameters.yml., _(*40)

You must set general configurations and zones., _(*41)

General configuration

app/config/parameters.yml, _(*42)

jordi_llonch_deploy:
    config:
        project: MyProject
        vcs: git
        local_repository_dir: /home/deploy/deploy_repository
        clean_max_deploys: 7
        sudo: true
        ssh:
            user: myuser
            public_key_file: '/home/myuser/.ssh/id_rsa.pub'
            private_key_file: '/home/myuser/.ssh/id_rsa'
            private_key_file_pwd: 'mypassword'

project

Your project name., _(*43)

mail_from

Mail from., _(*44)

mail_to

Mail to send emails about deployments. It is an array., _(*45)

servers_parameter_file

Path where the servers parameters file is. A common configuration could be app/config/parameters_deployer_servers.yml., _(*46)

Servers urls are in this file because it contains a dynamic configuration that would be changed in a scalable environment like AWS by some script., _(*47)

local_repository_dir

Directory where deployer clone your repositories, adapt code and save data about versions in the deploy system., _(*48)

clean_max_deploys

Used in the clean command to remove previous downloaded versions. Left clean_max_deploys downloaded versions., _(*49)

sudo

Add sudo to all commands send to remote servers. If you want to use, you should set your deploy user to sudoers on all remote servers with NOPASSWD:, _(*50)

/etc/sudoers:, _(*51)

deployer_user   ALL=(ALL) NOPASSWD: ALL

ssh

Ssh configuration to establish connections on remote servers to execute commands., _(*52)

Here an example of configuration:, _(*53)

ssh:
    proxy: cli
    user: jllonch
    password: 'mypassword'
    public_key_file: '/home/myuser/.ssh/id_rsa.pub'
    private_key_file: '/home/myuser/.ssh/id_rsa'
    private_key_file_pwd: 'mykeypassword'

proxy

Client to use to execute remote commands., _(*54)

It could be:, _(*55)

• pecl: php-ssh2 extension
• cli: console ssh command

If it is not defined it will try to use pecl if it is installed, otherwise it will use cli., _(*56)

user

Username used in auth by password and auth by public key., _(*57)

password

Password used in auth by password., _(*58)

public_key_file

Public key file path., _(*59)

private_key_file

Private key file path., _(*60)

private_key_file_pwd

Private key password., _(*61)

helper

Helper parameters that can be get in your deploy class., _(*62)

Zones configuration

You need to set a minimum of one zone. Here is created your zone prod_myproj:, _(*63)

app/config/parameters.yml, _(*64)

jordi_llonch_deploy:
...
    zones:
        prod_myproj:
            deployer: myproj
            environment: prod
            checkout_url: 'git@github.com:myrepo/myproj.git'
            checkout_branch: master
            checkout_proxy: true
            repository_dir: /var/www/production/myproj/deploy
            production_dir: /var/www/production/myproj/code
            custom:
                my_key1: value1
                my_key2: value2

app/config/parameters_deployer_servers.yml, _(*65)

prod_myproj:
    urls:
        - deploy@testserver1:8822
        - deploy@testserver2:8822

deployer

Name of the service used to deploy the zone., _(*66)

<service id="myproj.deployer.test" class="MyProj/DeployBundle/Service/Test">
    <tag name="jordi_llonch_deploy" deployer="myproj"/>
</service>

environment

Environment., _(*67)

urls

Remote servers where deployed code will be copied and set to production., _(*68)

Format is: [user]@[server]:[port], _(*69)

checkout_url

Url to git repository., _(*70)

checkout_branch

Git branch to clone., _(*71)

checkout_proxy

Deployer always clone a repository for every deploy. If you want to avoid to download from remote server you can clone your repository locally, then set checkout_url to your local local repository (file:///home/deploy/proxy_repositories/myproj). Then before download operation, deployer execute a git pull to your local repository., _(*72)

repository_dir

Path on remote servers where to copy new deployed code., _(*73)

production_dir

Path that is updated by deployer when new deployed code is set to production. So you must set this path to your webserver as a root path., _(*74)

custom

Custom parameters that can be get in your deploy class., _(*75)

helper

Helper parameters that must be set to use some helpers., _(*76)

If helper parameters are set in general configuration those configuration are merged here., _(*77)

Helpers

Hepers automatize common operations like to install composer dependencies, to do a cache warm up for Symfony2 projects, to restart php-fpm after to put code to production, to get url to GitHub with differences of new deployed code..., _(*78)

How use it?

You can use Helpers in your Deployer classes to execute common operartions., _(*79)

Here and example:, _(*80)

src/MyProj/DeployBundle/Service/Test.php:, _(*81)

<?php

namespace MyProj/DeployBundle/Service;

use JordiLlonch\Bundle\DeployBundle\Service\BaseDeployer;

class Test extends BaseDeployer
{
    public function initialize()
    {
        parent::initialize();

        // Shared dirs
        $this->getHelper('shared_dirs')->initialize('logs');
    }

    public function downloadCode()
    {
        $this->logger->debug('Downloading code...');
        $this->output->writeln('<info>Downloading code...</info>');
        $this->downloadCodeVcs();

        $this->logger->debug('Adapting code');
        $this->output->writeln('<info>Adapting code...</info>');

        // composer
        $this->getHelper('composer')->install();
        $this->getHelper('composer')->executeInstall();

        // cache warm:up
        $this->getHelper('symfony2')->cacheWarmUp();

        // shared directories
        $this->getHelper('shared_dirs')->set('app/logs', 'logs');

        $this->logger->debug('Copying code to zones...');
        $this->output->writeln('<info>Copying code to zones...</info>');
        $this->code2Servers();
    }

    public function downloadCodeRollback()
    {
    }

    protected function runClearCache()
    {
        $this->logger->debug('Clearing cache...');
        $this->output->writeln('<info>Clearing cache...</info>');
        $this->getHelper('phpfpm')->refresh();
    }
}

• As you can see we used some methods prefixed by helper… to install composer dependencies, doing cache warm up, setting shared directories and restarting php-fpm.

Provided helpers

SharedDirs

Easy way to manage shared directories., _(*82)

$this->getHelper('shared_dirs')->initialize($path), _(*83)

• Initialize given path in the shared directory.

$this->getHelper('shared_dirs')->set($pathInAppToLink, $pathInSharedDir), _(*84)

• Set shared directory for a given path in the project that will be removed and replaced by a symlink to given path to shared directory.

PhpFpm

Provides methods to restart php-fpm gracefully., _(*85)

$this->getHelper('phpfpm')->refresh(), _(*86)

• Refresh php-fpm gracefully but you must configure your webserver (e.g. Nginx) to retry the request.

$this->getHelper('phpfpm')->refreshCommand(), _(*87)

• Command used to reload php-fpm

Composer

Helpers to manage composer installation and some commands., _(*88)

$this->getHelper('composer')->install(), _(*89)

• Install composer.phar in the new repository dir.

$this->getHelper('composer')->executeInstall(), _(*90)

• Executes composer install in the new repository dir.
• If environment is dev or test --dev parameter is added to composer install
• If environment is prod --no-dev parameter is added to composer install

Symfony2

For now only provides a method to do a cache warm up., _(*91)

$this->getHelper('symfony2')->cacheWarmupOnServers(), _(*92)

• Do a cache:warmup for production environment. This operation should be executed after code2Servers() operation.

Deprecated:

$this->getHelper('symfony2')->cacheWarmUp(), _(*93)

• This warmup it is not a secure operation because serialized data could be corrupted.

GitHub

Useful methods to have feedback of your deploy in GitHub., _(*94)

$this->getHelper('github')->getCompareUrl($gitUidFrom, $gitUidTo), _(*95)

• Give an url comparing two commits
• You must set your http url to your GitHub repository in jordi_llonch_deploy.zones parameters:

helper:
    github:
        url: https://github.com/YourUser/Repository

$this->getHelper('github')->getCompareUrlFromCurrentCodeToNewRepository(), _(*96)

• Give an url comparing commits between current running code and the new downloaded code.

HipChat

Provides a method to send messages to a room in a HipChat., _(*97)

$this->getHelper('hipchat')->send($msg, $color='purple'), _(*98)

• Send a message to a given room
• You must set your token and room_id to your HipChat in jordi_llonch_deploy.general parameters:

helper:
    hipchat:
        token: your_token
        room_id: your_room_id

TODO

• Tests
• Ssh with Process component

Author

Jordi Llonch - llonch.jordi at gmail dot com, _(*99)

License

DeployBundle is licensed under the MIT License. See the LICENSE file for full details., _(*100)