# continuous integration using sparrow tool chain

[Sparrow](https://github.com/melezhik/sparrow) is a tool to automate testing infrastructure. Automated testing
is essential part of continuous integration processes as it provides fast feedback
on cases when something goes wrong.


Consider simple example of how sparrow could be used to build up some basic parts
of your infrastructure.

# web application development 

We have a simple Dancer2 application we are going to deploy on development environment:


app.psgi

    #!/usr/bin/env perl
    use Dancer2;

    get '/' => sub {
        "Hello World!"
    };

    dance;


Let's keep source code at git repository:


    git init
    git add app.psgi
    git commit -a -m 'my web application'

    git remote add origin https://github.com/melezhik/webapp.git
    git push -u origin master

# development server

We are going to deploy application on dedicated server used for development environment:

    ssh dev.server
    git clone https://github.com/melezhik/webapp.git
    cd webapp
    cpanm Dancer2
    plackup


In these lines we fetch source code from remote git repository and run dancer application. Good so far.
These steps could be automated in various ways ( jenkins, crontab , whatever your favorite CI tool ).

Last command should emit following:

    HTTP::Server::PSGI: Accepting connections at http://0:5000/

Which means our application is running.


# building up test harness 

As we need to ensure that app is running correctly after being deployed we have to add  some integration tests for it.
With sparrow it is as simple as writing a few lines of code:

    git init # let's keep test suite case under git

    echo 127.0.0.1:5000 > host

    nano get.txt
        200 OK
        Hello World!

    echo '{}' > sparrow.json

    echo "requires 'swat';" > cpanfile

    git add .
    git commit -a -m 'basic test suite'
    git remote add origin https://github.com/melezhik/webapp-basic-check.git
    git push -u origin master 


Now when we are done with creating a very simple test suite let's go to development server and create some check points:

    ssh dev.server
    cpanm Sparrow

    echo "basic-check https://github.com/melezhik/webapp-basic-check.git" > ~/sparrow.list      
    sparrow index update
    sparrow plg install basic-check

    sparrow project create webapp
    sparrow check add webapp basic
    sparrow check set  webapp basic basic-check
    
    sparrow check run webapp basic

Output of last command will be:

    # running cd /home/vagrant/sparrow/plugins/private/basic-check && carton exec 'swat ./   ' ...
    
    /home/vagrant/.swat/.cache/30818/prove/00.GET.t ..
    # trying ... curl -X GET -k --connect-timeout 20 -m 20 -L -f -D - '127.0.0.1:5000/'
    ok 1 - server returned successful response
    ok 2 - output match '200 OK'
    ok 3 - output match 'Hello World!'
    1..3
    ok
    All tests successful.
    Files=1, Tests=3,  0 wallclock secs ( 0.03 usr  0.00 sys +  0.04 cusr  0.00 csys =  0.07 CPU)
    Result: PASS
    

Ok, we see that our tests succeed and we can continue with development.


# adding new feature to web application

Let's add authentication to your application:


app.psgi

    #!/usr/bin/env perl

    use Dancer2;
    use Dancer2::Plugin::Auth::Tiny;

    set show_errors => 1;
    set session     => 'Simple';


    get '/' => sub {
        "Hello World!"
    };


    get '/public' => sub { return 'public area' };

    get '/private' => needs login => sub { return 'private area' };

    get '/login' => sub {
        session "user" => "Robin Good";
        return "login and to back to " . params->{return_url};
    };

    get '/logout' => sub {
        app->destroy_session;
        redirect uri_for('/public');
    };

    dance;


Let's create a check list we need to ensure:

* when user hits /public route he sees 'public area'
* when user hits /private route for a first time he gets redirected to /login page and then gets a session cookies
* when user hits /private routes for a second time he sees 'private area'

Now create another suite case for the stories above, we are going to keep under another git repository and
then deliver tests as another sparrow plugin as we did with the basic suite case:


## creating test suite skeleton

    git init
    echo 127.0.0.1:5000 > host
    echo '{}'> sparrow.json
    echo "requires 'swat';" > cpanfile
    git add .
    git commit -a -m 'authentication test suite'
    git remote add origin https://github.com/melezhik/webapp-auth-check.git
    git push -u origin master 


## public area test

    mkdir public

    nano public/get.txt
        200 OK
        public area


## private area first time 

    mkdir private-first-time
    nano private-first-time/hook.pm

        run_swat_module( GET => '/logout' );
        run_swat_module( GET => '/private' , { auth => 0 } );
        set_response('done');

    echo done > private-first-time/get.txt

    mkdir logout/
    echo swat_module=1 > logout/swat.ini
    echo 200 OK > logout/get.txt

    mkdir private/

    nano private/swat.ini

        swat_module=1
        curl_params="-b ${test_root_dir}/cook.txt"

    nano private/get.txt

        generator: [ module_variable('auth') ? 'private area' : 'login and to back to' ]

## private area second time 

    mkdir private-second-time
    nano private-second-time/hook.pm

        run_swat_module( GET => '/login' );
        run_swat_module( GET => '/private' , { auth => 1 } );
        set_response('done');

    mkdir login/

    nano login/swat.ini

        swat_module=1
        curl_params="-c ${test_root_dir}/cook.txt"

    echo 200 OK > login/get.txt




And finally we can commit changes to git and push them to remote.

    git add .
    git commit -a -m 'authentication check test suite'
    git push

    
# testing new features

As we did with basic test suite we now are able to run tests for authentication feature:


    ssh dev.server
    echo "auth-check https://github.com/melezhik/webapp-auth-check.git" >> ~/sparrow.list      
    sparrow index update
    sparrow plg install auth-check

    sparrow check add webapp auth
    sparrow check set  webapp auth auth-check
    
    sparrow check run webapp auth


The output of the second test suite will be:

    # running cd /home/vagrant/sparrow/plugins/private/testapp2 && carton exec 'swat ./   ' ...
    
    /home/vagrant/.swat/.cache/1085/prove/private-first-time/00.GET.t ...
    # trying ... curl -X GET -k --connect-timeout 20 -m 20 -L -f -D - '127.0.0.1:5000/logout'
    ok 1 - server returned successful response
    ok 2 - output match '200 OK'
    # trying ... curl -X GET -k --connect-timeout 20 -m 20 -L -b /home/vagrant/.swat/.cache/1085/prove/cook.txt -f -D - '127.0.0.1:5000/private'
    ok 3 - server returned successful response
    ok 4 - output match 'login and to back to'
    ok 5 - server response is spoofed
    # response saved to /home/vagrant/.swat/.cache/1085/prove/xDQuPGlVog
    ok 6 - output match 'done'
    1..6
    ok
    /home/vagrant/.swat/.cache/1085/prove/public/00.GET.t ...............
    # trying ... curl -X GET -k --connect-timeout 20 -m 20 -L -f -D - '127.0.0.1:5000/public'
    ok 1 - server returned successful response
    ok 2 - output match '200 OK'
    ok 3 - output match 'public area'
    1..3
    ok
    /home/vagrant/.swat/.cache/1085/prove/private-second-time/00.GET.t ..
    # trying ... curl -X GET -k --connect-timeout 20 -m 20 -L -c /home/vagrant/.swat/.cache/1085/prove/cook.txt -f -D - '127.0.0.1:5000/login'
    ok 1 - server returned successful response
    ok 2 - output match '200 OK'
    # trying ... curl -X GET -k --connect-timeout 20 -m 20 -L -b /home/vagrant/.swat/.cache/1085/prove/cook.txt -f -D - '127.0.0.1:5000/private'
    ok 3 - server returned successful response
    ok 4 - output match 'private area'
    ok 5 - server response is spoofed
    # response saved to /home/vagrant/.swat/.cache/1085/prove/hvtwlrTqEZ
    ok 6 - output match 'done'
    1..6
    ok
    All tests successful.
    Files=3, Tests=15,  0 wallclock secs ( 0.03 usr  0.01 sys +  0.17 cusr  0.01 csys =  0.22 CPU)
    Result: PASS
    
    
# Summary

Sparrow tool chains has following features:

* integration tests suites are decoupled from application code

* tests could be grouped by types/environments and developed/delivered as dedicated test suites - aka sparrow plugins

* sparrow test infrastructure is easy to bootstrap on different environments (dev,test,prod) - everybody involved
is able now to run desired test suite with easiness

* swat output is intended to simplify debugging/troubleshooting  process, in most cases you just rerun curl command ( against tested server )
with parameters extracted from test output.


# PS 

## paper source code
the source code for this post could be found [here](https://github.com/melezhik/sparrow/tree/master/papers/CI)

## code  examples :

* [webapp](https://github.com/melezhik/webapp)
* [webapp-basic-check](https://github.com/melezhik/webapp-basic-check)
* [webapp-auth-check](https://github.com/melezhik/webapp-auth-check)

