=head1 NAME

Fennec::Manual::User - The user manual for Fennec

=head1 DESCRIPTION

Covers usage of Fennec when writing tests and managing test suites.

=head1 CHAPTER 1 - INTRODUCTION

Fennec is a workflow based testing framework for Perl. It is designed to be
easy to use, easy to extend, and to provide an incredible array of tools while
staying out of your way. Fennec provides everything your used to in Perl
testing framework along with enhancements inspired from other frameworks.

Fennec works just like traditional L<Test::Builder> tools in that it allows
you to write script style tests. Fennec takes this a step further by
introducing behavior driven development with the concepts of test groups and
workflows. Fennec also requires that each test file define a class which Fennec
will instantiate.

=head2 CHAPTER 1.1 - WHY USE FENNEC?

Better tools lead to better tests, which in turn lead to better code. Perl has
fallen behind the times when it comes to testing tools. Behavior driven
development has proven it's worth in other laungages. A few projects have
attempted to bring BDD to Perl, but they generally do not play well with
existing Perl tools such as L<Test::Builder>.

Fennec brings these new tools to Perl in a way that plays nicely with existing
tools. Fennec will set you do things the way you already are, while allowing
you to adopt new tools and features as you desire. Fennec also provides an
excellent framework on which you can develop new tools that are compatible and
re-usable.

=head1 CHAPTER 2 - USING FENNEC

=head2 CHAPTER 2.1 - WRITING A BASIC TEST CLASS

The most simple Fennec test class is a script that declares a package, uses
L<Fennec::Standalone> and makes assertions.

For example, F<t/my_test.t> might look like:

    #!/usr/bin/perl
    package My::Test
    use strict;
    use warnings;
    use Fennec::Standalone;

    ok( 1, "1 is true, perl is not broken" );

    done_testing();

*done_testing() is required at the end of all standalone tests.

=head2 CHAPTER 2.2 - ASSERTIONS

Fennec refers to test functions such as ok(), is(), and is_deeply() by the term
'assertion'. Using Fennec automatically gives you all the assertions normally
provided by L<Test::More>, L<Test::Warn>, and L<Test::Exception>. Please see
L<Test::Simple> if you are not already familiar with testing in Perl.

It is worth noting that these asserts are re-implemented the Fennec way. They
do not use Test::Builder. This was done in order to provide better diagnostics
and output. Test::Builder tools will work under Fennec, in most cases
without any modification. It is also possible to tell Fennec to use the
Test::Builder based tools instead of its own implementations, See Chapter 7.2.

Sample assertions:

    ok( $should_be_true, "name of check" );
    is( $have, $want, "name of other check" );

Fennec also allows for traditional assertions which die on failure, but only
when tests are grouped. See Chapter 3.2 for more on groups.

    tests my_tests {
        die "check failed" unless $a == $b;
    }

Using a Test::Builder library usually just works:

    use Test::Thing qw( my_assert );

    my_assert( ... );

=head2 CHAPTER 2.3 - DEFAULT ASSERTIONS

By default, Fennec will provide the following assert functions. See each
referenced package for more details.

=head4 L<Fennec::Assert::Core::Simple>

=over 2

=item ok( $check, $name )

=item diag( $message )

=item TODO { ... } $reason

=back

=head4 L<Fennec::Assert::Core::More>

=over 2

=item is( $got, $want, $name )

=item isnt($got, $want; $name)

=item like($got, $want; $name)

=item unlike($got, $want; $name)

=item can_ok(*thing; @list )

=item isa_ok(*thing; @list )

=item is_deeply($got, $want; $name)

=item advanced_is(got => $got, want => $want, name => $name, %options)

=back

=head4 L<Fennec::Assert::Core::Exception>

=over 2

=item lives_ok { ... } $name

=item dies_ok { ... } $name

=item throws_ok { ... } qr/.../, $name

=item lives_and { ... } $name

=back

=head4 L<Fennec::Assert::Core::Warn>

=over 2

=item @list = capture_warnings { ... }

=item warning_is { ... } $want, $name

=item warnings_are { ... } \@want, $name

=item warning_like { ... } qr/.../, $name

=item warnings_like { ... } [ qr/.../, ... ], $name

=item warnings_exist { ... } [ $string, $regex, ... ], $name

=back

=head4 L<Fennec::Assert::Core::Package>

=over 2

=item require_ok Package::Bareword

=item require_ok( $package )

=item use_ok Package::Bareword

=item use_ok( $package, @args )

=item use_into_ok( $package, $dest_package, @args )

=back

=head4 L<Fennec::Assert::Core::Anonclass>

=over 2

=item $anonclass = anonclass( ... )

=back

=head2 CHAPTER 2.4 - CUSTOM INLINE ASSERTIONS

TODO - L<http://github.com/exodist/Fennec/issues/#issue/60>

=head2 CHAPTER 2.5 - TODO AND SKIP

Fennec has the concept of todo tests, tests which are expected to fail. You can
also mark groups as skip if they are really bad.

If an exception is thrown within a C<TODO> block or set, a failing C<TODO> result
will be generated alerting you. This is a todo test and will not count as a
failure in the grand scheme.

=over 2

=item Mark a test group as todo

    # Failures will not count.
    tests not_yet_implemented => (
        todo => "This will fail",
        method => sub {
            my $self = shift;
            ok( 0, "Hello world" );
        },
    );

=item Skip a group completely

    tests 'would die' => (
        skip => "This will die",
        method => sub {
            my $self = shift;
            die( "I eat you" );
        },
    );

=item Skip specific asserts

    tests 'some pass' => sub {
        ok( 1, 'pass' );
        TODO {
            ok( 0, 'fail' );
        } "This will fail, I will fix it later";
    }

=back

=head1 CHAPTER 2.6 - META DATA

All Fennec test classes have an associated L<Fennec::TestFile::Meta> object
associated with them. A classes meta object can always be accessed with the
$TEST_CLASS->fennec_meta() method. In addition the M() function can be used
within the test class to access that classes meta object.

The meta object stores information you will almost never need to mess with.
This includes data used by workflows, plugins, and other tools. B<NOTE> The
meta is associated with a class, not an object.

=head1 CHAPTER 2.7 - STASH

The stash is a hashref contained in the meta object. Because of this every test
class has a single stash hash. You can always access the stash using the
$TEST_CLASS->fennec_meta()->stash() method. In addition you can use the S()
function from within a test class to quickly access and manipulate the stash.

=over 4

=item my $stash_ref = S();

=item my %stash_copy = S();

The S() function returns a ref or hash pairs depending on the return context.

=item my $value = S( $key )

When given a single key is provided as an argument, the value for that key will
be returned.

=item S( %MERGE_DATA )

When given one or more key/value pairs, the new data will be merged in with the
old data.

=item S( \%OVERRIDE )

When given a single hashref, that hashref will replace the entire stash.

=back

=head1 CHAPTER 2.8 - PLAYING NICELY WITH PLUGINS AND EXTENTIONS

Many plugins and extentions rely on meta information being added to the stash.
The most prevelant is the 'testing' key. This key should contain the name of
the package being tested by the test class you are writing.

Example:

    Package TEST::My::Module;
    use strict;
    use warnings;
    use Fennec;
    ...
    S( testing => 'My::Module' );
    ...

You should check with specific plugins for other stash keys that should be set.

=head1 CHAPTER 3 - SUITE MANAGEMENT

There are two ways to manage a test suite with Fennec. The simple way is to
simply write .t files that use L<Fennec::Standalone>. The other way is to turn
control over to the Fennec 'runner' script.

=head2 CHAPTER 3.1 - STANDALONE FILES

Each Fennec test file is independent. Prove iterates the files for you running
them in order. Every test has the overheard of re-loading Fennec and its tools.
Files will not share a common configuration you must manually enter your
configuration into each file.

Using L<Fennec::Standalone> has already been introduced in Chapter 1.2.

=head2 CHAPTER 3.2 - SINGLE RUNNER

Fennec will be configured in t/Fennec.t, which will also serve as a runner. The
runner will find all your test files and run them for you. The benefits of this
system are reduced overhead, shared configuration, and the ability to
potentially pre-load items that are cpu intensive to load.

The downside of a runner system is that it hijacks prove. Runners
have been used in other Perl testing frameworks such as L<Test::Class>. Some
people have strong opinions about runners, usually unfavorable ones. Runners
often have the problem that it is difficult to run a single test file.

Fennec has many command-line and editor integration tools that solve the most
common complaints about runners. See Chapter 2.2.1 and Chapter 5 for more
details.

To generate t/Fennec.t use the C<fennec_init> command. This will create the file
with the default Fennec configuration. This file doubles as a configuration
file. See Chapter 2.3 - Configuration for more details.

=head3 CHAPTER 3.2.1 - COMMAND LINE TOOLS

Fennec's command-line tools are intended to simplify working with Fennec.

=over 4

=item fennec_init

    $ cd projectroot
    $ fennec_init
    $ ls t
        t/Fennec.t

This command-line tool will generate a default Fennec.t file under the t/
directory. This is where you start when writing a test-suite using fennec.

=item fennec_prove

This only applies to managed tests. This command will not work for standalone
tests.

    $ fennec_prove - - [prove options]
    $ fennec_prove t/My/Test.pm - [prove options]
    $ fennec_prove t/My/Test.pm test_name [prove options]
    $ fennec_prove t/My/Test.pm line_number [prove options]

This is a wrapper around prove that lets you run a specific fennec test file,
and optionally a specific test name or line number within that file. Running a
specific line number is not 100% accurate, but pretty close.

=item fennec_scaffold.pl

This will create test modules for all modules that do not currently have a
test. It will find all modules under F<lib/> and create a test under F<t/>. The
created test file will have a single C<require_ok( module_name )> test.

Test names are assumed to follow the standard format. F<t/> will mirror F<lib/>, and
test module names will be the same as the module being tested. Package names
will be the name of the tested package prefixed with C<TEST::>. If you deviate
from this style F<fennec_scaffold.pl> will not find your test files and will
assume they do not exist.

=item t/Fennec.t

This is the script that runs all your Fennec tests. It also doubles as a global
configuration file. Most config options can be overridden using environment
variables. See L<Chapter 5 - Configuration> for more info.

=item fennec_run.pl

This is identical to the default F<t/Fennec.t> file. It can be used to run Fennec
tests in a project without a F<t/Fennec.t>.

=back

=head1 CHAPTER 4 - TEST GROUPING

Test groups are methods containing assertions, typically defined using the
C<tests> keyword. Tests will typically be grouped based on the feature or
functionality being tested.

There are several benefits to grouping tests:

=over 4

=item Better structure leads to better readability

=item Groups can be run in parrallel

=item You can tell Fennec to run a specific group alone

=back

=head2 CHAPTER 4.2 WRITING A TEST GROUP

The 'tests' keyword uses L<Devel::Declare> to work similar to the 'sub'
keyword, that is no semicolon is required after the definition. The keyword is
further enhanced allowing for single or double quoted method names.

    tests 'name of group' {
        ok( 1, "name" );
        ...
    }

    tests bareword_name { ... }

    tests "double quoted" { ... }

=head1 CHAPTER 5 - BEHAVIOR DRIVEN DEVELOPMENT

Behavior driven development or BDD is a way to structure tests in a more
readable or manageable way than script style tests. The most well known example
of BDD is Ruby's RSPEC, of which Fennec has an implementation.

Fennec implements BDD using the concept of workflows. Workflows are an
abstraction of the common logic needed for BDD systems. Most Fennec BDD systems
will be referred to generically as workflows.

When a Fennec test is run, it will first run all your package level assertions
(ok, is, is_deeply, etc). While doing so it will also build and prepare your
test groups and workflows. Once the building phase is complete, Fennec will
instantiate your test class, and run all groups and workflows on that instance.

=head2 CHAPTER 5.1 - RSPEC

RSpec is a workflow originally created for Ruby. this manual will not go over
RSpec in detail. Here is an example to get you started. It is worth noting that
describe blocks are run as methods and get $self automatically.

    describe 'test feature a' {
        my $target;

        before_each { $target = My::Class->new }
        after_each { My::Class->reset_test_data }

        # These will run in random order wrapped with the before and after:
        it { ok( $target->feature_a( 'a' ), 'takes arg a' }
        it { ok( $target->feature_a( 'b' ), 'takes arg b' }
        it { ok( $target->feature_a( 'c' ), 'takes arg c' }

        # You can nest describe methods
        describe 'feature a other stuff' { ... }
    }

The RSpec reference can be found here: L<http://rspec.info/>

TODO - Need an RSpec guru to add more here.

=head2 CHAPTER 5.2 - CASES

Cases are a way to run the same assertions under multiple scenarios. In the
following example each tests block will be run 4 times each with different
values in var_a and var_b.

B<Note>: Cases and tests can be define in any order. The resulting test groups
will be run in random order, after all have been built.

    cases 'test all combinations' {
        my ( $var_a, $var_b );

        # Scenarios
        case set_a {( $var_a, $var_b ) = ( 5, 5 )}
        case set_b {( $var_a, $var_b ) = ( 4, 6 )}
        case set_c {( $var_a, $var_b ) = ( 3, 7 )}
        case set_d {( $var_a, $var_b ) = ( 2, 8 )}

        # Checks
        tests 'equals 10' { is( $var_a + $var_b, 10, "sum is ten" )}
        tests 'both vars are true' {
            ok( $var_a, "var_a is true" );
            ok( $var_b, "var_b is true" );
        }
    }

=head2 CHAPTER 5.3 - OBJECT METHODS

You can also define test groups, setups, and teardowns as methods on your test
object.

    sub setup_my_setup {
        my $self = shift;
        print "methods prefixed by setup_ will be run before tests defined as methods.";
    }

    sub test_method_as_test_by_prefix {
        my $self = shift;
        ok( 1, "methods prefixed by test_ will be run as method." );
    }

    sub teardown_my_teardown {
        my $self = shift;
        print "method prefixed by teardown_ will be run after tests defined as methods."
    }

=head1 CHAPTER 6 - CONFIGURATION

Fennec is highly configurable. By default Fennec provides a sane configuration.
The two main items to configure are the individual test files themselves, and
the runner. When using L<Fennec::Standalone> this distinction remains, however
both are configured within each test file.

=head2 CHAPTER 6.1 - IN-TEST CONFIGURATION

These options are consistent between managed and standalone tests. These
options are specified in test files when importing Fennec or
Fennec::Standalone.

    use Fennec OPTION => VALUE, ...;
    # or
    use Fennec::Standalone OPTION => VALUE, ...;

=over 4

=item random => $bool

Default is true. When true test groups and workflows will be run in random
order. When false tests will run in hash key order, which is consistant though
unpredictable.

=item sort => $bool

Default is false. When true test groups and workflows will be run in sorted
order.

=item asserts => \@list

Used as a shortcut for loading additional assertion libraries. Each item should
be a package name with the leading Fennec::Assert removed.

=item workflows => \@list

Used as a shortcut for loading additional workflow libraries. Each item should
be a package name with the leading Fennec::workflow removed.

=item meta => { ... }

*YOU ALMOST CERTAINLY WILL NOT NEED TO SET THIS*

TODO - Fill this in

=back

=head2 CHAPTER 6.2 - CONFIGURING THE RUNNER

=over 2

=item Managed Suite

For managed tests these configuration items should go in t/Fennec.t:

    #!/usr/bin/perl
    ...
    Fennec::Runner->init(
        #### CONFIG OPTIONS GO HERE ####
        OPTION => VALUE,
        ...
    );
    ...

=item Standalone

For Standalone tests the runner is configured using an import option.

    use Fennec::Standalone runner => { OPTION => VALUE, ... };

=back

=head3 CHAPTER 6.2.1 - Runner Configuration Options

These are the runner configuration options, many of these do not make sense in
a standalone test.

=over 4

=item parallel_files => $INTEGER

Default: 2

How many test files to run in parallel.

=item parallel_tests => $INTEGER

Default: 2

How many test groups to run in parallel.

=item default_asserts => \@LIST

Default: [ 'Core' ]

Assertion libraries to load by default. The leading Fennec::Assert may be
omitted.

=item default_workflows => \@LIST

Default: [qw/Spec Case Methods/],

Workflow libraries to load by default. The leading L<Fennec::Workflow> may be
omitted.

=item load => [ [ $package1, @args1 ], [ $package2, @args2 ], ... ]

Should be an array of arrays. The inner arrays should contain a package name,
followed by import arguments. Every package listed will be loaded into every
test class.

Example:

    load => [
        [ qw/Package::A a b c /],
        [ qw/Package::B d e f /],
    ],

Is equvilent to adding the following to every single test file.

    use Package::A qw/a b c/;
    use Package::B qw/d e f/;

=item handlers => \@LIST

Default: [ 'TAP' ]

Output handlers to load by default. The leading L<Fennec::Handler> may be omitted.

=item collector => 'CLASS::NAME'

Default: 'Files'

Collector to use by default. The leading L<Fennec::Collector> may be omitted.

=item cull_delay => $SECONDS_FLOAT

Default: 0.01

How frequently the collector should check for new results. Lower is more
frequently.

=item filetypes => \@LIST

Default: [ 'Module' ]

File types to load by default. The leading L<Fennec::FileType> may be omitted.

=item ignore => \@LIST

Default: undef

List of filenames and regexes that specify files that should be ignored.

=item root_workflow_class

Default: L<Fennec::Workflow>

The root workflow class to which all workflows will be children.

=item random => $BOOL

Default: TRUE

When true test groups and workflows will be run in random order. When false
tests will run in hash key order, which is consistent though unpredictable.

=back

=head3 CHAPTER 6.2.2 - Overriding Runner Options on The Command-Line

Almost every option above can be overridden on the command line using
environment variables. Each items variable is its name in all caps prefixed by
C<FENNEC_>.

Example:

    $ FENNEC_PARRALLEL_FILES=5 FENNEC_PARRALLEL_TESTS=3 prove -I lib t/Fennec.t

=head1 CHAPTER 7 - PLUGINS

=head2 CHAPTER 7.1 - USEFUL PLUGINS

=over 4

=item L<Fennec::Template>

Essentially Roles for Fennec. Lets you create template test groups and
workflows that can be imported into any Fennec test class.

=item L<Fennec::Server>

B<UNDER DEVELOPMENT> - Not stable or usable at the time of this writing.

A webserver will act as collector. You can monitor test status by visiting the
server. There are also tools included for continuous integration testing of
multiple projects.

=back

=head2 CHAPTER 7.2 - GENERIC PLUGIN INSTRUCTIONS

Most plugins should be used in t/Fennec.t just after use Fennec::Runner. They
may then require additional meta information be added to the stash.

    #!/usr/bin/perl
    use strict;
    use warnings;
    use Fennec::Runner;
    use My::Plugin;

    ...

In standalone tests you want to use the plugin at the start of the file. Some
may require you to use them before using Fennec::Standalone, others will
require you do it afterwords, for most it probably does not matter.

=head1 CHAPTER 8 - EDITOR INTEGRATION

When editing a test file, it is useful to be able to run the test regularly on
its own. With regular t/*.t files this is easy enough, however when test files
are processed by a runner this can seem difficult. Fennec solves this problem
through environment variables the editor can set. In act Fennec provides a
mechanism to run specific test groups within a given file.

The Fennec runner script (t/Fennec.t) is designed to work well with editor
integration. Almost every configuration option can be overridden via an
environment variables (See Chapters 5.2.1-2). In addition to configuration
variables, the script recognizes the following variables:

=over 4

=item FENNEC_SEED

When Fennec runs it prints the random seed that is used. This seed can be used
with this variable to reproduce a previous run.

=item FENNEC_FILE

Used to tell the runner to only run the specified file. Useful when working on
a specific test and in editor integration.

=item FENNEC_ITEM

Used to tell the runner to only run the specified item. Useful when working on
a specific test and in editor integration.

The item can be either a group name, workflow name, or line number that defines
either of those.

=back

=head2 CHAPTER 8.1 - GENERIC EDITOR HOWTO

Editor integration generally means binding a key combination to run the current
test file. With Fennec it means the same, with the addition of binding a second
combination for running the current part of the current test file.

=over 4

=item Running the current file

Most editors that provide binding support can run a command and provide the
name of the current file. You should bind your key to run the t/Fennec.t script
while setting the FENNEC_FILE environment variable to the current test file.

Example Command:

    FENNEC_FILE='%CURFILE' %PDIR/t/Fennec.t

=item Running the current part of the current file

The traditional approach to this would be to train your editor to find the name
of the section your editing and pass that to a command. The problem with that
system is the existing and growing variety of ways to define groups and
workflows. Fennec does support this system, but also provides a simpler way.

Internally Fennec attempts to track line numbers as groups and workflows are
define. Since workflows are all defined before any are run, you can simply pass
in a line number and Fennec will figure out the rest for you. Line number
deduction is not 100% perfect, but is good enough.

The environment variable FENNEC_ITEM is used to provide either the line number
or group name.

Example Command:

    FENNEC_FILE='%CURFILE' FENNEC_ITEM='14' %PDIR/t/Fennec.t

=back

=head2 CHAPTER 8.2 - VIM INTEGRATION

Fennec provides 2 files in the dist used for vim integration.

=over 4

=item F<share/vim/functions.vim>

This file defines the C<RunFennecLine()> function. This function runs the current
test group in the current file.

=item F<share/vim/vimrc>

This file binds F10 to run the current file, and F8 to run the current part of
the current file.

=back

=head2 CHAPTER 8.3 - EMACS INTEGRATION

Currently there are no emacs users developing Fennec. We welcome anyone who
would fill in this section.

=head1 CHAPTER 9 - TEST::BUILDER INFORMATION

Fennec does not use or depend on L<Test::Builder>. However it has a module that
allows Test::Builder based tools to work with Fennec.
L<Fennec::Util::TBOverride> overrides several parts of Test::Builder in order
to send results to Fennec. It also provides a mechanism by which Test::Builder
tools can be wrapped by Fennec to improve diagnostic output.

=head2 CHAPTER 9.1 - USING TEST::BUILDER TOOLS

L<Fennec::Util::TBOverride> is loaded by default when Fennec loads. That means
that in most cases you can simply import the Test::Builder tool and expect it
to work fine. Fennec also provides wrappers around the most common
Test::Builder tools for people that prefer them to the Fennec implementations.

=head2 CHAPTER 9.2 - USING TEST::BUILDER INSTEAD OF FENNEC ASSERTS

By default Fennec Loads the Core assertion library (L<Fennec::Assert::Core>)
which in turn loads the following (among others):

=over 4

=item L<Fennec::Assert::Core::Simple>

=item L<Fennec::Assert::Core::More>

=item L<Fennec::Assert::Core::Exception>

=item L<Fennec::Assert::Core::Warn>

=back

B<These are re-implementations of their Test::Builder namesakes>. Fennec also
provides a way to use the original implementations.

    use Fennec asserts => ['TBCore'];
    # or
    use Fennec::Standalone asserts => ['TBCore'];

You can also do this globaly in t/Fennec.t:

    #!/usr/bin/perl
    ...
    'Fennec::Runner'->init(
        default_asserts => ['TBCore'],
        ...
    );
    ...

This will load L<Fennec::Assert::TBCore> instead of L<Fennec::Assert::Core>.
Which in turn loads the following:

=over 4

=item L<Fennec::Assert::TBCore::Simple>

=item L<Fennec::Assert::TBCore::More>

=item L<Fennec::Assert::TBCore::Exception>

=item L<Fennec::Assert::TBCore::Warn>

=back

These are minimal wrappers around the actual Test::Builder based tools.

=head1 CHAPTER 10 - ADVANCED TOPICS

=head2 CHAPTER 10.1 - OUTPUT HANDLERS

By default Fennec produces TAP output, however it provides the tools to output
any format you can think of. You can even output to multiple formats at once.
Output plugins are called Handlers, they can be found in the Fennec::Handler::
namespace. See L<Chapter 5 - Configuration> for the proper options to use.

=head2 CHAPTER 10.2 - COLLECTORS

Fennec allows for forking and parallelization, it needs a way to send results
from the child processes to the parent. This is the role of the collector.
Collector plugins can be found in the Fennec::Collector:: namespace. See
L<Chapter 5 - Configuration> for the proper options to use.

The default collector is based on temporary files. This collector is the most
cross platform IPC available. A Socket based collector is in the works.

=head1 AUTHORS

Chad Granum L<exodist7@gmail.com>

=head1 COPYRIGHT

Copyright (C) 2010 Chad Granum

Fennec is free software; Standard perl licence.

Fennec is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.  See the license for more details.
