=head1 NAME

Fennec::Manual::Developer - Developer manual for extending or enhancing Fennec.

=head1 DESCRIPTION

Guide to developing plugins or enhancements for Fennec.

=head1 CHAPTER 1 - INNER WORKINGS

Fennec starts by loading a runner, even in standalone. This runner does the
following:

=head3 STARTUP PHASE

=over 4

=item Gather process information

=item Take over Test::Builder if needed

=item Start a collector to gather results

=item Initialize output handlers

=item Create a fork controller aka threader (NOT USING I-THREADS)

Threader should not be confused with perls horrible threading.

=back

=head3 TESTING PHASE

The following is run for each test file the runner finds. For standalone tests
it occurs only once; for only the one file. The runner may do this in multiple
processes, one per file.

=over 4

=item Run test file

Creates test groups and workflow objects, as well as running any package level
asserts.

=item Process Workflows

Workflow codeblocks are run as methods and produce additional test groups.

=item Run all test groups from all workflows

All the test groups defined in the file and the workflows are randomized and
run as methods. The file process may run multiple groups at once in several
processes.

=back

=head3 CLEANUP PHASE

The cleanup phase tears everything down after tests have been run.

=over 4

=item Wait on child processes

=item Shutdown output handles

=back

=head1 CHAPTER 2 - NAMESPACES

=over 4

=item Fennec

This is the primary Fennec namespace.

=item Fennec::Workflow

This is the namespace for workflow implementations. Things like RSPEC an Case
live here.

=item Fennec::Util

Used for misc utility items.

=item Fennec::TestSet

TestSet is the class used to implement test group methods, and any workflow
test grouping.

=item Fennec::TestFile

The base class for all Fennec test classes. Custom file types may subclass this
for their own base classes.

=item Fennec::Parser

Namespace for Devel::Declare Magic.

=item Fennec::Output

This is where output objects such as results, diag, etc. live. These are the
objects that get serialized and sent to the handlers in the runner.

=item Fennec::Handler

This is where output handlers go. These objects take the output objects and
present them to the user in a useful way such as TAP.

=item Fennec::FileType

The namespace for filetype classes which are used to load test files.

=item Fennec::Collector

Namespace for collectors which funnel results to the parent thread.

=item Fennec::Base

This is where various base classes go.

=item Fennec::Assert

This is the namespace for libraries of assert functions.

=back

=head1 CHAPTER 3 - TESTING FENNEC ITSELF

Fennec provides a module that makes it easy to test itself. Anyone who has
written libraries for Test::Builder has probably used the nightmare that is
L<Test::Builder::Tester>. Let me assure you Fennec learned from this nightmare
and does not repeat it.

Fennec provides L<Fennec::Assert::Interceptor> which can override the collector
and intercept result objects without sending them to the main process. This
library provides the capture() function for this task. Here is a snippet from
t/Fennec/Assert/Core/Simple.pm showing its use:

    package TEST::Fennec::Assert::Simple;
    use strict;
    use warnings;
    use Fennec asserts => [qw/ Core Interceptor /];

    tests 'ok' => sub {
        my $output = capture {
            ok( 1, 'pass' );
            ok( 0, 'fail' );
            ok( 1 );
            ok( 0 );
        };

        is( @$output, 4, "4 results" );

        is( $output->[0]->pass, 1, "passed" );
        is( $output->[0]->name, 'pass', 'name' );

        is( $output->[1]->pass, 0, "failed" );
        is( $output->[1]->name, 'fail', 'name' );

        is( $output->[2]->pass, 1, "passed" );
        is( $output->[2]->name, 'nameless test', 'name' );

        is( $output->[3]->pass, 0, "failed" );
        is( $output->[3]->name, 'nameless test', 'name' );
    };

Each item will be one of the subclasses of L<Fennec::Output>, such as
L<Fennec::Output::Result> and L<Fennec::Output::Diag>.

=head1 CHAPTER 4 - WRITING ASSERTION LIBRARIES

Assertion libraries are the most common testing tool type written. Fennec makes
the task of writing assertion libraries very simple. Fennec provies utilities
that let you focus on your assertion library instead of worrying about the
framework details.

=head2 CHAPTER 4.1 - WRITING NEW LIBRARIES

An assertion library must first import L<Fennec::Assert>, this will turn the
calling package into an L<Exporter::Declare> subclass. This will also provide
several helper functions:

=over 4

=item util

Used to export functions that will not generate result objects.

All of the following are valid:

    # Export and define seperately
    util 'my_util';
    sub my_util { ... }

    # export and declare together
    util my_util => sub { ... };

    # export and declare using Devel::Declare shortcuts
    util my_util { ... }

    # export and declare attaching Devel::Declare magic
    util my_util PARSER { ... }

=item tester

Used to export functions that generate results. This will wrap your function
with code to provide diagnostics, line numbers, package names, and other
contextal information you dont want figure out yourself.

    # Export and define seperately
    tester 'my_assert';
    sub my_assert { ... }

    # export and declare together
    tester my_assert => sub { ... };

    # export and declare using Devel::Declare shortcuts
    tester my_assert { ... }

    # export and declare attaching Devel::Declare magic
    tester my_assert PARSER { ... }

=item result

Use this function in your asserts to generate a result object. It will build
the result object, add context, and send the result to the collector.

In most cases you simply provide a name and a boolean:

    result(
        pass   => $BOOL,
        name   => $NAME,
    );

Here are all options:

    result(
        # Required:
        pass => $BOOL,

        # Optional:
        name   => $NAME,
        stderr => \@ERRORS,
        stdout => \@NOTES,

        # Filled in automatically, listed here so you can override
        line           => $LINE_NO,
        file           => $FILE_NAME,
        todo           => $REASON,
        skip           => $REASON,
        testset_name   => $NAME_OF_TEST_GROUP,
        workflow_stack => \@NAMES,
    );

Only specify file and line if Fennec gets it wrong. Deducing these is not
something you want to figure out yourself.

=item diag

Messages that should always be displayed.

    diag( $message1, $message2, @other_messages )

=item note

Messages that should be displayed in verbose mode.

    note( $message1, $message2, @other_messages )

=item test_caller

Returnes a hash with line and file as keys. Used to find context for results.

    my %context = test_caller();

=item tb_wrapper

    Used to improve Test::Builder asserts. See Chapter 4.2 for more details

=back

=head4 COMPLETE EXAMPLE

    package Fennec::Assert::MyAssert;
    use strict;
    use warnings;
    use Fennec::Assert;

    tester my_ok {
        my ( $ok, $name ) = @_;
        result(
            pass => $ok ? 1 : 0,
            name => $name || 'nameless test',
        );
    }

    util log_time {
        diag( "Time: " . time());
    }

    1;

=head2 CHAPTER 4.2 - WRAPPING TEST::BUILDER BASED TOOLS

Wrapping a Test::Builder library is exceedingly simple:

=over 4

=item use Fennec::Assert

=item import the functions you wish to wrap

=item use tb_wrapper() to wrap each function

=item export the resulting coderefs using tester() or util()

=back

=head4 EXAMPLE

    package Fennec::Assert::TBCore::More;
    use strict;
    use warnings;

    use Fennec::Assert;
    use Fennec::Output::Result;
    require Test::More;

    our @LIST = qw/ ok is isnt like unlike cmp_ok can_ok isa_ok new_ok pass fail
                    use_ok require_ok is_deeply /;

    for my $name ( @LIST ) {
        no strict 'refs';
        next unless Test::More->can( $name );
        tester( $name => tb_wrapper( \&{ 'Test::More::' . $name }));
    }

    1;

=head1 CHAPTER 5 - WRITING CUSTOM OUTPUT HANDLERS

An output handler must subclass Fennec::Handler. The baseclass provides a
generic constructor which you may override if you wish. You can also hook into
construction by writing an init method. Handlers are constructed with no
arguments.

The primary method to implement is handle( $output ), here is a complete list
of methods that you may want to implement:

=over 4

=item handle( $output )

Every output object generated is passed to each handler via this method. Every
item passed in will be a subclass of L<Fennec::Output>. You should familiarize
yourself with the 3 main output types, L<Fennec::Output::Result>,
L<Fennec::Output::Diag>, L<Fennec::Output::Note>. You should handle these three
types as well as handle unknown types generically.

=item finish()

This is called with no arguments when testing is complete and no more results
will be recieved. This is your last chance to do anything.

=item start()

This is called when Fennec starts testing.

=item starting_file( $filename )

This will be called whenever a process starts running a new test file.

=item fennec_error( @errors )

This is called when fennec encounters an error not related to testing.

=back

=head4 EXAMPLE

This is an example handler that ignores everything but errors, which it writes
to a file.

    package Fennec::Handler::MyHandler;
    use strict;
    use warnings;

    use base 'Fennec::Handler';

    use Fennec::Util::Accessors;
    Accessors qw/ fh count files /;

    sub start {
        my $self = shift;
        open( my $fh, ">", "failures.log" ) || die( "Error opening failure log: $!" );
        $self->fh( $fh );
        $self->count( 0 );
        $self->files( {} );
    }

    sub handle {
        my $self = shift;
        my ( $item ) = @_;
        return unless $item
               && $item->isa( 'Fennec::Output::Result' )
               && !$item->pass;

        $self->count( $self->count + 1 );
        my $file = $item->file;
        $file =~ s|.*(perl_lib/)|$1|g;
        $self->files->{ $file }++;

        my $fh = $self->fh;
        print $fh <<EOT
    ===========================================================
    File: @{[ $item->file ]}
    Test: @{[ $item->name() ]}
    Workflow Stack: @{[ join( ', ', @{ $item->workflow_stack() || [] })]}
    Err:
    @{[ join( "\n", @{ $item->stderr || [] })]}
    EOT
    }

    sub finish {
        my $self = shift;
        my $fh = $self->fh;
        print $fh "=======END========\n";
        print $fh "$_: " . $self->files->{$_} . "\n" for keys %{ $self->files };
        print $fh "\nTotal Errors: " . $self->count . "\n";
        close( $fh );
    }

    sub fennec_error {
        my $self = shift;
        my $fh = $self->fh;
        print $fh "=====FENNEC ERRORS====\n", @_;
    }

    1;

=head1 CHAPTER 6 - WRITING CUSTOM COLLECTORS

The collector is an abstracted IPC interface. All collectors must subclass
L<Fennec::Collector>. There are 2 methods to implement:

=over 4

=item write( $output )

This method takes a single output object. This method needs to pass the object
to the main thread.

=item @outputs = cull()

This should return a list of output objects collected from the child processes.

=back

You may also override these, but you must remember to call SUPER.

=over 4

=item start()

Called in the parent process to initialize the handlers.

=item finish()

Called in the parent process to shutdown the handlers.

=back

You should familiarize yourself with L<Fennec::Collector>, and avoid overriding
any other methods.

=head1 CHAPTER 7 - WRITING CUSTOM FILE TYPES

You must implement the following methods.

=over 4

=item $bool = $class->valid_file( $filename )

Return true if the given filename is a valid file of the type you expect.

=item $package = $obj->load_file()

Must load the test file. This must also make sure that a package name is
generated for the file. When finished the method must return the name of the
package for the test file. This package must also be a subclass of
Fennec::TestFile.

=item @list = $obj->paths()

A list of paths, relative to the project root, in which to search for test
files.

=back

=head4 TRUNCATED SYNOPSIS

    package Fennec::FileType::MyType;
    use strict;
    use warnings;

    use base 'Fennec::FileType';

    sub valid_file {
        my $class = shift;
        my ( $file ) = @_;
        ...
        return $bool;
    }

    sub load_file {
        my $self = shift;
        my $file = $self->filename;
        ...
        return $package;
    }

    sub paths {qw( t/ ... )}

    1;

=head1 CHAPTER 8 - WRITING CUSTOM WORKFLOWS

=head2 CHAPTER 8.1 - WHY WRITE A WORKFLOW

In fennec a workflow is a way to structure tests. There are many popular
testing frameworks that owe their popularity to how they structure their tests.
A notable example is Ruby's RSPEC. Structured tests can simplify your job as a
tester.

Within a workflow you can define child workflows, and groups of tests.
Ultimately your workflow will return a list of testset objects. You can be as
creative as you want in designing your workflow. You can also subclass
L<Fennec::TestSet> or use L<Fennec::TestSet::SubSet> to wrap groups with
buildup or teardown functions.

Using the L<Fennec::Workflow> system it was a minimal effort to write an RSPEC
like workflow. Less than 100 lines of code in SPEC.pm at the time this was
written. If you do not like how Fennec implements SPEC you can probably
implement it your own quite easily.

=head2 CHAPTER 8.2 - OVERVIEW

When a test file is loaded L<Fennec::Runner> creates a root workflow, this root
workflow is an L<Fennec::Workflow> object. Every testset or workflow defined in
the test file is a blessed method. The blessed methods are passed to a parent
workflow. Those defined outside of a workflow are passed to a 'root' workflow.

A test classes root workflow is stored inthe classes meta object. Calling
$test_class->fennec_meta() will return the meta object for a testclass. The
meta object has 2 methods for workflows: $meta->root_workflow() which always
returns the root workflow, and $meta->workflow() which returns the highest
workflow on the stack, also known as the 'current' workflow.

Any testsets or workflows defined inside the method of a parent workflow are
passed to that parent workflow. Each workflow object must inherit or
implement the add_item() method which takes a workflow or testset. Testsets
are retrieved from the workflows after they have been traversed to the deepest
level.

Example:

    package MyTest;
    use Fennec workflows => [ 'MyWorkflow' ];

    my_workflow parent => sub {
        tests 'parent tests' => {
            ...
        };
        my_workflow nested => sub {
            tests 'child tests' => {
                ...
            };
        };
    };

    1;

When Fennec runs it will load the file, it will create a root workflow, the
workflow sub named 'parent' will be blessed and passed to the root workflow as
a child workflow. The workflow sub 'nested' will be blessed as a workflow and
passed to 'parent'. While this is going on the testsets will also be blessed
and passed into their parent workflows.

=head2 CHAPTER 8.3 - CUSTOM WORKFLOW SYNOPSIS

    package Fennec::Workflow::MyWorkflow;
    use strict;
    use warnings;

    use Fennec::Workflow qw/:subclass/;

    build_hook { ... };

    build_with 'my_workflow';

    sub testsets {
        my $self = shift;
        ...
        return @testsets;
    }

    sub add_item {
        my $self = shift;
        my ($item) = @_;

        if ( we_handle( $item )) { ... }
        else { $self->SUPER::add_item( $item )}
    }

    1;

=head2 CHAPTER 8.4 - METHODS THAT CAN BE OVERRIDEN

Overriding these is optional, not overriding them will result in a workflow
that acts just like the root workflow.

=over 4

=item @testsets = $wf->testsets()

testsets() should return an array with 0 or more testset objects, or objects
that subclass testset. It is also responsible for returning the testsets from
child workflows. If this method does not get testsets from child workflows they
will not be run (which will generate a warning).

=item $wf->build()

The inherited method sets $wf as the current workflow, it then runs the method
that was blessed into as the workflow object. Overriding this is not
recommended, but may be necessary for some complicated workflows.

You should always call $self->built(1) at the end of your custom build()
method. Not doing this may cause problems with items being added to a workflow
AFTER build().

=item $wf->build_children()

This should rarely need to be overriden, calls $child->build() on all child
workflows. If you override add_items to add items other than workflows and
testsets, or to disallow adding items at all then you will probably want to
override this to reflect the change.

=item $wf->add_item( $item )

Add $item to the workflow. The inherited method will add workflows or testsets,
it will throw an exception for anything else. You can access the added items
via $self->_workflows() and $self->_testsets().

If you override this method you should die with a useful message if
$self->built() is true.

=back

=head2 CHAPTER 8.5 - HELPFUL FUNCTIONS

=over 4

=item build_with( $name )

=item build_with( $name, $class )

Exports a method ($name) that when called will create an instance of $class, or
the class in which build_with() was called. The exported method will take a
name and codeblock as arguments. After being build the instance will be added
to the current in which it was defined.

=item build_hook { ... }

=item build_hook( sub { ... })

Add code that should be run just after building the workflows and just before
running the tests.

=back

=head2 CHAPTER 8.6 - OTHER METHODS TO KNOW

=over 4

=item import()

L<Fennec::Workflow> has a complicated import() method, in order to simplify it
all classes that sublcass L<Fennec::Workflow> have a new import() method
exported to their package. It is important that you do not try to override
import(), or that you are at least aware that you cannot call
$wf->SUPER::import() and get the expected behavior. Defining your own import()
method will also throw a redefine warning.

=item @wfs = $wf->workflows()

Returns a list of all the workflows added as children.

=item @testsets = $wf->testsets()

Returns a list of all the testsets in this workflow, and all of its children.

=item $tf = $wf->testfile()

Returns the L<Fennec::TestFile> object currently being run.

=item $pwf = $wf->parent()

Returns the parent workflow object to which this one is a child, the root
workflow will return the TestFile object.

=item $testsets = $wf->_testsets()

=item $wf->_testsets( \@testsets )

Get/Set the list of testsets, if you override add_item() and never caller
SUPER::add_item() then you will need to manually add TestSets to the arrayref
returned by _testsets().

=item $workflows = $wf->_workflows()

=item $wf->_workflows( \@workflows )

Get/Set the list of workflows, if you override add_item() and never caller
SUPER::add_item() then you will need to manually add Workflows to the arrayref
returned by _workflows().

=item run_tests()

In a normal Fennec run this will only be called on the root Workflow object.
Overriding this in your subclass will have NO EFFECT.

=item $wf->add_items( @items )

Calls $wf->add_item() for each item in @items.

=back

=head1 CHAPTER 9 - RUNNER PLUGINS

It is also possible to create plugins that extend or modify the runner itself.
There are a few ways in which you can hook into the runner.

=head2 CHAPTER 9.1 - CONFIGURATION OPTIONS

L<Fennec::Runner> can export the add_config() function. This function allows
you to add configuration items to the runner. These items can be listed in
t/Fennec.t along with the usual ones. This will create the config option
itself, as well as an accessor on the runner singleton by which it can be
accessed.

=over 2

=item add_config $CONFIG_NAME

=item add_config $CONFIG_NAME => $default

=item add_config $CONFIG_NAME => ( %OPTIONS )

Options:

=over 4

=item default => $VALUE

=item default => sub { my ($data) = @_; ... }

Specify the default value when none is specified. Can either be scalar value
(including refs) or a coderef. The only argument provided to the coderef is the
$data hash. The $data hash contains all the options that have been calculated
so far for the runner.

You absolutely should not modify the $data hash in coderefs passed to this
parameter. The only fields guaranteed to be set are those listed in the depends
option.

=item env_override => $VALUE

Set to false in order to disable environment overrides to this parameter. When
not specified it defaults to true. When true (numeric) this parameter can be
overriden using the FENNEC_[NAME] envireonment variable.

You can also specify a string name that will be used instead of FENNEC_[NAME].

=item modify => sub { my ($value, $data) = @_; ... }

This hook can be used to modify the value the user provides. It can also modify
the default value that you provide. The first argument is the unaltered value.
The second argument is the $data hash. See the section for the 'default' option
for more info about $data.

=item depends => \@LIST

Arrayref of config options that must be calculated before this one.

=back

=back

=head2 CHAPTER 9.2 - TEST HOOKS

Test hooks are run after the root workflow has been instantiated, but before it
has been processed. Test hooks are coderefs that receive the runner and test
object as arguments.

L<Fennec::Runner> can export the add_test_hook() function. This function allows
you to add test hooks to the runner.

=over 2

=item add_test_hook( sub { my( $runner, $test ) = @_; ... })

=back

=head2 CHAPTER 9.3 - FINISH HOOKS

These are run just before the threader and collector are stopped. These are
coderefs that recieve no arguments. L<Fennec::Runner> can export the
add_finish_hook() function. This function allows you to add finnish hooks to
the runner.

=over 2

=item add_finish_hook(sub {...})

=back

=head1 AUTHORS

Chad Granum L<exodist7@gmail.com>

=head1 COPYRIGHT

Copyright (C) 2010 Chad Granum

Fennec is free software; Standard perl licence.

Fennec is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.  See the license for more details.
