NAME
    AnyEvent::Task - Client/server-based asynchronous worker pool

SYNOPSIS 1: PASSWORD HASHING
  Server
        use AnyEvent::Task::Server;
        use Authen::Passphrase::BlowfishCrypt;

        my $dev_urandom;
        my $server = AnyEvent::Task::Server->new(
                       listen => ['unix/', '/tmp/anyevent-task.socket'],
                       setup => sub {
                         open($dev_urandom, "/dev/urandom") || die "open urandom: $!";
                       },
                       interface => {
                         hash => sub {
                           my ($plaintext) = @_;
                           read($dev_urandom, my $salt, 16) == 16 || die "bad read from urandom";
                           return Authen::Passphrase::BlowfishCrypt->new(cost => 10,
                                                                         salt => $salt,
                                                                         passphrase => $plaintext)
                                                                   ->as_crypt;

                         },
                         verify => sub {
                           my ($crypted, $plaintext) = @_;
                           return Authen::Passphrase::BlowfishCrypt->from_crypt($crypted)
                                                                   ->match($plaintext);
                         },
                       },
                     );

        $server->run; # or AE::cv->recv

  Client
        use AnyEvent::Task::Client;

        my $client = AnyEvent::Task::Client->new(
                       connect => ['unix/', '/tmp/anyevent-task.socket'],
                     );

        my $checkout; $checkout = $client->checkout( timeout => 5, );

        $checkout->hash('secret',
          sub {
            my ($checkout, $crypted) = @_;

            print "Hashed password is $crypted\n";

            $checkout->verify($crypted,
              sub {
                my ($checkout, $result) = @_;
                print "Verify result is $result\n":
              });
          });

SYNOPSIS 2: DBI
  Server
        use AnyEvent::Task::Server;
        use DBI;

        my $dbh;

        my $server = AnyEvent::Task::Server->new(
                       listen => ['unix/', '/tmp/anyevent-task.socket'],
                       setup => sub {
                         $dbh = DBI->connect(...);
                       },
                       interface => sub {
                         my ($method, @args) = @_;
                         $args[0] = $dbh->prepare_cached($args[0]) if defined $args[0];
                         $dbh->$method(@args);
                       },
                     );

        $server->run; # or AE::cv->recv

  Client
        use AnyEvent::Task::Client;

        my $dbh_pool = AnyEvent::Task::Client->new(
                         connect => ['unix/', '/tmp/anyevent-task.socket'],
                       );

        my $username = 'jimmy';

        my $dbh = $dbh_pool->checkout;

        $dbh->selectrow_hashref(q{ SELECT email FROM user WHERE username = ? },
                                undef, $username,
          sub {
            my ($dbh, $row) = @_;
            print "User's email is $row->{email}\n";
            ## Use same $dbh here if using transactions
          });

DESCRIPTION
    WARNING: The above client examples don't implement error handling. See
    the "ERROR HANDLING" section for details on how to add this.

    The synopsis makes this module sounds much more complicated than it
    actually is. Worker processes are forked off by a server when a client
    needs one, and the client can communicate with many workers using
    asynchronous communication.

    Another way of saying it is that AnyEvent::Task is a fork-on-demand but
    persistent-worker server (AnyEvent::Task::Server) combined with an
    asynchronous interface to a request queue and pooled-worker client
    (AnyEvent::Task::Client).

    Both client and server are of course built with AnyEvent because it's
    awesome. However, workers can't use AnyEvent (yet).

    A server is started with "AnyEvent::Task::Server->new". This should at
    least be passed the "listen" and "interface" arguments. Keep the
    returned server object around for as long as you want the server to be
    running. "interface" is the code that should handle each request. See
    the interface section below for its specification. A "setup" coderef can
    be passed in to run some code when a new worker is forked. A
    "checkout_done" coderef can be passed in to run some code whenever a
    checkout is released (see below).

    A client is started with "AnyEvent::Task::Client->new". You only need to
    pass "connect" to this. Keep the returned client object around as long
    as you wish the client to be connected.

    After both the server and client are initialised, each process must
    enter AnyEvent's "main loop" in some way, possibly just "AE::cv->recv".

    In the client process, you may call the "checkout" method on the client
    object. This checkout object can be used to run code on a remote worker
    process in a non-blocking manner. The "checkout" method doesn't require
    any arguments, but "timeout" is recommended.

    You can treat a checkout object as an object that proxies its method
    calls to a worker process or a function that does the same. You pass the
    arguments to these method calls as an argument to the checkout object,
    followed by a callback as the last argument. This callback will be
    called once the worker process has returned the results. This callback
    will normally be passed two arguments, the checkout object and the
    return value. In the event of an exception thrown, an error is raised in
    the dynamic context of the callback (see the "ERROR HANDLING" section).

DESIGN
    Each client maintains a "pool" of connections to worker processes. Every
    time a checkout is issued, it is placed into a first-come, first-serve
    queue. Once a worker process becomes available, it is associated with
    that checkout until that checkout is garbage collected which in perl
    means as soon as it is no longer needed. Each checkout also maintains a
    queue of requests, so that as soon as this worker process is allocated,
    the requests are filled also on a first-come, first-served basis.

    "timeout" can be passed as a keyword argument to "checkout". Once a
    request is queued up on that checkout, a timer of "timout" seconds
    (default is 30, undef means infinity) is started. If the request
    completes during this timeframe, the timer is cancelled. If the timer
    expires, the worker connection is terminated and an exception is thrown
    in the dynamic context of the callback (see the "ERROR HANDLING"
    section).

    Note that since timeouts are associated with a checkout, the client
    process can be started before the server and as long as the server is
    started within "timeout" seconds, no requests will be lost. The client
    will continually try to acquire worker processes until a server is
    available, and once one is available it will attempt to fill all queued
    checkouts.

    Because of checkout queuing, the maximum number of worker processes a
    client will attempt to obtain can be limited with the "max_workers"
    argument when creating a client object. If there are more live checkouts
    than "max_workers", the remaining checkouts will have to wait until one
    of the other workers becomes available. Because of timeouts, some
    checkouts may never be serviced if the system can't handle the load
    (those requests should be sent "service temporarily unavailable"
    errors).

    The "min_workers" argument can be used to pre-fork "hot-standby" worker
    processes when creating the client. The default is 2 though note that
    this may change to 0 in the future.

STARTING THE SERVER
    Technically, running the server and the client in the same process is
    possible, but is highly discouraged since the server will "fork()" when
    the client desires a worker process. In this case, all descriptors in
    use by the client would be duped into the worker process and the worker
    would have to close all the extra- descriptors created by the client.
    Also, forking a busy client may be memory-inefficient.

    Since it's more of a bother than it's worth to run the server and the
    client in the same process, there is an alternate server constructor,
    "AnyEvent::Task::Server::fork_task_server". It can be passed the same
    arguments as the regular "new" constructor:

        ## my ($keepalive_pipe, $pid) =
        AnyEvent::Task::Server::fork_task_server(
          listen => ['unix/', '/tmp/anyevent-task.socket'],
          interface => sub {
                             return "Hello from PID $$";
                           },
        );

    The only differences between this and the regular constructor is that
    this will fork a process which becomes the server, and that it will
    install a "keep-alive" pipe between the server and the client. This
    keep-alive pipe will be used by the server to detect when the
    client/parent process exits.

    If "AnyEvent::Task::Server::fork_task_server" is called in a void
    context, then the reference to this keep-alive pipe is pushed onto
    @AnyEvent::Task::Server::children_sockets. Otherwise, the keep-alive
    pipe and the server's PID are returned. Closing the pipe will terminate
    the worker gracefully. Kill the PID to terminate it immediately.

    Since the "fork_task_server" constructor forks and requires using
    AnyEvent in both the parent and child processes, it is important that
    you not install any AnyEvent watchers before calling it. The usual
    caveats about forking AnyEvent applications apply (see AnyEvent docs).

INTERFACE
    There are two formats possible for the "interface" option when creating
    a server. The first (and most general) is a coderef. This coderef will
    be passed the list of arguments that were sent when the checkout was
    called in the client process (without the trailing callback of course).

    As described above, you can use a checkout object as a coderef or as an
    object with methods. If the checkout is invoked as an object, the method
    name is prepended to the arguments passed to "interface":

        interface => sub {
          my ($method, @args) = @_;
        },

    If the checkout is invoked as a coderef, method is omitted:

        interface => sub {
          my (@args) = @_;
        },

    The second format possible for "interface" is a hash ref. This is a
    simple method dispatch feature where the method invoked on the checkout
    object is the key used to lookup to which coderef to run in the worker:

        interface => {
          method1 => sub {
            my (@args) = @_;
          },
          method2 => sub {
            my (@args) = @_;
          },
        },

    Note that since the protocol between the client and the worker process
    is JSON-based, all arguments and return values must be serializable to
    JSON. This includes most perl scalars like strings, a limited range of
    numerical types, and hash/list constructs with no cyclical references.

    A future backwards compatible RPC protocol may use Storable or something
    else, although note that you can already serialise an object with
    Storable manually, send the resulting string over the existing protocol,
    and then deserialise it in the worker.

LOGGING
    Because workers run in a separate process, they can't directly use
    logging contexts in the client process. That is why this module is
    integrated with Log::Defer.

    A Log::Defer object is created on demand in the worker process and once
    the worker is done an operation, any messages in the object will be
    extracted and sent back to the client. The client then merges this into
    its main Log::Defer object that was passed in when creating the
    checkout.

    In your server code, use AnyEvent::Task::Logger. It exports the function
    "logger" which returns a Log::Defer object:

        use AnyEvent::Task::Server;
        use AnyEvent::Task::Logger;

        AnyEvent::Task::Server->new(
          listen => ['unix/', '/tmp/anyevent-task.socket'],
          interface => sub {
            logger->info('about to compute some operation');
            {
              my $timer = logger->timer('computing some operation');
              select undef,undef,undef, 1; ## sleep for 1 second
            }
          },
        )->run;

    Note: Portable server code should not call "sleep" because on some
    systems it will interfere with the recoverable worker timeout feature
    implemented with "SIGALRM".

    In your client code, pass a Log::Defer object in when you create a
    checkout:

        use AnyEvent::Task::Client;
        use Log::Defer;

        my $client = AnyEvent::Task::Client->new(
                       connect => ['unix/', '/tmp/anyevent-task.socket'],
                     );

        my $log_defer_object = Log::Defer->new(sub {
                                                 my $msg = shift;
                                                 use Data::Dumper; ## or whatever
                                                 print Dumper($msg);
                                               });

        $log_defer_object->info('going to compute some operation in a worker');

        my $checkout = $client->checkout(log_defer_object => $log_defer_object);

        my $cv = AE::cv;

        $checkout->(sub {
          $log_defer_object->info('finished some operation');
          $cv->send;
        });

        $cv->recv;

    When run, the above client will print something like this:

        $VAR1 = {
              'start' => '1363232705.96839',
              'end' => '1.027309',
              'logs' => [
                          [
                            '0.000179',
                            30,
                            'going to compute some operation in a worker'
                          ],
                          [
                            '0.023881061050415',
                            30,
                            'about to compute some operation'
                          ],
                          [
                            '1.025965',
                            30,
                            'finished some operation'
                          ]
                        ],
              'timers' => {
                            'computing some operation' => [
                                                            '0.024089061050415',
                                                            '1.02470206105041'
                                                          ]
                          }
            };

ERROR HANDLING
    If you expected some operation to throw an exception, in a synchronous
    program you might wrap it in "eval" like this:

        my $crypted;

        eval {
          $crypted = hash('secret');
        };

        if ($@) {
          say "hash failed: $@";
        } else {
          say "hashed password is $crypted";
        }

    But in an asynchronous program, typically the "hash" operation would
    initiate some kind of asynchronous operation and then return
    immediately. The error might come back at any time in the future, in
    which case you need a way to map the exception that is thrown back to
    the $conn object.

    AnyEvent::Task accomplishes this mapping with Callback::Frame. For
    example:

        use Callback::Frame;

        frame(code => sub {

          $client->checkout->hash('secret', sub {
            my ($checkout, $crypted) = @_;
            say "Hashed password is $crypted";
          });

        }, catch => sub {

          my $back_trace = shift;
          say "Error is: $@";
          say "Full back-trace: $back_trace";

        })->(); ## <-- frame is created and then executed

    Why not just an error callback supplied by AnyEvent::Task?

    Callback::Frame lets you pass dynamic state (like error handlers and
    dynamic variables) through nested chains of callbacks even if some of
    those callbacks don't support your error handling system.

    Additionally, Callback::Frame provides an error handler stack so you can
    have nested error handlers (similar to nested "eval"s). This is useful
    when you wish to have a top-level "bail-out" error handler and also
    nested error handlers that know how to retry or recover from an error in
    an async sub-operation.

    Callback::Frame is not tied to AnyEvent::Task, AnyEvent or any other
    async framework and can be used with almost all most callback-based
    libraries. But when using AnyEvent::Task, your libraries that you use in
    the client must be AnyEvent compatible.

    Instead of passing "sub { ... }" into libraries, pass in "fub { ... }".
    When invoked, the wrapped callback will first re-establish any error
    handlers installed with "frame", and then run your actual callback code.
    Note that it's important that all callbacks be created with "fub" (or
    "frame") even if you don't expect them to fail so that the dynamic
    context is preserved for nested callbacks that might.

    If you have a callback and wish to raise an exception inside its dynamic
    context, use "existing_frame":

        frame(existing_frame => $current_cb,
              code => sub {
          die $err;
        })->();

    In addition to errors that occur while running your callbacks,
    AnyEvent::Task uses this feature of Callback::Frame to throw errors if
    the worker process dies or times out.

COMPARISON WITH HTTP
    Why a custom protocol, client, and server? Can't we just use something
    like HTTP?

    It depends.

    AnyEvent::Task clients send discrete messages and receive ordered,
    discrete replies from workers, much like HTTP. The AnyEvent::Task
    protocol can be extended in a backwards compatible manner like HTTP.
    AnyEvent::Task communication can be pipelined and possibly in the future
    even compressed like HTTP.

    AnyEvent::Task servers (currently) all obey a very specific
    implementation policy: They are like CGI servers in that each process in
    that each process they fork is guaranteed to be handling only one
    connection at once so it can perform blocking operations without
    worrying about holding up other connections.

    But since a single process can handle many requests in a row without
    exiting, the AnyEvent::Task server is more like a FastCGI server. The
    difference however is that while a client holds a checkout it is
    guaranteed an exclusive lock on that process. With a FastCGI server it
    is assumed that requests are stateless so you can't necessarily be sure
    you'll get the same process for two consecutive requests. In fact, if an
    error is thrown in the FastCGI handler you may never get the same
    process back again, preventing you from being able to recover from the
    error, retry, or at least collect process state for logging reasons.

    The fundamental difference between the AnyEvent::Task protocol and HTTP
    is that in AnyEvent::Task the client is the dominant protocol
    orchestrator whereas in HTTP it is the server.

    In AnyEvent::Task, the client manages the worker pool and the client
    decides if/when the worker process should terminate. In the normal case,
    a client will just return the worker to its worker pool. A worker is
    supposed to accept commands for as long as possible until the client
    dismisses it.

    Client processes can be started and checkouts can be obtained before the
    server is even started. The client will continue to try to connect to
    the server to obtain worker processes until either the server starts or
    the checkout's timeout period lapses.

    The client decides the timeout for each checkout and different clients
    can have different timeouts while connecting to the same server.

    The client even decides how many minimum and maximum workers it requires
    at once. The server is really just a simple fork-on-demand server and
    most of the sophistication is in the asynchronous client.

SEE ALSO
    <The AnyEvent::Task github repo>

    AnyEvent::Task is integrated with Callback::Frame. In order to handle
    exceptions in a meaningful way, you will need to use this module.

    There's about a million CPAN modules that do similar things.

    This module is designed to be used in a non-blocking, process-based unix
    program. Depending on your exact requirements you might find something
    else useful: Parallel::ForkManager, Thread::Pool, an HTTP server of some
    kind, &c.

    If you're into AnyEvent, AnyEvent::DBI and AnyEvent::Worker (based on
    AnyEvent::DBI), and AnyEvent::ForkObject send and receive commands from
    worker processes similar to this module. AnyEvent::Worker::Pool also has
    an implementation of a worker pool. AnyEvent::Gearman can interface with
    Gearman services.

    If you're into POE there is POE::Component::Pool::DBI, POEx::WorkerPool,
    POE::Component::ResourcePool, POE::Component::PreforkDispatch,
    Cantella::Worker, &c.

BUGS
    This module is still being developed although the interface should be
    mostly stable.

AUTHOR
    Doug Hoyte, "<doug@hcsw.org>"

COPYRIGHT & LICENSE
    Copyright 2012-2013 Doug Hoyte.

    This module is licensed under the same terms as perl itself.

