##========================================================================
## POD DOCUMENTATION, auto-generated by podextract.perl

##========================================================================
## NAME
=pod

=head1 NAME

DiaColloDB::Client::list - diachronic collocation db: client: distributed

=cut

##========================================================================
## DESCRIPTION
=pod

=head1 DESCRIPTION

DiaColloDB::Client::list is a subclass of
L<DiaColloDB::Client|DiaColloDB::Client> for
accessing a set of distributed L<DiaColloDB|DiaColloDB> databases
via a C<list://> URL whose path part is a space- or colon-separated list
of sub-URLs supported by L<DiaColloDB::Client|DiaColloDB::Client>.
It supports the L<DiaColloDB::Client|DiaColloDB::Client> API
by calling the relevant methods on each of its sub-clients.

new() options and object structure:

 ##-- DiaColloDB::Client: options
 url  => $url,       ##-- list url (sub-urls, separated by whitespace, "+SCHEME://", or "+://")
 ##
 ##-- DiaColloDB::Client::list
 urls  => \@urls,     ##-- sub-urls
 opts  => \%opts,     ##-- sub-client options
 fudge => $fudge,     ##-- get ($fudge*$kbest) items from sub-clients (-1:all; 0|1:none; default=10)
 fork => $bool,       ##-- run each subclient query in its own fork? (default=if available)
 lazy => $bool,       ##-- use temporary on-demand sub-clients (true,default) or persistent sub-clients (false)
 extend => $boo,      ##-- use extend() queries to acquire correct f2 counts? (default=true)
 logFudge => $level,  ##-- log-level for fudge-coefficient debugging (default='debug')
 logThread => $level, ##-- log-level for thread operations (default='none')
 ##
 ##-- guts
 #clis => \@clis,     ##-- per-url sub-clients for "busy" (non-"lazy") mode

The most important client parameter is the fudge-coefficient option C<fudge=E<gt>$fudge>, which requests
that up to C<$fudge*$kbest> items be retrieved from sub-clients for each L<profile()|profile>
call.  If C<$fudge E<lt> 0>, all collocates will be retrieved from each sub-client,
and trimming will be performed exclusively by the superordinate DiaColloDB::Client::list object.
If C<$fudge == 0>, only the C<$kbest> collocates from each sub-client will be retrieved.
The default value of 10 should return reasonable results without too large of
a performance penalty in most cases, but be aware that the results for C<$fudge E<gt> 0> may not be strictly correct
due to sub-client local pruning; see L<|/KNOWN BUGS> for details.

This module supports parallel processing of sub-client queries using whatever
threading implementation (if any) is provided by the L<DiaColloDB::threads|DiaColloDB::threads> module.
Parallel sub-client processing is enabled by default if
a working
L<threads|threads> or L<forks|forks> module was found by
L<DiaColloDB::threads|DiaColloDB::threads>,
but can be disabled by specifying
the C<fork=E<gt>0> option to the list-client.

=head2 List URLs

List URLs passed as the the C<url> option to the constructor can be either ARRAY-refs
of sub-URLs or simple strings with an optional C<list://> scheme.
In the latter case, sub-URLs in the argument string are separated by whitespace
or by a plus character ("+") followed by the sub-URL scheme, e.g.:

 ["file://a","file://b"]        ##-- ARRAY-ref of explicit file URLs
 ["a"       , "b"      ]        ##-- ARRAY-ref of implicit file URLs
 
 "list://file://a file://b"     ##-- string with space-separated explicit file URLs
 "list://a b"                   ##-- string with space-separated implicit file URLs
 
 "list://file://a+file://b"     ##-- list with "+"-separated explicit file URLs
 "list://a+://b"                ##-- list with "+"-separated implicit file URLs

Options can be passed to the appropriate sub-URLs via those URLs' query strings,
as described in L<DiaColloDB::Client/open>.
Options to the DiaColloDB::Client::list object itself can be passed in by using
a sub-URL consisting of a HASH-ref or only a query string, e.g.:

 ["a","b",{fudge=>0}]           ##-- ARRAY-ref with local options as HASH-ref
 ["a","b","?fudge=0"]           ##-- ARRAY-ref with local options as query-string
 
 "list://a b ?fudge=0"          ##-- space-sparated string with local options
 "list://a+://b+://?fudge=0"    ##-- "+"-separated string with local options

=cut

##======================================================================
## Footer
##======================================================================
=pod

=head1 KNOWN BUGS

=head2 Incorrect Independent Collocate Frequencies

Prior to the introduction of L<extend()|DiaColloDB/extend> queries in
DiaCollODB v0.11.000, the list-clients were B<always>
apt to return incorrrect independent collocate frequencies I<f2> whenever
the queried subcorpora were not partitioned explicitly by date,
even with C<$fudge=-1>.
Although the reported joint frequencies I<f12>
ought to have been correct in this case, it could easily happen that the independent collocate
frequencies I<f2> got mis-reported, leading to incorrect computations of I<f2>-sensitive
association scores such as C<milf> (pointwise mutual information * log-frequency product),
C<ll> (log likelihood), or the default C<ld> (log Dice).  Such errors occurred whenever
the pre-v0.11.000 list client accessed multiple sub-clients (e.g. C<$a> and C<$b>) and some candidate
collocate C<$v> occured in both of the subcorpora, but only occured together with the target term C<$w>
in one of the sub-clients' indices.

Suppose C<$v> occurs in subcorpus C<$a> with frequency C<f_a($v)>
and in subcorpus C<$b> with frequency C<f_b($v)>, but only occurs together with
C<$w> in subcorpus C<$a> with frequency C<f_a($w,$v)>; i.e. C<f_b($w,$v)==0>.
Since only collocates with nonzero
co-occurrence frequencies are collected in subcorpus profiles, the sub-profile for C<$w>
over subcorpus C<$b> will not contain an entry for C<$v> at all.  This is fine if
we are only interested in the total co-occurrence frequency C<f($w,$v) = f_a($w,$v) + f_b($w,$v)>,
but if we are using an "interesting" association score, we also need to refer to the total
independent collocate frequency C<f($v) = f_a($v) + f_b($v)>, but
since C<f_b($v)> will not have been reported by the subprofile for corpus C<$b>,
its value will be treated as 0 (zero), leading to an incorrect estimate of the
association score.

As of v0.11.000, each list client is queried a second time using its
L<extend()|DiaColloDB/extend> method to acquire independent collocate frequencies
for "missing" keys such as C<$v> in the example above.  This introduces additional
processing overhead, which can be disabled by setting
the C<extend=E<gt>0> option to the list-client to simulate the old, incorrect,
pre-v0.11 behavior.

=head2 Incorrect Joint Frequencies

Similar to the case for L<independent collocate frequencies|/Incorrect Independent Collocate Frequencies>,
the joint frequencies I<f12> reported by this module prior to v0.12.016 were incorrect
whenever the pre-v0.12.016 list client accessed multiple sub-clients (e.g. C<$a> and C<$b>), and some candidate
collocate C<$v> occured together with the target term C<$w> in both sub-clients' indices, but
was among the C<$fudge*$kbest> items per epoch for only one of the sub-clients.

The L<extend()|DiaColloDB/extend> method was re-implemented in v0.12.016 to perform a full
L<profile()|DiaColloDB/profile> on the "missing" candidate collocates, ensuring correct
acquisition of both joint and independent collocate frequencies.

=cut

##======================================================================
## Footer
##======================================================================
=pod

=head1 AUTHOR

Bryan Jurish E<lt>moocow@cpan.orgE<gt>

=head1 COPYRIGHT AND LICENSE

Copyright (C) 2015-2020 by Bryan Jurish

This package is free software; you can redistribute it and/or modify
it under the same terms as Perl itself, either Perl version 5.14.2 or,
at your option, any later version of Perl 5 you may have available.

=head1 SEE ALSO

L<DiaColloDB::Client(3pm)|DiaColloDB::Client>,
L<DiaColloDB(3pm)|DiaColloDB>,
L<perl(1)|perl>,
...



=cut
