@node Search/Base, Search/Dict, Safe, Module List
@unnumbered provide framework for multiple searches


@unnumberedsec NAME

Search.pm - provide framework for multiple searches

@unnumberedsec SYNOPSIS

@example
use Search::TextSearch;
use Search::Glimpse;
@end example

@unnumberedsec DESCRIPTION

This module is a base class interfacing to search engines. It defines an
interface that can be used by any of the Search search modules, such as
Search::Glimpse or Search::TextSearch, which are the standard ones included
with the module.

It exports no routines, just provides methods for the other classes.

@unnumberedsubsec Search Comparisons

@example
Text           Glimpse 
                        -------        --------
 Speed                   Medium        Dependent on uniqueness of terms
 Requires add@'l software No            Yes   
 Search individually     Yes           Yes
  for fields               
 Search multiple files   Yes           Yes
 Allows spelling errors  No            Yes
 Fully grouped matching  No            No
@end example

@unnumberedsec Methods

Only virtual methods are supported by the Search::Base class. Those search
modules that inherit from it may export static methods if desired. There are
the @samp{Global Parameter Method} in this node, @samp{Column Setting Functions} in this node, and 
@samp{Row Setting Functions} in this node. @samp{SEE ALSO} in this node Search::Glimpse, Search::TextSearch.

@unnumberedsubsec Global Parameter Method

@example
$s->global(param,val); 
$status = $s->global(param); 
%globals = $s->global();
@end example

Allows setting of the parameters that are global to the search. The standard
ones are listed below.

@table @asis
@item base_directory
Those engines which look for matches in index files can read this
to get the base directory of the images.

@item case_sensitive
This is a global version of the cases field. If set, the search
engine should return only matches which exactly match, including
distinction between lower- and upper-case letters.  The default is not
set, to ignore case.

@item error_page
A page or filename to be displayed when a user error occurs.
Passed along with a single string parameter to the error_routine,
as in:
  
     &@{$self->@{global@}->@{error_routine@}@}
        ($self->@{global@}->@{error_page@}, $msg);

@item error_routine
Reference to a subroutine which will send errors to the user. The
default is @'\&croak@'.

@item first_match
The number of the first match returned in a of more_matches. This
tells the calling program where to start their increment.

@item history
The number of searches to keep in the history.

@item head_skip
Used for the TextSearch module to indicate lines that should be skipped
at the beginning of the file. Allows a header to be skipped if
it should not be searched.

The number of searches to keep in the history.

@item index_delim
The delimiter character to terminate the return code in an ascii
index file. In Search::Glimpse and Search::TextSearch, the default
is "\t", or TAB.  This is also the default for @{return_delim@}
if that is not set.

If field-matching is being used, this the character/string used
for splitting the fields.  If properly escaped, and @{return_delim@}
is used for joining fields, it can be a regular expression -- Perl
style.

@item index_file
A specification of an index file (or files) to search.  The usage
is left to the module -- it could, for example, be an anonymous array
(as in Search::TextSearch) or wild-card specification for multiple indices
(as in Search::Glimpse).

@item log_routine
A reference to a subroutine to log an error or status indication.
By default, it is @'\&Carp::carp@';

@item match_limit
The number of matches to return.  Not to be confused with max_matches,
at which number the search will terminate. Additional matches will be
stored in the file pointed to by save_dir, session_id, and search_mod.
The default is 50.

@item matches
Set by the search routine to indicate the number of matches in the
last search. If the engine can return the total number of matches
(without the data) then that is the result.

@item min_string
The minimum size of search string that is supported. Using a size of less
than 4 for Glimpse, for example, is not wise.  

@item next_pointer
The pointer to the next list of matches. This is for engines that
can return only a subset of the elements starting from an element.
For making a next match list.

@item next_url
The base URL that should be used to invoke the more_matches
function. Provided as an object-contained scratchpad value for
the calling routine -- it will not be used or modified by Search::Base.

There are a couple of useful ways to use this to invoke the 
proper more_matches search that are shown in the example
search CGI provided with this module set. Both involve setting
the next_url and combining it with a random session_id and
search_mod.

@item or_search
If set, the search engine should return matches which match any of the
search patterns.  The default is not set, requiring matches to all
of the keywords for a match return.

@item overflow
Set by the search routine if it matched the maximum number before reaching
the end of the search. Set to undef if not supported.

@item record_delim 
This sets the type of record that will be searched. For the
Search::TextSearch module, this is the same as Perl -- in fact,
the default is to use $/ (at the time of object creation) as
the default.

For the Search::Glimpse module, the following mappings occur:

@example
$/ = "\n"        Glimpse default
$/ = "\n\n"      -d @'$$@'
$/ = ""          -d @'$$@'
$/ = undef       -d @'NeVAiRbE@'
anything else    passed on as is (-d @'whatever@')
@end example

One useful pattern is @'^From @', which will make each email
message in a folder a separate record.

If you are doing this, and expect to be doing field returns,
it will probably be useful to set "\n\n" or "\n" as the
default index_delim. If used in combination with the obscure
anonymous hash definition of return_fields, you can
search and return mail headers on each message that matches.

@item return_delim
The delimiter character to join fields that are cut out of
the matched line/paragraph/page. The default is to set it
to be the same as @{index_delim@} if not explicitly set.

@item return_fields 
The fields that will be returned from the line/paragraph/page
matched.  This is not to be confused with the fields setting --
it will not affect the matching, only the returned fields.
The default (when it is undefined) is to return only the first
field.  There are several options for this field.

If the value is an ARRAY reference, an integer list of the columns to be
returned is assumed.

If the value is a HASH reference, then all words found AFTER the
@emph{key} of the hash (with assumed but not required whitespace as a separator)
up to the value of the item (used as a delimiter).  The following  example
will print the value of the From:, Reply-to: and Date: headers from any
message in your (UNIX) system mailbox that contains @'foobar@'.

@example
$s = new Search::TextSearch
        return_fields => @{
                        From: => "\n",
                        Reply-To: => "\n",
                        Date: => "\n",
                        @},
        record_delim    => "\nFrom ",
        search_file     => $ENV@{MAIL@};
@end example

@example
print $s->search(@'foobar@');
@end example

@item return_file_name
All return options will be ignored, and the file names of any matches
will be returned instead. The limit match-to-field routines are still
enabled for Search::TextSearch, but not for Glimpse, since the @'glimpse -l@'
option is used for that.

@item save_dir
The directory that search caches (for the more_matches function)
will be saved in.  Only applies to file save mode.

@item search_history 
An anonymous array [size history] of @'unevaled@' strings that contain
the search history.  When evaled by the Search::Base module, will
create an environment in the object that duplicates previous searches.
It is up to the application to save the value.

@item search_mod
This is used to develop a unique search save id for a user with
a consistent session_id.  For the more_matches function.

@item search_port
The port (passed to glimpse with the @strong{-K} option) that is to be
used for a network-attached search server.

@item search_server
The host name of a network-attached search server, passed to glimpse
with the @strong{-J} option.

@item session_id
This is used to determine the save file name or hash key used to
cache a search (for the more_matches) function.

@item speed
The speed of search desired, in an integer value from one to 10. Those engines
that have a faster method to search (possibly at a cost of comprensivity) can
adjust their routines accordingly.

@item spelling_errors
Those engines that support "tolerant matching" can honor this parameter
to set the number of spelling errors that they will allow for.
@emph{This can slow search dramatically on big databases.} Ignored by
search engines that don@'t have the capability.

@item substring_match
If set, the search engine should return partial, or substring, matches.
The default is not set, to indicate whole word matching.
@emph{This can slow search dramatically on big databases.} 

@item uneval_routine
A reference to a subroutine to save the search history to a cache.
By default, it is @'\&uneval@', the routine supplied with Search::Base.

@end table
@unnumberedsec METHODS

@unnumberedsubsec Virtual methods provided

@table @asis
@item more_matches
Given a file with return codes from previous searches, one per line,
returns an array with the correct matches in the array.  Opens the
file in directory save_dir, with session information appended
(the session_id and search_mod), and returns match_limit matches,
starting at next_pointer.

@item search
This is the main method defined in the individual search engine.
You can submit a single parameter for a quick search, which will
be interpreted as the one and only search specification, overriding
any settings residing in the specs array.  Options can be specified
at object creation, or separately with the @emph{global} method. Or a
@emph{search_spec} can be specified, which will temporarily override
the setting in specs (for that invocation only).

Otherwise, the parameters are named search options as documented
above.  Examples:

@example
# Simple search with default options for @'foobar@' in the named file
$s = new Search::TextSearch search_file => @'/var/adm/messages@');
@@found = $s->search(@'foobar@');
@end example

@example
# Search for @'foobar@' in /var/adm/messages, return only fields 0 and 2
# where fields are separated by spaces
$s = new Search::TextSearch;
@@found = $s->search( search_file   => @'/var/adm/messages@',
                     search_spec   => @'foobar@',
                     return_fields => [0,2],
                     return_delim  => @' @',
                     index_delim   => @'\s+@'                 );
@end example

@example
# Search for @'foobar@' in any file containing @'messages@' in
# the default glimpse index, return the file names
$s = new Search::Glimpse;
@@found = $s->search( search_spec   => @'foobar@',
                     search_file   => @'messages@',
                     return_file_name  => 1,       );
@end example

@example
# Same as above except use glimpse index located in /var directory
$s = new Search::Glimpse;
@@found = $s->search( search_spec       => @'foobar@',
                     base_directory    => @'/var@',
                     search_file       => @'messages@',
                     return_file_name  => 1,       );
@end example

@example
# Search all files in /etc
# Return file names with  lines that have @'foo@' in field 1
# and @'bar@' in field 3, with case sensitivity
# (using the default field delimiter of \t)
$s = new Search::TextSearch;
$s->rowpush(@'foo@', 1);
$s->rowpush(@'bar@', 3);
chop(@@files = @`ls /etc@`);
@@found = $s->search( search_file   => [@@files],
                     case_sensitive  => 1,
                     return_file_name  => 1,       );
@end example

@example
# Same as above using direct access to specs/fields
$s = new Search::TextSearch;
$s->specs(@'foo@', @'bar@');
$s->fields(1, 3);
chop(@@files = @`ls /etc@`);
@@found = $s->search( search_file   => [@@files],
                     case_sensitive  => 1,
                     return_file_name  => 1,       );
# Repeat search with above settings, except for specs,
# if less than 4 matches are found
if(@@found < 4) @{
    @@found = $s->search(@'foo@');
@}
@end example

@end table
@unnumberedsubsec Column Setting Methods

Column setting functions allow the setting of a series of columns of
match criteria:

@example
$search->specs(@'foo@', @'foo bar@', @'foobar@');
$search->fields(1, 3, 4);
@end example

This is an example for the specs and fields match criteria, which
are the search specifications and  the the fields to search,
respectively. Similar functions are provided for mods, links,
cases, negates, open_parens, and close_parens.

For the included Search::Glimpse and Search::TextSearch modules, an item will
match the above example only if field (or column) 1 contains @'foo@',
field 3 contains @'foo@' and/or @'bar@', and field 4 contains foobar.  The
setting of the case_sensitive, or_search, and substring_match terms will
be honored as well.

For simple searches, only one term need be set, and the grouping
functions links, open_parens, and close_parens are ignored.
In most cases, if the setting for a particular column is not defined
for a row, the value in the global setting is used.

@table @asis
@item specs
The search text, raw, per field. This is the only item that
necessarily needs to be set to do a search.

If more than one specification is present, there are three forms of
behavior supported by the included Search::TextSearch and Search::Glimpse
modules. First, if there are multiple search specifications, they are
combined together, just as they would if separated by spaces (and not
quoted).  Second, if the number of specs matches the number of
fields, each spec must match the field that it is associated with
(subject to the or_search and case_sensitive settings within that
field). Last, if there are more fields than specs, only the columns
in fields are searched for the combined specs.

@item fields
The column @strong{numbers} to search, where a column is a field separated by
index_delim. In the Search::TextSearch and Search::Glimpse modules,
this becomes operative in one of two ways. If the number of specs
match the number of fields, each specification is separately checked
against its associated field.  If the number of fields is different from
the number of specs, all specs are applied, but only to the text in
the specified fields.  Both first match on all of the text in the row,
then filter the match with another routine that checks for matches in
the specified fields.

@item mods        
Modifies the match criteria. Recognized modifications might be:

@example
start   Matches when the field starts with the spec
sub     Match substrings.
@end example

Not supported in the included modules.

@item links
The link to the previous row. If there are two fields to search, with
two different specs, this determines whether the search is AND, OR,
or NEAR.  For engines that support it, NEAR matches with in
$self->global(@'near@') words of the previous word (forward only). Not
supported in the included modules.

@item cases
For advanced search engines that support full associative
case-sensitivity.  Determines whether the particular match in this set
will be case-sensitive or not.  If the search engine doesn@'t support
independent case-sensitivity (like the Search::TextSearch and Search::Glimpse
modules), the value in or_search will be used. Not supported in
the included modules.

@item negates
Negates the sense of the match for that term.  Allows searches like 
"spec1 AND NOT spec2" or "NOT spec1 AND spec2". Not supported in
the included modules.

@item open_parens
Determines whether a logical parentheses will be placed on the left 
of the term. Allows grouping of search terms for more expressive matching,
i.e. "(AUTHOR Shakespeare AND TYPE Play ) NOT TITLE Hamlet". Not supported in
the included modules.

@item close_parens
Determines whether a logical parentheses will be placed on the right
of the term. Not supported in the included modules.

@end table
@unnumberedsubsec Row Setting Methods

Row setting functions allow the setting of all columns in a row.

@example
$query->rowpush($field,$spec,$case,$mod,$link,$left,$right);
($field,$spec,$case,$mod,$link,$left,$right) = $query->rowpop();
@@oldvals = $query->rowset(n,$field,$spec,$case,$mod,$link,$left,$right);
@end example

You can ignore the trailing parameters for a simple search. For example:

@example
$field = @'author@';
$spec = @'forsythe@';
$limit = 25;
$query = new Search::Glimpse;
$query->rowpush($field,$spec);
@@rows = $query->search( match_limit => 25 );
@end example

This searches the field @'author@' for the name @'forsythe@', with all other
options at their defaults (ignore case, match substrings, not negated, no
links, no grouping), and will return 25 matches (sets the matchlimit global).
For a more complex search, you can add the rest of the parameters as needed.

@unnumberedsec SEE ALSO

glimpse(1), Search::Glimpse(3L), Search::TextSearch(3L)

@unnumberedsec AUTHOR

Mike Heins, <mikeh@@iac.net>

