HTML-Parser
-----------

This is a collection of modules that parse HTML text documents. These
modules used to be part of the libwww-perl distribution, but are now
unbundled in order to facilitate a separate development track.  Bug
reports and discussions about these modules can still be sent to the
<libwww-perl@ics.uci.edu> mailing list.  Remember to also take a look at
the HTML-Tree module collection that create and extract information from
HTML syntax trees.

The modules present in this collection are:

  HTML::Parser - The parser base class.  It receives arbitrary sized
        chunks of the HTML text and will tokenize it by calling
        appropriate methods on itself.

  HTML::Entities - Provides functions to encode and decode text with
        embedded HTML &lt;entities&gt;.

  HTML::Filter - An HTML::Parser subclass that filters HTML text. You
        will need to make a subclass if you want it to do more than
        cat(1).

  HTML::HeadParser - A lightweight HTML::Parser subclass that extract
        information from the <HEAD> section of an HTML document.

  HTML::LinkExtor - An HTML::Parser subclass that extract links from
        an HTML document.

  HTML::TokeParser - An alternative interface to the basic parser
        that does not require subclassing.



PREREQUISITES

In order to install and use this package you will need Perl version
5.004 or better.  If you intend to use the HTML::HeadParser you need to
have the libwww-perl distribution installed.



INSTALLATION

Just follow the usual procedure:

   perl Makefile.PL
   make
   make test
   make install



COPYRIGHT

  © 1995-1998 Gisle Aas. All rights reserved.

This library is free software; you can redistribute it and/or modify
it under the same terms as Perl itself.




-------------------------------------------------------------------------

NAME
    HTML::Parser - SGML parser class

SYNOPSIS
     require HTML::Parser;
     $p = HTML::Parser->new;  # should really a be subclass
     $p->parse($chunk1);
     $p->parse($chunk2);
     #...
     $p->eof;                 # signal end of document

     # Parse directly from file
     $p->parse_file("foo.html");
     # or
     open(F, "foo.html") || die;
     $p->parse_file(\*F);

DESCRIPTION
    The `HTML::Parser' will tokenize an HTML document when the
    parse() method is called by invoking various callback methods.
    The document to be parsed can be supplied in arbitrary chunks.

    The external interface the an *HTML::Parser* is:

    $p = HTML::Parser->new
        The object constructor takes no arguments.

    $p->parse( $string );
        Parse the $string as an HTML document. Can be called
        multiple times. The return value is a reference to the
        parser object.

    $p->eof
        Signals end of document. Call eof() to flush any remaining
        buffered text. The return value is a reference to the parser
        object.

    $p->parse_file( $file );
        This method can be called to parse text from a file. The
        argument can be a filename or an already opened file handle.
        The return value from parse_file() is a reference to the
        parser object.

    $p->strict_comment( [$bool] )
        By default we parse comments similar to how the popular
        browsers (like Netscape and MSIE) do it. This means that
        comments will always be terminated by the first occurence of
        "-->". This is not correct according to the "official" HTML
        standards. The official behaviour can be enabled by calling
        the strict_comment() method with a TRUE argument.

        The return value from strict_comment() is the old attribute
        value.

    In order to make the parser do anything interesting, you must
    make a subclass where you override one or more of the following
    methods as appropriate:

    $self->declaration($decl)
        This method is called when a *markup declaration* has been
        recognized. For typical HTML documents, the only declaration
        you are likely to find is <!DOCTYPE ...>. The initial "<!"
        and ending ">" is not part of the string passed as argument.
        Comments are removed and entities will not be expanded.

    $self->start($tag, $attr, $attrseq, $origtext)
        This method is called when a complete start tag has been
        recognized. The first argument is the tag name (in lower
        case) and the second argument is a reference to a hash that
        contain all attributes found within the start tag. The
        attribute keys are converted to lower case. Entities found
        in the attribute values are already expanded. The third
        argument is a reference to an array with the lower case
        attribute keys in the original order. The fourth argument is
        the original HTML text.

    $self->end($tag, $origtext)
        This method is called when an end tag has been recognized.
        The first argument is the lower case tag name, the second
        the original HTML text of the tag.

    $self->text($text)
        This method is called when plain text in the document is
        recognized. The text is passed on unmodified and might
        contain multiple lines. Note that for efficiency reasons
        entities in the text are not expanded. You should call
        HTML::Entities::decode($text) before you process the text
        any further.

        A sequence of text in the HTML document can be broken
        between several invokations of $self->text. The parser will
        make sure that it does not break a word or a sequence of
        spaces between two invokations of $self->text().

    $self->comment($comment)
        This method is called as comments are recognized. The
        leading and trailing "--" sequences have been stripped off
        the comment text.

    The default implementation of these methods do nothing, i.e.,
    the tokens are just ignored.

    There is really nothing in the basic parser that is HTML
    specific, so it is likely that the parser can parse other kinds
    of SGML documents. SGML has many obscure features (not
    implemented by this module) that prevent us from renaming this
    module as `SGML::Parser'.

EFFICIENCY
    The parser is fairly inefficient if the chunks passed to $p-
    >parse() are too big. The reason is probably that perl ends up
    with a lot of character copying when tokens are removed from the
    beginning of the strings. A chunck size of about 256-512 bytes
    was optimal in a test I made with some real world HTML
    documents. (The parser was about 3 times slower with a chunck
    size of 20K).

SEE ALSO
    the HTML::Entities manpage, the HTML::TokeParser manpage, the
    HTML::Filter manpage, the HTML::HeadParser manpage, the
    HTML::LinkExtor manpage

    the HTML::TreeBuilder manpage (part of the *HTML-Tree*
    distribution)
