=head1 Introduction to the prototype Perl6 Compiler

=head2 Synopsis

A simple introduction and overview to P6C.  This document is only to be thought
of as a reference to the general flow of things; to get complete details, refer
to the source code.  If you ignore this bit now, don't worry, it will be said
many more times :)

=head2 Objects

There are 3 main types of objects that are used in P6C; it's best to 
become familiar with them before anything else:

=over 3

=item B<Raw Parse Object>

Raw parse objects are generated by Parse::RecDescent when it uses the grammar
found in F<P6C/Parser.pm>.  They contain the raw data from the parse.  After 
the parse, these objects process themselves using their C<tree> methods, 
defined in F<P6C/Tree.pm>.

=item B<Node Object>

Nodes are processed Parser objects, and are defined in F<P6C/Nodes.pm>.  A 
tree of these objects is called an op tree.  This tree is used in generating 
IMC code with F<P6C/IMCC.pm>.  Names mostly correspond to the rulenames from 
the grammar, but not entirely.  Some are omitted, and two -- ValueList and 
Register -- are added.

=item B<Context Object>

Context objects are defined in F<P6C/Context.pm>, and are actually a data 
member of Node objects (although they are added during the context 
propagation phase, rather than the parse tree deciphering phase.) Context 
objects contain the current context information of the associated node.

=back

=head2 Flow Control

As a brief overview, the whole process works like this:  First, perl6 code is
parsed using F<P6C/Parser.pm>, which generates a raw parse tree. The raw tree 
is then processed by methods found in F<P6C/Tree.pm>, turning the raw parse 
objects into an optree.  Context is propagated onto the optree with 
F<P6C/Context.pm>, and then evaluated and compiled to intermediate code by 
C<imcc>.  C<imcc> then translates...this into parrot assembly code, which is 
then assembled into bytecode by the parrot assembler and run.  It's just that 
simple.

=head2 In more detail

The first thing perl6 will do is start the first pass over your code: parse it.
Assuming a normal run (no flags), perl6 will use the precompiled parser,
F<Perl6Grammar.pm>. If you made any changes to the grammar in F<P6C/Parser.pm>,
you'll need to run your code with the --force-grammar flag to recompile it.

=head2 The Parser

Before the grammar itself is described, an overview of the parser will be
helpful.  The parser works by recursively defining a statement down to its
most basic levels, and then turning the matched statement into a tree-like
structure of objects.  For instance, a small program like this:

 $x = 4;
 $y = $x + 6;
 print $y;

Could be visualized as:

 Program
    scalar expression
        operator '='
            left  => variable
                        sigil   => '$'
                        varname => 'x'
            right => scalar literal
                        type  => number
                        value => 4
    scalar expression
        operator '='
            left  => variable
                        sigil   => '$'
                        varname => 'y'
            right => scalar expression
                        operator '+'
                            left  => variable
                                        sigil   => '$'
                                        varname => 'x'
                            right => scalar literal
                                        type  => number
                                        value => 6
    scalar expression
        prefix 'print'
            args => variable
                        sigil   => '$'
                        varname => 'y'

Each node on this tree is actually an object in the form of a blessed
anonymous array, created as a result of the grammar's autoaction.  It holds
the raw data from the parse; one element per production.

Next, on to the grammar.  As stated earlier, the grammar is written in
Parse::RecDescent.  For almost all cases, rules should be written without an
action as the final production, so that the autoaction that turns the match
into an object can be applied.	To get a real idea of what's going on, you
should read through the grammar; it's long, but most rules are short.  The base
rule for the grammar is C<prog>, but you can probably start at C<stmt> unless
you care about error handling and such.  

A few notes about the grammar in general:

=over 3

=item B<"Want" Rules>

Keywords and prefix operators use special "want_for" rules to match their 
correct definition.  This is for modularity purposes.  The prefix rule will 
match a keyword, and then do a hash lookup to find its corresponding 
"want_for" rule. For instance, if the "grep" keyword is found, then the 
lookup will then attempt to match the "want_for_grep" rule.  The "want_for" 
rules match the "body" of the corresponding rule.  Many builtin functions 
and unary operators will need their own individual "want_rule" for the parser 
to be accurate.

=item B<Regex as Speed Optimizations>

Simple rules have been redefined as compiled regular expressions for speed
purposes.  These include all of the binary operators, as well as the numerical
constant, error flusher, and special string delimiter.

=item B<Special Parser Functions>

A few notes about some of the functions in the parser: C<add_class> adds
user-defined classes to the parser, and C<add_function> does the same for
functions.  C<add_context> is used by C<add_function> to create the resulting
"want_for" rule for the function (to handle prototypes if it has them), as well
as generating the tree deciphering method (see "Deciphering the Object Tree"
for more details).  Finally, C<got_err> handles error messages.

=back

=head2 Deciphering the Object Tree

Once the parse is complete, the object tree is deciphered by a depth-first
recursive traversal of the parse tree using the C<tree> methods found in
Tree.   While there is a matching "tree" method for nearly every single rule
defined in Parser, there is no way to "generate" the tree methods as they
need to handle the data very specifically.  However, the general gist of what
they do is like this:

=over 3

=item B<Remove junk data from the raw object>

Removing junk amounts to removing the syntax that won't really matter to the
compiler.  For instance: removing the "," from argument lists, removing special
quote delimiters, or ignoring unnecessary whitespace.  None of this syntax
really matters internally, so it is thrown away.

=item B<Turn the data into a processed node for "base" types>

If the object is a "base" type (i.e has a corresponding node type in
F<P6C/Nodes.pm>), the proper information is extracted from the parse object and
used to create and return a new node of that type.

=item B<Otherwise, call tree on the remaining sub-objects>

Calls the C<tree> methods of its members, gathers up the return values, and
builds a branch of the optree.

=back

=head2 Propagating Context

Propagating Context is another way of saying "Figuring out what kind of value a
node is expected to yield during compilation."	Context information is added to
a node by adding another member, C<ctx>, to the node, and setting its value
equal to a context object.  Context objects are defined in F<P6C/Context.pm>,
which also contains the functions for figuring out context. 
F<P6C/Addcontext.pm> defines the C<ctx_left> and C<ctx_right> methods for 
each Node type (which deal with lvalue and rvalue context, respectively), 
that define how each Node type propagates its context.

=head2 Compiling to IMC

Once the parse tree has been post-processed, the next stage is to travel
through the op tree and compile it to IMC code.  Each node type has a C<val>
method that will be called by its parent node.	This method gathers values from
child nodes (by calling their C<val> methods), emits the code for the
node's operation (using C<P6C/IMCC.pm::code>), and then returns the proper type
for its context.  Code is appended to the current function in the order in
which it is generated, so subnodes have to be evaluated in the proper order.
Things acting as lvalues need to define an C<assign> function.  Currently only
variables, variable declarations, and the ternary operator do so, but other
things will need to as well at some point.  Finally, regex nodes actually use
an C<rx_val> function instead, which behaves differently, but serves basically
the same purpose.

The compiler is very large, but there are a few things to note:

=over 3

=item B<scope>

The compiler maintains a "current function" in which code is emitted, locals
are declared, and symbol lookups begin.	C<P6C/IMCC.pm::code($x)> will add the
IMC code $x to the current function.  Scope can further be manipulated "inside"
a function by using C<push_scope>, which increases the depth of the scope by
one, and C<pop_scope>, which lowers it by one.


=item B<symbol table>

The compiler also maintains a primitive symbol table.  Locally scoped variables
(right now, meaning "function" variables) can be added with C<add_localvar>.
Global variables can be added with C<add_globalvar>.  Lookups can be done with
C<findvar>, which first searches the active function's locals, then parameters,
and then finally globals.  This implementation may change when lexical scoping
is fully implemented.

=item B<code generation>

There are many functions that aid in the generation of IMC code; see the
F<P6C/IMCC.pm> documentation for more details.

=back

=head2 Finishing the process

From this point on, most of the work is done by external parrot processes.
First, F<P6C/IMCC.pm> is used to convert the op tree to .imc code. IMCC 
converts this code to .pasm.  The assembler assembles the the .pasm to 
parrot bytecode, and then finally the bytecode is run by the driver.

=head2 Author of this document

Joseph F. Ryan (ryan.311@osu.edu)

$Id: overview.pod 7736 2005-03-30 23:15:18Z chromatic $

=cut
