                                 Muldis::D
---------------------------------------------------------------------------

Following is a summary of things that still need doing.  It is specific to
the Muldis D specification distribution only, and doesn't talk about things
that would go in other distributions, including implementations.  (But,
look at lib/Muldis/D/SeeAlso.pod for a list of actual or possible
implementations.)

Alternately, this list deals with possible ideas to explore, which may or
may not be good ideas to pursue.

The following list is loosely ordered by priority, but list items may
actually be addressed in a different order.  There is no specific time
table for these items; they are simply to be done "as soon as possible".

* Generally speaking, make a new release to CPAN once every week, assuming
the progress is non-trivial, so there are regular public snapshots with
nicely rendered documentation.

* Consider changing the keywords in system-defined routine signatures,
specifically [result, params, update, read], to something better sounding.

* Add ext/TAP.pod, which is a partial port of Perl 5's Test::More / Perl
6's Test.pm / David Wheeler's pgTAP to Muldis D; assist users in testing
their Muldis D code using TAP protocol.  The TAP messages have type Text.

* IN PROGRESS ...
Reorganize the Tiny dialects docs so that plain-text-Tiny now has most of
the detail info and is standalone, and hosted-data-Perl-Tiny is just
defined as being the same as the former but for specified format
differences.

* Update the plain-text Tiny dialect to support more Perl 6 isms directly,
such as "0(b|o|d|x)" prefixes rather than "max-col-val;" prefixes (and the
same can be done for Blob but just b|o|x opts).

* Update the Tiny dialects with special syntax for declaring whole depots
in a single expression, catalog+data.  This can be useful for example with
a simple Muldis D implementation that wants to represent a persisted depot
on disk as a single file that is Muldis D source code.  This would mostly
be like a bootloader definition except that details related to the name
that a depot is mounted with aren't given as they would with an
application-side bootloader.

* Update the Tiny dialects with special syntax for some nonscalar catalog
types, particularly definitions of routines and types, so that both doing
this can be made even more like a normal language, and also the missing
pieces to support more appropriate syntax coloring and routine locating in
text editors looking at Muldis D source, eg a 'function' keyword at the
start of each function.  Here are some keywords that can then be added:
function, updater, procedure, type, param, ro_param, upd_param, expr, stmt,
subtype, key, possrep, etc; they apply to both 'outer' and 'inner'
types/routines; also keywords for declaring whole packages or something.
Also consider that some pragmas can go away, their actions made always-on
or 'lexical'.

* Consider adding type introspection routines like: is_incomplete() or
is_quasi() or is_primitive|structure|reference|enumerated etc.  Or don't
since one could look that up in the system catalog.  But more tests on
individual values might be useful, or maybe we have enough already.

* Consider adding a direct analogy to Perl 6 "roles", whose primary purpose
is to help with managing software reuse, mainly when multiple types have a
number of attributes in common, a role can define these and then the
multiple types can compose that role.  This concept should be workable with
both InnerScalarType and InnerNonscalarType.  A role can compose other
roles.  A role or type that composes a role can both add additional
attributes of its own to what the role defines, and the composer can add
extra constraints over the composed attributes like forcing a subtype.  Now
on one hand we could make a role a distinct concept, which by itself can't
be instantiated as a type, or we could reuse the above 2 (IST/INT) type
definition facilities and say an existing such type can be composed into
another.  Now that latter case would in a manner of speaking give Muldis D
the advantages of other OO languages or SQL that work using "specialization
by extension" except that composing an existing type in this way will
actually, for a q/scalar base, create a disjoint type not a subtype, so we
get the code reuse but Muldis D is still specialization by constraint.  On
the other hand, if we just declare roles as roles that can't be
instantiated themselves, then declaring a role could also serve to declare
a quasi- union type name that can be referred to in type constraint
definitions, that union type implicitly ranging over all actual types that
compose it, and so in this case the composers actually are subtypes; the
problem with this is, what to do about a default value for said union type
whose role isn't composed into anything ... maybe each one needs an
implicit DVPT subtype?  But maybe by TTM terms, what a role actually is is
a dummy type of sorts, where you declare the union name before what it
unions, and the regular types it unions declare themselves part of the
union rather than the other way around.  In any event, a composed role
would never appear in a value's MST or root type list, same as any type
declared as a union.  I should look at TTM and see if they intended for
dummy types to fulfil a role such as this.  Except all directly declared
subtypes of a role union are always disjoint, for q/scalar ones anyway; the
role is defin shorthand.

* IN PROGRESS ...
A Muldis D host|peer language can probably hold lexical-to-them variables
whose Muldis D value object is Reference typed, and so they could
effectively pass around an anonymous Muldis D FuncRef etc like a closure of
their own language.  Such a value object would be a black box to the host
and can't be dumped to Muldis D source code.

* IN PROGRESS ...
Fully support direct interaction with other languages, mainly either peer
Parrot-hosted languages or each host language of the Muldis D
implementation.  Expand the definition of the "reference" main type
category (or if we need to, create a 5th similarly themed main category) so
that it is home to all foreign-managed values, which to Muldis D are simply
black boxes that Muldis D can pass around routines, store in transient
variables, and use as attributes of quasi-tuples or quasi-relations.  These
of course can not be stored in a Muldis D depot/database, but they can be
kept in transiant values of Muldis D collection types which are held in
lexical variables by the peer or host language; that language is then
really just using Muldis D as a library of relational functions to organize
and transform its own data.  We also need to add a top level namespace by
which we can reference or invoke the opaque-to-us data types and routines
of the peer or host language.  This can not go under sys.imp or
sys.anything because these are supposed to represent user-defined types and
routines, which in a dynamic peer language can appear or disappear or
change at a moment's notice, same as in Muldis D; on the other hand, types
or routines built-in to the peer/host language that we can assume are as
static as sys.std, could go under sys.imp or something.  This also doesn't
go under fed etc since fed is reserved for data under Muldis D control and
only ever contains pure s/t/r types.  Presumably this namespace will be
subdivided by language analagously to sys.imp or whatever syntax Perl 6
provides for calling out into foreign languages co-hosted on Parrot.  Since
all foreign values are treated as black boxes by Muldis D, it is assumed
that the Muldis D implementation's bindings to the peer/host language will
be providing something akin to a simple pointer value, and that it would
provide the means to know what foreign values are mutually distinct or
implement is_identical for them.  One thing for certain is that every
foreign value is disjoint from every Muldis D value, and by default every
foreign value is mutually distinct from every other foreign too, unless
identity is overloaded by the foreign, like how Perl 6's .WHICH works.
The foreign-access namespace may have a simple catalog variable
representing what types and routines it is exposing, but to Muldis D this
would be we_may_update=false.  It stands to reason that Muldis D's standard
FuncRef etc can point to the name of a foreign routine as much as a native.
TODO:  Figure out how to declare a Muldis D quasi- type whose attributes
are declared of some foreign type ... or maybe we can't and all such
attributes would simply be declared Universal all the time, might be enuf.

* IN PROGRESS ...
First round to above:  Add system-defined subtype of Reference called
sys.std.Core.Type.Foreign (itself 'special' like FuncRef etc), the new
one being a root type, and then we simply say that all values passed in
from a host or peer language are of that root type, period; this new one is
declared in Types.pod probably maybe I guess.  Then update Perl5_Tiny and
Perl6_Tiny to add a new PHMD node kind 'Foreign' which takes any Perl
value or object as its payload; this is treated completely as a black box
in general within the Muldis D implementation.  For matters of identity
within the Muldis D envirnment, it works as follows:  Under Perl 6, the
Perl value's .WHICH result determines its identity.  Under Perl 5, if the
value is a Perl ref ('ref $obj' returns true) then its memory address is
used, and this applies to all objects also (since all refs are mutable,
this seems to be the safest bet); otherwise ('ref $obj' is false) then the
value's result in a string context, "$obj", is used as the identity; the
mem addr and stringification would both be prefixed with some constant to
distinguish the 2 that might stringify the same.  By default, a
ForeignValue supports no operators but is/not_identical.

* Add catalog types that represent thrown exceptions.

* Also study SQL concept of conditions and handlers, looks sort of like
something between exception handling, signals; or it is their exceptions.

* Update the virtual attributes maps so there is a way to manually specify
a reverse function, as meanwhile all the virtuals don't have to be either
read-only or updatable due to an automatically generated reverse function,
which might vary by implementation, which may be considered broken.  Note
that the reverse functions might have to be defined as per-tuple
operations, separately for insert/substitute/delete.

* Add transition constraints.  The initial version would simply take a
'before' and an 'after' state argument.  But it may be good to provide an
alternative where arguments show the delta since often that is what
constraint writers want to know anyway.

* Add other kinds of triggered routines.

* Enhance the catalog so that one when defining values like
(Tup|Rel)LitExprNodeSet, one can tag such values with the data type the
value is supposed to be in, so they can auto-complete missing attributes as
per the treat_as_type lexical-scope pragma of the Tiny dialects; this would
also work for when a definition isn't a valid relation since the actual
Node value is valid.  In the absense of or in addition to auto-complete,
this feature could be used as treat-like operators to help with code
validation or allow more at-compile-time validation.

* Update Text.pod (or add other extensions) to add general search and
replace or transliteration or regular expression et al functionality,
besides the bare minimum SQL-LIKE-alike currently provided.  Also add
functions for splitting strings on delimiters or catenating them with such.

* Add functions for comparing or processing continuous ranges or intervals
as per is_inside_range ... the tests are for eg if 2 ranges do or don't
overlap, or whether an overlap is specifically between the start of one and
the end of another, or whether ranges simply touch, or whether a range is
entirely within another or not, etc ... some are analagous to set
operations, but not so much as they talk about continous intervals, or as
continuous such as the type in question supports ... useful particularly
for use with date data types.  as for supporting non-continuous ranges,
which are probably represented by sets of continuous ranges, that might be
simple enough to support here too ... or some range stuff might go off to
another extension perhaps or perhaps not.

* Consider supplementing the current generic infinite-loop-until-leave
statement with alternatives that use boolean control variables, such as
'while' or 'repeat-while'.

* Consider adding an imperative for-each looping statement; the main
question here is whether it should work on any (unordered) relation or just
on an Array (in which case it iterates through the tuples in sequence by
index); the question is what tasks the for-each would be used for; perhaps
both versions are useful; presumably the main reason to have for-each at
all is when I/O is involved and some derivative needs to be output either
where order matters or where order does not matter; but perhaps only a
routine is needed here such as a catenate function plus normal I/O output.

* Consider converting the try-catch routine into a statement node, an
optional part of which is to make the insides of the try/catch blocks part
of the same lexical scope as the invoking routine.  But there is still a
lot to like about having each of the try/catch blocks being a separate
(inner) routine with its own lexical scope, such as in predicting what a
rollback would affect, and it might be easier to implement as-is.

* Add sequence generator updaters|procedures in Integer.pod.

* Consider adding random value generators for data types other than integer
and rational numerics, such as for character strings or binary strings.

* Make further updates to Temporal.pod to improve correctness or
normalization, particularly with durations, maybe add some duration
subtypes.  Also expand the selection of routines to allow more control over
semantics.  Seek advice from the Perl DateTime people about this stuff.

* Consider updates to Temporal.pod that let one record time zones etc
attached to instant values; this would be a non-central feature.

* Provide operators for the spatial data types, maybe overhaul the types.

* Update the system catalog concerning managing shared|exclusive locks or
looking for consistent reads between statements, etc.

* Update the system catalog to deal with database users and privileges etc.

* Flesh out matters related to starting or communicating between multiple
autonomous in-DBMS processes, in general, besides the special case about
sequence generators.

* Consider some more elegant solution about matters of closures et al.

* Whatever else needs doing, such as, fixing bugs.
