JEL manual

# Java Expressions Library

## Structure of this manual

This is the first version of JEL manual. It is mostly examples based. There are two examples(with static and dynamic libraries) discussed here step by step. The main purpose of this document is to supplement`javadoc` generated documentation and provide several starting points to the exploration of it. There are following sections in the manual:

Features and design goals of JEL.
  Highlighting JEL main features.
How to use JEL.
  Walking through the example of use of static libraries.
Using libraries
  Walking through the example of use of dynamic (virtual) libraries.
Error detection and reporting
  Describing how JEL reacts on various errors in expressions.

This manual is by no means compherensive, but I feel it to be enough to start using JEL. If I'm wrong, please send me suggestions on what You would like to see in the next version of this manual.

## Features and design goals of JEL.

- JEL expression language is intuitively simple.
  In fact the language of JEL expressions resembles Java Language as close as possible. JEL supports ALL Java primitive types, plus object types even in form of constants ( which is available in Java only for `java.lang.String` class ).
- JEL allows to call functions.
  Functions in JEL are just methods of the Java objects, no front ends for functions have to be written and no additional (with respect to normal Java calls) run-time costs is associated with function calls. It is possible to use Java objects directly, exporting their functions through `gnu.jel.Library` class. JEL example calculator ( present in the distribution ) uses `java.lang.Math` methods directly.
- JEL is the compiler.
  Expressions entered are converted into the Java bytecodes on the fly. Then, expressions are loaded into the Java virtual machine and returned to the user of JEL as an instance of the class, whose superclass is `gnu.jel.CompiledExpression`
- JEL performs optimizations.
  Currently evaluation of the constant expressions is supported by optimizer. It means the expression "2*2" will be compiled into the Java analog `{return 4;}`. Moreover it is possible to mark functions as "stateless", which effectively allows their evaluation at compile time, namely, "sin(0)" to be compiled into Java equivalent of `{return 0.0D;}`.

## How to use JEL.

Simplicity of use was one of the design goals. Fortunately the interface to JEL can be made very simple, because it is well defined. Let's follow step by step a simple program, evaluating the expression given in it's command line arguments. Similar program exists in the distribution under the name `./samples/Calculator.java`

    public static void main(String[] args) {

        // Assemble the expression
        StringBuffer expr_sb=new StringBuffer();
        for(int i=0;i<args.length;i++) {
          expr_sb.append(args[i]);
          expr_sb.append(' ');
        };
        String expr=expr_sb.toString();

This first part of the program is not related to JEL. It's purpose is to assemble the expression, possibly, containing spaces into the single line. This has to be done, because shells tend to tokenize parameters... well that's their job, but we don't need it here.

        // Set up library
        Class[] staticLib=new Class[1];
        try {
          staticLib[0]=Class.forName("java.lang.Math");
        } catch(ClassNotFoundException e) {
          // Can't be ;)) ...... in java ... ;)
        };
        Library lib=new Library(staticLib,null);
        try {
        lib.markStateDependent("random",null);
        } catch (NoSuchMethodException e) {
          // Can't be also
        };

As JEL allows to call functions, from expression it certainly needs to know which ones You allow it to call. All information about functions, available for use in expressions, is contained within the Library object we just have constructed.

There are two classes of functions in the Library : static and virtual (dynamic).

Functions of the first class are assumed (by default) to be dependent only on their arguments i.e. they do not save any information from call to call(they are "stateless")... Examples of such functions are mathematical functions like `sin`, `cos`, `log`, for such functions it does not matter how many times (and when) they will be called they will always return the same result, provided their arguments are the same. Static functions can be evaluated by JEL at compile time if their arguments are constants (known at compile time). To define set of static functions it is needed to pass the array of Class objects, defining those functions, as the first parameter of the library constructor (see example above). Note ONLY STATIC functions of the Classes, passed in the first argument of the `gnu.jel.Library` constructor will be defined. Moreover, by default all static functions are marked as stateless.

Some static functions still save their state (in static variables) in between calls. Thus having different results, depending on when they are is called (even if arguments are the same). If such function is evaluated at compile time, we have troubles , because it will be evaluated only once during expression lifetime and it's state dependence will be lost. Typical example of the static function, having a state is `java.lang.Math.random()`. JEL has special mechanism, provided by `gnu.jel.Library` class to mark static functions as state dependent. (see the above example to find out how it was done for the `java.lang.Math.random()`)

There is the separate class of functions, understood by JEL, which are EXPLICITLY state dependent. We shall discuss these functions later in the same document, they are not used in the example we currently consider. However, virtual functions are, actually, MOST IMPORTANT to JEL. This is because expression, containing all stateless functions is a constant, it will be evaluated at compile time, and there is absolutely no sense to evaluate such expression repeatedly (this is what JEL was designed for). Still we shall continue with this simple example as the following code is mostly independent of whether we use virtual functions or not...

        // Compile
        CompiledExpression expr_c=null;
        try {
          expr_c=Evaluator.compile(expr,lib);
        } catch (CompilationException ce) {
          System.err.print("–––COMPILATION ERROR :");
          System.err.println(ce.getMessage());
          System.err.print("                       ");
          System.err.println(expr);
          int column=ce.getColumn(); // Column, where error was found
          for(int i=0;i<column+23-1;i++) System.err.print(' ');
          System.err.println('^');
        };

This chunk of code is for the expression compilation. The crucial line is the call to `Evaluator.compile(...)`, it is the point, where expression gets transformed into Java bytecode, loaded into the Java Virtual Machine using JEL Classloader and returned to caller as an instance of the subclass of `gnu.jel.CompiledExpression`. Typical user of JEL is not required to know what magic is going on inside of `Evaluator.compile(...)`. Other code in this chunk is for the error reporting and will be discussed in the specialized section, below.

        if (expr_c !=null) {

          // Evaluate (Can do it now any number of times FAST !!!)
          Number result=null;
          try {
    	result=(Number)expr_c.evaluate(null);
          } catch (Throwable e) {
    	System.err.println("Exception emerged from JEL compiled"+
    			   " code (IT'S OK) :");
    	System.err.print(e);
          };

Now we came to the evaluation of the function. It is done by calling the `evaluate` method of the JEL compiled class. This method is defined abstract in `gnu.jel.CompiledExpression` but it is redefined in the class, compiled by JEL. The argument of this method is discussed in the section on virtual functions below. If only static functions are present in the library it is safe to pass the `null` pointer as the argument to `evaluate`.

Result of the evaluate is always an object. JEL converts primitive numeric types into instances of the `java.lang.Number` subclasses. Actually, if the result of evaluation has the integral type it is returned as instance of the `java.lang.Long`, if it is a floating point number it is returned as `java.lang.Double`, if result is an arbitrary Java object it is returned as the reference to that object. The possible type of the result depends on the library You use. For the library, consisting of `java.lang.Math` methods only, result will always be either `java.lang.Long` or `java.lang.Double`. In other words it will always be a `java.lang.Number`.

The `try ... catch` clause around the call to `evaluate()` will be enforced by the Java compiler. It is required as errors can appear during evaluation. The general rule is : syntax, types incompatibility and function resolution errors will be reported at compile time ( as thrown instance of gnu.jel.CompilationException), while the errors in the values of numbers will be reported at the execution time. For example expression "1/0" will generate no error at compile time (nevertheless it is the constant expression and its evaluation is attempted), but at the time of calling `execute` You will get a java.lang.ArithmeticError (division by zero) as it should be.

          // Print result
          if (result==null)
    	System.out.println("void");
          else
    	System.out.println(result.toString());
        };
    };

This last piece of code will print the result. And is concluding our brief tour of the JEL usage.

## Using libraries

It was already said in this document, that the dynamic libraries are the most important for JEL because functions, from those libraries will be called each time the expression is evaluated (as opposed to once, when expression is compiled, for static libraries ).

To be able to call the method from the JEL expressions it is needed to define (export) it to JEL. This is done by constructing the library object, which hashes exported functions.

From the point of view of JEL there are, in fact three classes of functions. These classes of functions are defined below together with the hints on how to export them :

Static stateless functions
  These are static methods of Java classes, which do not save state in between calls. Evaluation of these functions will be attempted at compile time. Most of Java static functions are stateless. In JEL all static methods of classes in the array, supplied as a first argument of `gnu.jel.Library` constructor are exported and assumed stateless.
Static functions, saving the state
  These are static methods of Java classes which DO save state (in class static variables). Evaluation of these functions will not be attempted at compile time, they will be called each time the expression is evaluated. In JEL this kind of functions must be explicitly marked stateDependent after the `gnu.jel.Library` is created.
Virtual functions
  These functions save their state explicitly in the instance variables of some object (requiring the reference to that object to be supplied at execution time). Virtual functions are never evaluated at compile time.In JEL all virtual methods of classes in the array, supplied as a second argument of `gnu.jel.Library` constructor are exported as virtual functions.

Because class designation is omitted in JEL ( user enters "sin(x)" instead of "java.lang.Math.sin(x)") all methods, in the constructed Library are subjected to the same constraints as the methods of the single Java class (see § 8.4 in the Java Language Specification (JLS) ). For example, overloading of methods is allowed, but having two methods with the same names and parameter types is not (§ 8.4.2 in JLS). Java will not let You create such methods in single class, but when You construct a JEL library methods of several classes are merged making such conflicts possible. Be careful, when constructing libraries.

Let's finally walk through the example, which calculates function of the single variable many times, usage of dynamic libraries is necessary here. This example will consist of two classes : a user written class( which provides access to the variable) and the main class. First start with the variable provider :

    public class VariableProvider {
      public double x;

      public double x() {return x;};
    };

This class is trivial, it just defines the function, returning the value of the variable `x`.

In the main class (see the first JEL example for headers) the code, constructing the library will be replaced with :

        // Set up library
        Class[] staticLib=new Class[1];
        try {
          staticLib[0]=Class.forName("java.lang.Math");
        } catch(ClassNotFoundException e) {
          // Can't be ;)) ...... in java ... ;)
        };

        Class[] dynamicLib=new Class[1];
        VariableProvider variables=new VariableProvider();
        Object[] variableObjects=new Object[1];
        variableObjects[0]=variables;
        dynamicLib[0]=variables.getClass();

        Library lib=new Library(staticLib,dynamicLib);
        try {
        lib.markStateDependent("random",null);
        } catch (NoSuchMethodException e) {
          // Can't be also
        };

Note the new piece of code, which creates the variable provider class with the reference, named `variables`, array of objects `variableObjects` (to be passed to the `evaluate` method of the compiled expression) and array of classes `dynamicLib` for the library construction.

Now the code for compilation will be exactly the same as in the example 1, but we have additional function `x` defined for use in expressions. JEL has the special notation for the functions, having no arguments, namely, brackets in "x()" can be omitted to be "x". This allows to compile now ( with the above defined library) the expressions like "sin(x)", "exp(x*x)", "pow(sin(x),2)+pow(cos(x),2)"...

The code for evaluation will be replaced with :

        if (expr_c !=null) {

          try {
             for(int i=0;i<100;i++) {
                variables.x=i;      // <- Value of the variable
                System.out.println(expr_c.evaluate(variableObjects));
                                 //^^^^^^^^^^^^^^^ evaluating 100 times
    	 };
          } catch (Throwable e) {
    	System.err.println("Exception emerged from JEL compiled"+
    			   " code (IT'S OK) :");
    	System.err.print(e);
          };
       };

Note the two major differences: 1. we have explicitly assigned the value to the variable; 2. the array of object references ( consisting of one element in this example) is passed to the `evaluate` method. This piece of code will evaluate expressions for `x=0..99` with step `1`.

Object, in the `variableObjects[i]` should be possible to cast to the class in the `dynamicLib[i]` array for any `i`, otherwise ClassCastException will be thrown from `evaluate`.

This concludes our dynamic library example. Try to modify the ./Calculator.java sample yourself to allow compilation of virtual functions as described above.

## Error detection and reporting

Expressions are made by human, and this human is not assumed to be the programmer. Of course he should know what functions are, but no Java experience is needed. Making errors is the natural property of humans, consequently JEL has to be aware of that.

There are two places, where errors can appear. First are the compilation errors, which are thrown in the form of `gnu.jel.CompilationException` by the `gnu.jel.Evaluator.compile(...)`. These errors signal about syntax problems in the entered expressions, wrong function names, illegal types combinations, but NOT about illegal values of arguments of functions. The second source of errors is the compiled code itself, Throwables, thrown out of `gnu.jel.CompiledExpression.evaluate()` are primarily due to the invalid values of function arguments.

Compilation errors are easy to process. Normally, You should surround compilation by the `try ... catch (CompilationException )...` block. Caught `CompilationException` can be interrogated, then on the subject of WHERE error has occurred (`gnu.jel.CompilationException.getCol()`) and WHAT was the error `gnu.jel.CompilationException.getMessage()`. This information should then be presented to user. It is wise to use information about error column to position the cursor automatically to the erroneous place in the expression.

Errors of the second type are appearing during the function evaluation and can not be so nicely dealt with by JEL. They depend on the actual library, supplied to the compiler. For example methods of `java.lang.Math` do not generate any checked exceptions at all (still, Errors are possible), but You may connect library, of functions throwing exceptions. As a general rule : exceptions thrown by functions from the library are thrown from `evaluate` method.

## Summarizing remarks

I hope You found JEL useful. Don't hesitate to contact me if there are any problems with JEL, please, report BUGS, suggest tests, send me Your patches,... There are still many improvements to be done.

Most current information about JEL should be available at http://galaxy.fzu.cz/JEL/.

JEL is the "free software" and is distributed to You under terms of GNU General Public License. Find the precise terms of the license in the file ./COPYING in the root of this distribution.

JEL © 1998 by Konstantin Metlov (metlov@fzu.cz).