Motivations and Design (ne’s manual)

Next: The Encoding Mess, Previous: Hints and Tricks [Contents][Index]

## 7 Motivations and Design

In this chapter I will try to outline the rationale behind `ne`’s design choices. Moreover, some present, voluntary limitations of the current implementation will be described. The intended audience of such a description is the programmer wanting to hack up `ne`’s sources, or the informed user wanting to deepen his knowledge of the limitations.

The design goal of `ne` was to write an editor that is easy to use at first sight, powerful, and completely configurable. Making `ne` run on any terminal that `vi` could handle was also a basic issue, because there is no use getting accustomed to a new tool if you cannot use it when you really need it. Finally, using resources sparingly was considered essential.

`ne` has no concept of mode. All shortcuts are defined by a single key, possibly with a modifier (such as `Control` or `Meta`). Modality is in my opinion a Bad Thing unless it has a very clear visual feedback. As an example, menus are a form of modality. After entering the menus, the alphabetic keys and the navigation keys have a different meaning. But the modality is clearly reflected by a change in the user interface. The same can be said about the input line, because it is always preceded by a (possibly highlighted) prompt ending with a colon.

`ne` has no sophisticated visual updating system similar to, for instance, the one of `curses`. All updating is done while manipulating the text, and only if the turbo flag is set can some iterated operations delay the update. (In this case, `ne` keeps track in a very rough way of the part of the screen that changed.) Moreover, the output is not preempted by additional input coming in, so that along a slow connection the output could not keep up with the input. However, along reasonably fast connections, the responsiveness of the editor is greatly enhanced by the direct update. And since we update the screen in parallel with the internal representation, we can exploit our knowledge to output a very small number of characters per modification. As it is typical in `ne`, when such design tradeoffs arise, preference is given to the solution that is effective on a good part of the existing hardware and will be very effective on most future hardware.

`ne` uses a particular scheme for handling text. There is a doubly linked list of line descriptors that contain pointers to each line of text. The lines themselves are kept in a list of pools, which is expanded and reduced dynamically. The interesting thing is that for each pool `ne` keeps track just of the first and of the last character used. A character is free iff it contains a null, so there is no need for a list of free chunks. The point is that the free characters lying between that first and the last used characters (the lost characters) can only be allocated locally: whenever a line has to grow in length, `ne` first checks if there are enough free characters around it. Otherwise, it remaps the line elsewhere. Since editing is essentially a local activity, the number of such lost characters remains very low. And the manipulation of a line is extremely fast and independent of the size of the file, which can be very huge. A mathematical analysis of the space/time tradeoff is rather difficult, but empirical evidence suggests that the idea works.

`ne` takes the POSIX standard as the basis for UN*X compatibility. The fact that this standard has been designed by a worldwide recognized and impartial organization such as IEEE makes it in my opinion the most interesting effort in its league. No attempt is made to support ten thousand different versions and releases by using conditional compilation. Very few assumptions are made about the behaviour of the system calls. This has obvious advantages in terms of code testing, maintenance, and reliability. For the same reasons, the availability of an ANSI C (C99) compiler is assumed.

If the system has a `terminfo` database and the related functions (which are usually contained in the `curses` library), `ne` will use them. The need for a terminal capability database is clear, and the choice of `terminfo` (with respect to `termcap`) is compulsory if you want to support a series of features (such as more than ten function keys) that `termcap` lacks. If `terminfo` is not available, `ne` can use a `termcap` database, or, as a last resort, a built-in set of ANSI control sequences. Some details about this can be found in Portability Problems.

`ne` does not allow redefinition of the `Escape`, `Tab` or `Return` keys, nor of the interrupt character `Control-\`. This decision has been made mainly for two reasons. First of all, it is necessary to keep a user from transforming `ne`’s bindings to such a point that another unaware user cannot work with it. These two keys and the alphabetic keys allow activating any command without any further knowledge of the key bindings, so it seems to me this is a good choice. As a second point, the `Escape` key usage should generally be avoided. The reason is that most escape sequences that are produced by special keys start with the escape character. When `Escape` is pressed, `ne` has to wait for one second (this timing can be changed with the `EscapeTime` command), just to be sure that it did not receive the first character of an escape sequence. This makes the response of the key very slow, unless it is immediately followed by another key such as ‘`:`’, or by `Escape`, again. See Hints and Tricks.

Note that, as has been stated several times, the custom key bindings also work when doing a long input, navigating through the menus or browsing the requester. However, this is only partially true. To keep the code size and complexity down, in these cases `ne` recognizes only direct bindings to commands, and discards the arguments. Thus, for instance, if a key is bound to the command line `LineUp 2`, it will act like `LineUp`, while a binding to `Macro MoveItUp` would produce no result. Of course full binding capability is available while writing text. (This limitation will probably be lifted in a future version: presently it does not seem to limit seriously the configurability of `ne`.)

`ne` has some restrictions in its terminal handling. It does not support highlighting on terminals that use a magic cookie. Supporting such terminals correctly is a royal pain, and I did not have any means of testing the code anyway. Moreover, they are rather obsolete. Another lack of support is for the capability strings that specify a file to print or a program to launch in order to initialize the terminal.

The macro capabilities of `ne` are rather limited. For instance, you cannot give an argument to a macro: macros are simply sequences of commands that can be played back automatically. This makes them very useful for everyday use in a learn/play context, but rather inflexible for extending the capabilities of the editor.

`ne` has been written with sparing resource use as a basic goal. Every possible effort has been made to reduce the use of CPU time and memory, the number of system calls, and the number of characters output to the terminal. For instance, command parsing is done through hash techniques, and the escape sequence analysis uses the order structure of strings for minimizing the number of comparisons. The optimal cursor motion functions were directly copied from `emacs`. The update of files using syntax highlighting is as lazy as possible: modifications cause just the update of the current line, and the rest of the screen is updated only when you move away. The search algorithm is a simplified version of the Boyer-Moore algorithm that provides high performance with a minimal setup time. An effort has been taken to move to the text segment all data that do not change during the program execution. When the status bar is switched off, additional optimizations reduce the cursor movement to a minimum.

A word should be said about lists. Clearly, handling the text as a single block with an insertion gap (a la `emacs`) allows you to gain some memory. However, the management of the text as a linked list requires much less CPU time, and the tradeoff seems to be particularly favorable on virtual memory systems, where moving the insertion gap can require a lot of accesses to different pages.

Next: The Encoding Mess, Previous: Hints and Tricks [Contents][Index]