# EZTrace Developper Manual

## Introduction

The aim of this document is to describe EZTrace internals.

In addition to the content of this document, a few research papers
describe some aspects of EZTrace internals:

- (*EZTrace: a generic framework for performance
  analysis*)[https://hal.inria.fr/inria-00587216/] briefly describes the architecture
  of EZTrace.
- (*An open-source tool-chain for performance
  analysis*)[http://hal.archives-ouvertes.fr/hal-00707236] describes the EZTrace + GTG
  + ViTE tool chain. This paper includes explanations on the way
  traces are generated with EZTrace-1.0. **Please note that the trace generation changed with EZTrace 2.0**
- (*Runtime function instrumentation with
  EZTrace*)[http://hal.inria.fr/hal-00719979] describe how EZTrace instruments
  applications using binary instrumentation

## Instrumentation

In order to intercept the application calls to the EZTrace modules
function, EZTrace needs to *instrument* the application. The
goal of the instrumentation of function `foo` is to be able to
record an event before and after each occurence of `foo`.

Depending on which type of function is `foo`, EZTrace uses two
mechanisms:

- if `foo` is in a shared library, EZTrace uses a
  mechanism based on LD_PRELOAD.
- otherwise, EZTrace uses PPTrace

### LD_PRELOAD

When instrumenting a program in order to intercept the calls to
function `f()` located in a shared library, EZTrace uses
LD_PRELOAD.

As depicted in Figure 1, the basic idea is to ask
the program loader to pre-load a library that defines `f()`
(eg `libeztrace-foo.so`) when loading the application. This
way, the application calls to function `f()` are directed to
EZTrace implementation of `f()` (let's call this
implementation `eztrace_f()`).

Once `eztrace_f()` has recorded an event, it needs to call the
actual `f()` function. To do this, EZTrace retrieves the
address of `f()` using the `dlsym()` function.  This is
done at when `libeztrace-foo.so` is loaded (
`DYNAMIC_INTERCEPT_ALL()` in the `libinit` function).

!(Figure 1: Instrumentation using LD_PRELOAD)[Figures/instrumentation_ldpreload.png]

### PPTRace

When the function to instrument is not in a shared library, the
LD_PRELOAD mechanism cannot work. In that case, EZTrace modifies the
application binary using PPTrace.

The main idea is that PPTrace load the application binary and uses
`ptrace` in order to modify the application code section. When
instrumenting a function, PPTrace injects a jump instruction at the
beginning of the function to instrument. This Section the detailed
procedure to do so.

Two techniques can be used depending on the where EZTrace managed to
allocate memory:

- If the memory allocated ``close'' to the code segment (for
  instance on 32-bits architectures like ARMv7), PPTrace uses a long
  jump
- If the memory is allocated ``far'' from the code segment, (for
  instance, on 64-bits architectures like x86_64),
  PPTrace uses a trampoline technique


Both strategies are implemented in the `src/pptrace/hijack.c`
file.

#### Using long jump

In some cases, PPTrace uses a long jump for changing the execution
flow of the application. The resulting execution flow is depicted in
Figure 2.


!(Figure 2: Instrumentation using a long jump)[Figures/instrumentation_pptrace_arm.png]

PPTrace first allocates a buffer (pptrace_allocated in the Figure)
where it stores the first opcodes of function `foo()`. These
opcodes are now located at address `reloc_addr`. They are followed
by a jump (`back_jump`) to the remaining opcodes of `foo`.

PPTrace then changes the first opcode of `foo()` to the
instruction that jumps (`first_jump`) to `repl_addr`
(which contains the address of EZTrace implementation of
`foo`).

The address of the relocated code (`reloc_addr`) is then
assigned to the module callback for function foo (stored at address
`orig_addr`).

The drawback of this technique is that the `first_jump` may
overwrites many opcodes. Some of these opcodes may be problematic (for
instance, `ret` or `mov (%rip+1042), %eax`). When it
is possible, another technique that uses a *trampoline* can be
used.

### Using a trampoline
In order to reduce the number of opcodes that need to be relocated,
PPTrace can use the *trampoline* technique depicted in
Figure 3. The idea is to first do a ``small
jump'' to a trampoline that jumps (`long_jump`) to EZTrace
implementation of `foo`.

The small jump can be implemented as a relative jump (ie ``jump to
%rip + 1000'').

!(Figure 3: Instrumentation using a trampoline)[Figures/instrumentation_pptrace_x86_64.png]
