Description: An automatic generator of basic unit tests for a shared C/C++ library
Keywords: C, C++, automatic, tests, checker

# API Sanity Checker

An automatic generator of basic unit tests for a shared C/C++ library

API Sanity Checker is an automatic generator of basic unit tests for a shared C/C++ library. It is able to generate reasonable (in most, but unfortunately not all, cases) input data for parameters and compose simple ("sanity" or "shallow"-quality) test cases for every function in the API through the analysis of declarations in header files.

The quality of generated tests allows to check absence of critical errors in simple use cases. The tool is able to build and execute generated tests and detect crashes (segfaults), all kinds of emitted signals, non-zero program return code and program hanging.

It may be considered as a tool for out-of-the-box low-cost sanity checking (fuzzing) of the library API or as a test development framework for initial generation of templates for advanced tests. Also it supports universal T2C format of tests, random test generation mode, specialized data types and other useful features.

The tool is developed by Andrey Ponomarenko: http://abi-laboratory.pro/

Table of Contents Downloads License Supported Platforms Dependencies Installation Usage Examples Detectable Problems Specialized Types Tutorial How the Tool Works Bugs Maintainers Changes Articles

## Downloads

All releases can be downloaded from this page.

Latest release: api-sanity-checker-1.98.7.tar.gz

Read-only access to the latest development version:

`git clone git://github.com/lvc/api-sanity-checker`

## License

This program is free software. You may use, redistribute and/or modify it under the terms of the GNU GPL or GNU LGPL

## Supported Platforms

GNU/Linux, FreeBSD, Mac OS X.

## Dependencies

- ABI Compliance Checker (1.98.7 or newer)
- G++ (3.0-4.7, recommended 4.5 or newer)
- GNU Binutils (readelf, c++filt, objdump)
- Perl 5 (5.8 or newer)
- Ctags (5.8 or newer)

On Mac OS X it also requires Xcode for g++, c++filt, nm and otool.

## Installation

The tool is ready-to-use after extracting the archive.

You can also use a Makefile to install the tool into the system:

`sudo make install prefix=PREFIX [/usr, /usr/local]`

This command will install the `api-sanity-checker` program into the `PREFIX/bin` system directory and private modules into the `PREFIX/share`.

## Usage

For generating, building and running tests you should provide the XML descriptor for your library version. It is a simple XML-file that specifies version number, paths to header files and shared libraries and optionally some other information. An example of the descriptor is the following (0.3.4.xml):

    <version>
        0.3.4
    </version>

    <headers>
        /usr/local/libssh/0.3.4/include/
    </headers>

    <libs>
        /usr/local/libssh/0.3.4/lib/
    </libs>

Command to generate a test suite:

`api-sanity-checker -lib NAME -d VER.xml -gen`

You can view generated tests using the index file:

`tests/NAME/VER/view_tests.html`

or manually in the directory:

`tests/NAME/VER/groups/`

Command to build tests:

`api-sanity-checker -l NAME -d VER.xml -build`

Command to execute tests:

`api-sanity-checker -l NAME -d VER.xml -run`

The test report will be generated to:

`test_results/NAME/VER/test_results.html`

## Examples

Library | Version | Number of Tests | Problems Found
--- | --- | --- | ---
FreeType2 | 2.3.11 | 178 | 13
Glibc | 2.13 | 1996 | 340
libX11 | 1.3.4 | 778 | 286

## Detectable Problems

- Crash (segfault, signal SEGV)
- Abort (signal ABRT)
- All emitted signals: FPE, BUS, ILL and others
- Non-zero exit code
- Program hanging
- Requirement failure (if specified)

## Specialized Types

The specialized types are used to improve quality of generated tests. You can find more info here.

## Tutorial

See the article at glibc wiki.

## How the Tool Works

The basic idea of the test data generation algorithm is to recursively initialize parameters of a function using the values returned (or returned through the out-parameter) by other functions for structured data types (class, struct, union) or by some simple values for intrinsic data types (int, float, enum, ...). The recursion step includes the heuristic selection of the appropriate function, that should be called to initialize complex parameters for other functions. If some parameter of a function cannot be initialized, then the algorithm tries to select other function.

Let's see the example test for "FT_Activate_Size( FT_Size size )" function from the FreeType2 library:

    #include <freetype/freetype.h>
    int main(int argc, char *argv[])
    {
        FT_Library alibrary = 0;
        FT_Init_FreeType(&alibrary); //initialize "alibrary"

        FT_Face face = 0;
        FT_New_Face(
            alibrary,
            "sample.ttf",
            0,
            &face); //initialize "face"

        FT_Size size = 0;
        FT_New_Size(face, &size); //initialize "size"

        FT_Activate_Size(size); //target call
        return 0;
    }

In this test case the parameter "size" of target function FT_Activate_Size is initialized through the call of FT_New_Size function using its 2nd out-parameter. The first parameter "face" of FT_New_Size function is recursively initialized by the use of FT_New_Face function's 4th out-parameter. And finally the first parameter "alibrary" of FT_New_Face is initialized by the call of FT_Init_FreeType function on the 3rd recursion step. Other parameters of FT_New_Face function are initialized by intrinsic values.

## Bugs

Please post your bug reports, feature requests and questions to the issue tracker.

## Maintainers

The tool is developed by Andrey Ponomarenko.

## Changes

You can find changelog here.

## Articles

- "Sanity testing in software development", wikipedia.org
- "Fuzz testing", wikipedia.org
- "Automation of broad sanity test generation", R. Zybin , V. Kuliamin , A. Ponomarenko , V. Rubanov and E. Chernov