=head1 NAME

tconv - iconv-like interface with automatic charset detection

=head1 SYNOPSIS

  #include <tconv.h>

  size_t tconv(tconv_t cd,
               char **inbuf, size_t *inbytesleft,
               char **outbuf, size_t *outbytesleft);

=head1 DESCRIPTION

tconv is like iconv, but without the need to know the input charset. Caller might want to play with macros e.g.

  #define iconv_t                       tconv_t
  #define iconv_open(tocode, fromcode)  tconv_open(tocode, fromcode)
  #define iconv(cd, ipp, ilp, opp, olp) tconv(cd, ipp, ilp, opp, olp)
  #define iconv_close(cd)               tconv_close(cd)

When calling L<tconv_open>:

  tconv_open(const char *tocode, const char *fromcode)

it is I<legal> to have NULL for C<fromcode>. In this case the I<first chunk> of input will be used for charset detection, it is therefore recommended to use enough bytes at the very beginning. If C<fromcode> is not NULL, no charset detection will occur, and tconv will behave like L<iconv(3)>, modulo the engine being used (see below).
If C<tocode> is NULL, it will default to C<fromcode>.

Testing if the return value is equal to C<(size_t) -1> or not, together with C<errno> value when it is C<(size_t) -1> as documented for C<iconv>, is the only reliable check: in theory it should return the number of non-reversible characters, and this is what will happen is this is C<iconv> running behind. In case of another convertion engine, the return value depend on this engine capabilities, or how the corresponding plugin is implemented.

When the number of bytes left in the input is C<0>, the return value is equal to C<(size_t) -1>, and C<errno> is C<E2BIG>: you should not count on C<*ilp> position: the conversion engine may have an internal staging array that have consumed all the input bytes, but is waiting for more space to produce the output bytes. This is happening for instance:

=over

=item with the ICU convert engine

Regardless if you use C<//TRANSLIT> option or not, the ICU convert engine is always doing two conversions internally, one from input encoding to UTF-16, then from UTF-16 to output encoding. This means that it is always eating entirely the input bytes into an internal staging area.

=back

=head1 ENGINES

tconv support two engine types: one for charset detection, one for character conversion, please refer to the L<tconv_open_ext> documentation for technical details. Engines, whatever their type, are supposed to have three entry points: C<new>, C<run> and C<free>. They can be:

=over

=item external

The application already have the C<new>, C<run> and C<free> entry points.

=item plugin

The application give the path of a shared library, and tconv will look at it.

=item built-in

Python's L<cchardet> charset detection engine, bundled with tconv, is always available.
If tconv is compiled with ICU support, then ICU charset and conversion engines will be available.
The ICONV conversion engine is always available.

=back

=head1 DEFAULTS

=over

=item charset detection

The default charset detection engine is cchardet, bundled statically with tconv.

=item character conversion

The default character conversion engine is ICU, if tconv has been compiled with ICU support, else iconv.

=back

=head1 NOTES

=over

=item iconv

The iconv plugin is always available and is based on libiconv, bundled within tconv.

=item iconv compliance

=over

=item semantics

tconv() only guarantees that his plug-ins support the C<//TRANSLIT> and C<//IGNORE> iconv notation.

=item pointers in case of error

In some cases, an internal buffer is used. This mean that in case of a failure with errno being C<EINVAL> or C<EILSEQ>, input and output pointers are left in a state that, if being called again, will correctly handle the continuation of the conversion, but may not be exactly at the point of failure as per the original iconv specification.

=back

=back

=head1 SEE ALSO

L<tconv_ext(3)>, L<libiconv|https://www.gnu.org/software/libiconv>, L<cchardet|https://pypi.python.org/pypi/cchardet/>
