=head1 NAME

nbdkit-lzip-filter - nbdkit lzip filter

=head1 SYNOPSIS

 nbdkit --filter=lzip file FILENAME.lz

=for paragraph

 nbdkit --filter=lzip curl https://example.com/FILENAME.lz

=head1 DESCRIPTION

C<nbdkit-lzip-filter> is a filter for L<nbdkit(1)> which transparently
decompresses an L<lzip(1)>-compressed file.  You can place this filter
on top of L<nbdkit-file-plugin(1)> to decompress a local F<.lz> file,
or on top of other plugins such as L<nbdkit-curl-plugin(1)>:

 nbdkit curl --filter=lzip https://example.com/disk.lz

With L<nbdkit-partition-filter(1)> it can be used to mount a specific
partition of a compressed disk image:

 nbdkit curl --filter=partition --filter=lzip \
        https://example.com/disk.img.lz partition=2

The filter only supports read-only connections.

B<Note> that this filter was made primarily for multi-member files.
Single-member files are currently only supported up to a size of
C<lzip-max-block> and their contents are kept in memory.

=head2 Single-member files and multi-member files

An lzip-compressed file consists of one or more members, with each
member containing an LZMA stream and some metadata.  Most lzip files
consist of only a single member holding all of the compressed data.
You can find out the number of members in an lzip file by doing:

 $ lzip --list --verbose lfs.img.lz
   dict   memb  trail   uncompressed   compressed   saved  name
   8 MiB     1      0    10737418240   1808781538  83.15%  lfs.img.lz
           ↑↑↑
          members

Seeking an lzip file is done by seeking directly to the lower member
boundary, then decompressing data until the precise byte is reached.
Because of this, B<single-member files are not seekable and must be>
B<decompressed entirely>.

=head2 Getting the best random access performance

Use the L<plzip(1)> parallel compressor with the I<--data-size> option
set to a small-ish block size.  plzip first splits the input file into
I<fixed-size> data blocks before compressing each block as a separate
member.  For example this is the same image as above compressed with
plzip on default settings, resulting in a block size of 16 MiB:

 $ lzip --list --verbose lfs.img.lz
   dict   memb  trail   uncompressed   compressed   saved  name
   8 MiB   640      0    10737418240   1839569039  82.87%  lfs.img.lz
           ↑↑↑
          members

This file allows B<random access in constant time>.  At most 16 MiB
will have to be decompressed to seek to any byte.  The position of the
member that needs to be decompressed can be computed directly.

If you don't have access to plzip, you can achieve the same result
using the L<split(1)> utility together with regular lzip.

 $ split -b 16m disk.img disk.img.part
 $ lzip disk.img.part*
 $ cat disk.img.part*.lz > disk.img.lz

Do I<not> just use lzip with the I<--member-size> option, since that
option refers to member size, not data block size.

=head2 Multi-member files with variable-size data blocks

Some ways of creating multi-member lzip files, such as the
I<--member-size> option or the L<tarlz(1)> utility, result in data
blocks of varying sizes.  In this case, seeking requires a binary
search on the member index and random access takes logarithmic time.

=head1 PARAMETERS

=over 4

=item B<lzip-max-block=>SIZE

The maximum block size that the filter will read.  The filter will
refuse to read lzip files that contain any data block larger than this
size.  This value refers to size of the uncompressed block, not the
size of the member holding it.

This parameter is optional.  If not specified it defaults to 512M.

=item B<lzip-max-depth=>N

Maximum number of blocks stored in the LRU block cache.

This parameter is optional.  If not specified it defaults to 8.

The filter may allocate up to
S<maximum block size in file × maxdepth>
bytes of memory I<per connection>.

=back

=head1 FILES

=over 4

=item F<$filterdir/nbdkit-lzip-filter.so>

The filter.

Use C<nbdkit --dump-config> to find the location of C<$filterdir>.

=back

=head1 VERSION

C<nbdkit-lzip-filter> first appeared in nbdkit 1.42.  It is derived
from C<nbdkit-xz-filter> which first appeared in nbdkit 1.10.

=head1 SEE ALSO

L<nbdkit(1)>,
L<nbdkit-filter(3)>,
L<nbdkit-curl-plugin(1)>,
L<nbdkit-file-plugin(1)>,
L<nbdkit-bzip2-filter(1)>,
L<nbdkit-gzip-filter(1)>,
L<nbdkit-partition-filter(1)>,
L<nbdkit-xz-filter(1)>,
L<lzip(1)>,
L<plzip(1)>,
L<tarlz(1)>.

=head1 AUTHORS

Jan Felix Langenbach

Richard W.M. Jones

=head1 COPYRIGHT

Copyright Jan Felix Langenbach

Copyright Red Hat
