Concurrency control issues in approx

    Eric Cooper <ecc@cmu.edu>
    June 2008

The priorities for dealing with concurrency in approx are:

1. ensure correctness (no corrupt files delivered to clients or stored
   in the cache)

2. maintain good performance (minimize delays due to serialization,
   minimize number of downloads from remote repositories)

A conflict can occur between concurrent approx processes.

To prevent one approx process from delivering a file that has been
partially downloaded, approx downloads partial files with a unique
temporary extension, and only renames them upon successful completion.
Since the rename is atomic, this potential conflict is a non-problem.

But suppose a large download, like the main Packages.bz2 file, is in
progress.  Another approx process might decide to download it also.
To avoid this, approx creates a hint file in the cache directory
before starting the download.  We use the presence of a current hint
file as an indication that the download is already in progress, and
wait for it (at least for a while).

There is still a race condition between the time that an approx
process checks for the non-existence of a hint file and the time it
creates it, so it is possible for two approx processes to download the
same file simultaneously.  The atomic rename of unique temporary files
ensures correctness in this case, at the cost of some network
bandwidth.
