Paivana
=======

Paivana is an HTTP reverse proxy that gates access to a target website
behind a GNU Taler payment.  Unpaid visitors receive a paywall page
where they can pay using a GNU Taler wallet; once payment is confirmed
the request is forwarded transparently to the configured upstream
server.

The sole binary is `paivana-httpd`.


How it works
------------

0. `paivana-httpd` learns prices from the Paivana templates configured
   in the taler-merchant-backend.  Paivana templates include a regular
   expression which determines the set of pages the template applies to.
   That expression (POSIX extended) is matched against the *entire*
   URL and is anchored at both ends, so `/premium/` applies to nothing
   while `.*/premium/.*` applies to every URL containing it.
1. An HTTP client accesses a page at `paivana-httpd`.
2. If the paywall is enabled for the respective URL and no valid access
   cookie is present, `paivana-httpd` redirects the browser to
   a static paywall HTML page (customizable Mustache template)
   referencing the payment template.
   The page includes a "Paivana" HTTP header to also facilitate agentic
   payments.
3. The browser computes a unique payment identifier and
   renders a dynamic payment request (taler:// QR code
   or link) and long-polls the taler-merchant-backend awaiting
   completion of the payment.
4. The user instructs their Taler wallet to complete the payment.
5. The browser notices that the payment is complete and calls back to
   `POST /.well-known/paivana` (provided by `paivana-httpd`)
   with a reference to the unique payment identifier.
6. `paivana-httpd` verifies the payment with the merchant, sets
   an access cookie, and redirects the browser to the original URL.
   The access ends at the `expiration` the browser named in step 3 and
   repeated in step 5 — it is hashed into the payment identifier, so the
   two must agree — bounded above by the contract's `max_pickup_time`,
   which is what the merchant's `max_pickup_delay` on the template sets.
7. Requests with a valid cookie are forwarded to the upstream server
   via libcurl.  Both directions are streamed: bytes are passed on as
   they arrive, so the size of a proxied body is not bounded by memory
   (see "Architecture notes").

The cookie carries a keyed hash over `(expiration time, website, client
address)` keyed by a `paivana_secret` derived from the configured
`SECRET`.  The construction is `GNUNET_CRYPTO_hkdf_gnunet()`, which is
HMAC-based but is not itself an HMAC.  `SECRET` is required whenever the
paywall is on: it is the only input to that hash the client does not
supply, so a key invented afresh at each start would invalidate every
access anyone had already paid for.  Under `-n` no cookie is ever minted
and none is needed.

Keep the same `SECRET` on every `paivana-httpd` serving one site — a
cookie issued by one has to verify on the next — and treat it as
equivalent to the content itself: whoever holds it can mint access for
any visitor, URL and expiry.


Dependencies
------------

- GNUnet (libgnunetutil, libgnunetjson, libgnunetcurl)
- libmicrohttpd
- libcurl >= 7.62.0
- libjansson
- libgcrypt
- GNU Taler: libtalerutil, libtalerjson, libtalermerchant,
             libtalermhd, libtalertemplating

That is what the binary links against; `readelf -d` on it is the
authority.  Paivana does not use libtalerexchange, and compresses
nothing itself — that happens inside libtalermhd, which brings zlib
with it.


Build
-----

The project uses Meson but supports a GNU build process.

    ./bootstrap
    ./configure --prefix=$TARGET
    make
    sudo make install


Configuration
-------------

Paivana reads an INI-style `.conf` file.  The only section used is
`[paivana]`.  A minimal working configuration:

    [paivana]
    DESTINATION_BASE_URL = https://example.com/
    MERCHANT_BACKEND_URL  = https://backend.demo.taler.net/instances/sandbox/
    MERCHANT_ACCESS_TOKEN = secret-token:sandbox
    BASE_URL = http://localhost:9967/
    SERVE = tcp
    PORT  = 9967

### Required keys

  Key                     Description
  ----------------------  -----------------------------------------------------
  DESTINATION_BASE_URL    Upstream server to proxy to once payment is confirmed.
  MERCHANT_BACKEND_URL    Base URL of the Taler merchant backend.
  MERCHANT_ACCESS_TOKEN   Bearer token for all calls to the merchant backend.
  BASE_URL                Public base URL of Paivana.  Required unless `-f`
                          is given, in which case it is derived from the
                          forwarding headers (see below).
  SECRET                  Key for the access-cookie MAC.  Required unless
                          `-n`; see above for why there is no sensible
                          default.  It is *not* an input to the Paivana ID,
                          which the browser has to be able to recompute on
                          its own.  The Debian package generates one into
                          /etc/paivana/secrets/paivana.secret.conf, which
                          is not world-readable.
  SERVE                   `tcp`, `unix` (Unix-domain socket) or `systemd`
                          (socket activation).  There is no default: an
                          instance without it exits at startup, and it does
                          so only after the templates have been fetched.
  PORT                    TCP port.  Required when SERVE = tcp.
  UNIXPATH                Path to bind to.  Required when SERVE = unix.
  UNIXPATH_MODE           Access mode of that socket, octal.  Required
                          when SERVE = unix — it is what governs who may
                          reach Paivana, since a Unix-domain peer is
                          trusted to report the client address.

### Optional keys

  Key       Description
  --------  ---------------------------------------------------------------
  WHITELIST POSIX extended regular expression; matching request paths
            are forwarded without payment.  Matched against the
            *entire* path and anchored at both ends, so `/free/`
            whitelists nothing while `/free/.*` whitelists that
            subtree.
  TRUSTED_PROXIES
            IPv4 networks whose members are reverse proxies trusted to
            report the client address in `Forwarded` or
            `X-Forwarded-For`.  Only consulted with `-f`, and only
            needed when more than one proxy is in front.  See "Trusted
            proxies" below.
  TRUSTED_PROXIES6
            IPv6 counterpart of TRUSTED_PROXIES.
  CONNECTION_LIMIT
            Total number of concurrent client connections to accept,
            default 512.  Divided evenly over the listen sockets that
            come up, so the process-wide total is what you set --
            with no BIND_TO there are two (IPv4 and IPv6).  Paivana
            also spends file descriptors on outbound requests from
            the same table, so leave headroom below `ulimit -n`.
  PER_IP_CONNECTION_LIMIT
            Concurrent connections accepted from any one client
            address, default 32; 0 disables the check.  Set it to 0
            wherever the peer address is not the client's -- under
            SERVE = unix or systemd every client shares one peer
            address, and behind a reverse proxy or a NAT many clients
            do, so a limit there throttles everyone at once.
  BIND_TO   IP address to bind to; dual-stack wildcard if absent.
  DESTINATION_UNIXPATH
            Unix-domain socket to reach the upstream on instead of
            connecting to the authority in `DESTINATION_BASE_URL`.  That
            URL is still what the request line and `Host` are built
            from, so it remains required.
  MERCHANT_BACKEND_UNIX_PATH
            The same for the merchant backend, relative to
            `MERCHANT_BACKEND_URL`.  An unusable value is a warning and
            the setting is then ignored, not a startup failure.


Running
-------

$ paivana-httpd -c /etc/paivana/paivana.conf

Besides the options GNUnet gives every program (`-c` / `--config`,
`-L` / `--log`, `-l` / `--logfile`, `-h`, `-v`), Paivana takes four:

  -n, --no-payment    Bypass the paywall entirely — a pure reverse proxy,
                      which is what the test suite runs.  No cookie is
                      ever minted and `SECRET` is not required.
  -g, --global-payment
                      One payment grants access to the whole site rather
                      than to the URL it was made for.  The website goes
                      into the cookie's keyed hash as the empty string,
                      and the cookie is scoped to `/` rather than to the
                      page, so cookies minted under one setting do not
                      verify under the other: flipping it invalidates
                      whatever access is outstanding.
  -f, --respect-forwarded-headers
                      Take the client address from the forwarding
                      headers.  Only safe behind a proxy that overwrites
                      them; see "Deployment behind a reverse proxy".
  -u, --max-upload BYTES
                      Bytes of a request body held in memory at once
                      while relaying it upstream, default 262144.  A
                      throughput knob, not a limit: the largest body
                      accepted is MAX_REQUEST_SIZE.  For configurations
                      written when these were one number, setting this
                      and not MAX_REQUEST_SIZE still sets both.

The daemon does not serve requests until it has fetched paywall templates
from the merchant backend.  If template loading fails, startup is aborted:
a template that could not be loaded would otherwise leave everything it
covers unpaywalled.  Transient backend outages are the service manager's
job to ride out -- the shipped `paivana-httpd.service` restarts with an
increasing back-off, and does not restart on a configuration error.

An instance that offers no template at all is refused for the same
reason: with nothing to sell, no URL ever matches a paywall and the
entire site would be served for free without a word of warning.  Serving
a site without a paywall is what `-n` is for, and it has to be asked for.


Deployment behind a reverse proxy
----------------------------------

The recommended production setup runs Paivana over a Unix socket and
places nginx or Apache in front for TLS termination.

In that setup Paivana **must** be started with `-f` /
`--respect-forwarded-headers`.  A Unix-domain peer has no address of
its own, so without `-f` there is no client address at all: the access
cookie cannot be bound to a client, and `POST /.well-known/paivana`
fails.  `-f` makes Paivana take the client address from the forwarding
headers instead, and forward the chain it was given to the upstream
rather than replacing it.

Paivana reads both the RFC 7239 `Forwarded` header and the de-facto
`X-Forwarded-*` ones, preferring `Forwarded` where both are present,
and emits both upstream — the standardized one for origins that speak
it, the de-facto ones for the many that do not.  Of the `X-Forwarded-*`
family it emits `-For`, `-Proto`, `-Host` and `-Port`; the port is
taken from the front end's `X-Forwarded-Port` under `-f`, and otherwise
from the authority in `Host` when that names one.

What the origin sees as `Host` is **not** what the client sent: it is
the authority of `DESTINATION_BASE_URL`, because that is the name
Paivana connects to.  The client's own value survives as
`X-Forwarded-Host` (and as the `host` parameter of `Forwarded`).  An
origin doing virtual hosting must therefore be configured for the
`DESTINATION_BASE_URL` authority, and an origin that generates absolute
URLs should be told to build them from `X-Forwarded-Host` /
`X-Forwarded-Proto` / `X-Forwarded-Port`.  If it builds them from
`Host` instead, its `Location` values will name Paivana's view of the
origin — an internal host and port, which Paivana relays unchanged.
That leaks the internal name, and points the client straight at the
origin wherever the client can route to it, bypassing the paywall.

`-f` is only safe if the server in front **writes** the forwarding
headers itself, whether by overwriting them or by appending its own
element.  What it must not do is pass the client's copies through
untouched: Paivana believes the rightmost element (see "Trusted
proxies"), so a header no hop of yours has written is a header the
client filled in, and the client then chooses the identity its access
cookie is bound to.  The configurations below get this right; if you
write your own, note that nginx sets `X-Forwarded-For` only when told
to and forwards a client-supplied `Forwarded` verbatim — and
`Forwarded` is the one Paivana prefers.

Conversely, do not pass `-f` to a Paivana that clients can reach
directly — there it is the client, not a proxy, that is setting those
headers.

nginx (`/etc/nginx/sites-available/paivana`):

    server {
        listen 443 ssl;
        server_name example.com;

        location / {
            proxy_pass http://unix:/run/paivana/httpd/paivana-http.sock;
            proxy_set_header Host $host;

            # $remote_addr, not $proxy_add_x_forwarded_for: this is
            # the outermost hop, so these overwrite rather than
            # extend what the client claimed.
            proxy_set_header X-Forwarded-For   $remote_addr;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-Forwarded-Host  $host;
            proxy_set_header X-Forwarded-Port  $server_port;

            # RFC 7239; preferred by Paivana over the above.  The
            # element is built by a `map` — see the shipped config.
            proxy_set_header Forwarded \
                "$paivana_forwarded_elem;proto=$scheme;host=$host";
        }
    }

Apache (requires mod_proxy, mod_proxy_http and mod_headers):

    <Location "/">
        # mod_proxy appends the real client to any X-Forwarded-For the
        # client itself sent, so drop the client's copies first.
        RequestHeader unset X-Forwarded-For
        RequestHeader unset X-Forwarded-Proto
        RequestHeader unset X-Forwarded-Host
        RequestHeader unset X-Forwarded-Port
        RequestHeader unset Forwarded

        # RFC 7239; Apache emits none of its own.  It has to be an
        # expr= value: %{...}e reads the CGI environment, which is not
        # populated when mod_headers runs, so the %{REMOTE_ADDR}e form
        # yields the literal string "(null)".
        RequestHeader set Forwarded \
            "expr=for=%{REMOTE_ADDR};proto=%{REQUEST_SCHEME};host=%{HTTP_HOST}"

        ProxyPass "unix:/run/paivana/httpd/paivana-http.sock|http://example.com/"
    </Location>

Ready-made versions of both are shipped in `debian/examples/`, and
installed by the Debian package into
`/usr/share/doc/paivana-httpd/examples/`.  They are examples rather
than drop-ins on purpose: the package only *recommends* a web server,
so it must not create `/etc/nginx/` or `/etc/apache2/` on a system
that has neither (Debian Policy 9.1.1).  Copy the one you want into
place and enable it yourself.


Trusted proxies
---------------

`-f` on its own extends trust exactly one hop.  The chain is walked
from the right and the walk stops at once, so the client is the
*rightmost* element — the one the peer we accepted the connection from
wrote.  Entries a client prepends to its own header sit to the left of
that and cannot be promoted.

What `-f` alone therefore rests on is that the server in front sets or
appends those headers itself.  Appending is safe here, because the real
peer ends up rightmost; passing the client's own headers through
unchanged is not, and that is the failure to watch for, since nginx
forwards a client-supplied `Forwarded` verbatim and Paivana prefers
`Forwarded`.

`TRUSTED_PROXIES` and `TRUSTED_PROXIES6` are what let the walk step
further left, through hops you have listed, when there is more than one
proxy in front:

    [paivana]
    TRUSTED_PROXIES  = 10.0.0.0/8;192.168.0.0/16;
    TRUSTED_PROXIES6 = 2001:db8::/32;

The walk then steps over each listed proxy in turn; the first element
that is not one of them is the client.  Each step leftwards is
permitted only by the node being stepped over, so an element written by
someone you did not list is as far back as the chain can be believed.

Two things this does *not* do.  The address that connected is never
matched against these lists — `-f` is what says the peer may speak for
a client, and the lists only govern how far past it the walk may go.
And an element that names no address (RFC 7239 `unknown`, an obfuscated
identifier, a host name) is not skipped either: the walk stops there
and Paivana falls back to the socket peer, which under `SERVE = unix`
means no client address at all and a `POST /.well-known/paivana` that
fails.  A front server that emits a `Forwarded` element it cannot fill
in is therefore not a degraded paywall but a broken one.

`X-Forwarded-Proto`, `-Host` and `-Port` are outside all of this: they
are read as the leftmost value of the first such field line and the
walk never vets them.  Under `-f` with no `BASE_URL` they are what
Paivana rebuilds its own scheme and authority from, which is the
residual reason to care about the front server's configuration even
with `TRUSTED_PROXIES` set.

Syntax notes, inherited from GNUnet's network-policy parser:

  - entries are separated *and terminated* by `;` — a missing trailing
    semicolon means nothing is parsed;
  - `TRUSTED_PROXIES6` does not tolerate spaces between entries
    (`TRUSTED_PROXIES` does);
  - `0.0.0.0/0` and `::/0` cannot be expressed: they are
    indistinguishable from the end of the list.

Anything that parses to an empty list is refused at startup rather
than silently trusting nobody.

Put IPv4 proxies in `TRUSTED_PROXIES`, not in `TRUSTED_PROXIES6` as
`::ffff:a.b.c.d`: addresses are folded to their IPv4 form before
matching, so a mapped entry would never be hit.

Set `BASE_URL` in the configuration file to the public HTTPS URL so
that redirects and cookie domains are correct.  It may be omitted only
when `-f` is given: the flag asserts that a reverse proxy in front of
Paivana has already enforced a correct `Host`, which is what makes it
safe to reconstruct our own URL from the request.  Without `-f` the
client is assumed to have connected directly, `Host` is whatever it
chose to send, and `BASE_URL` is therefore mandatory.


Source layout
-------------

    src/backend/          Main binary and all subsystems
      paivana-httpd.c     Entry point, scheduler, global state, shutdown
      paivana-httpd_reverse.c   Request-proxying state machine (core)
      paivana-httpd_pay.c       POST /.well-known/paivana handler
      paivana-httpd_cookie.c    Access-cookie keyed hash, Paivana ID
      paivana-httpd_templates.c Paywall template loading and rendering
      paivana-httpd_helper.c    Client IP / base URL helpers
      paivana-httpd_daemon.c    MHD daemon startup
      paivana_pd.c              GNUnet project-data descriptor
    src/frontend/         The paywall page served to unpaid visitors
      paywall.en.must.j2  Mustache template source (Jinja2)
      paywall.js          Payment identifier, QR code, long poll
      generate-paywall.py Renders the Jinja2 source at build time
    src/include/platform.h  GNUnet-style platform header (include first)
    src/tests/              Automated reverse-proxy and unit tests
    doc/prebuilt/           Git submodule: taler-docs (man pages)


Architecture notes
------------------

Single-threaded event loop: GNUnet scheduler drives both inbound HTTP
(libmicrohttpd) and outbound requests (libgnunetcurl / libcurl multi).
Running multiple `paivana-httpd` processes on the same port is
supported as the main way to scale-up the system.

Requests and responses are streamed in both directions: each is moved
through a fixed-size ring buffer (REQUEST_BUFFER_MAX and
RESPONSE_BUFFER_MAX, 256 KiB each by default) rather than assembled
whole, so the size of a proxied body is bounded by nothing in Paivana.
When the client cannot keep up, Paivana stops reading from the origin;
when the origin cannot keep up, it stops reading from the client.  The
memory an in-flight request costs is therefore the two buffers, and the
worst case is that times CONNECTION_LIMIT.

An upload is still bounded, by MAX_REQUEST_SIZE (1 MiB by default),
because accepting one is a policy decision rather than a memory
constraint.  A response is not bounded at all: an operator who wants to
bound what their origin serves can do it at the origin.

The MHD daemon is not started until paywall templates have been fetched
from the merchant backend asynchronously.

A 1xx interim response is not forwarded — RFC 9110 §15.2 asks a proxy
to forward them, and Paivana instead drops them, because the response
MHD is handed is a single final one.  `103 Early Hints`
therefore does not reach clients through Paivana.  Its header fields
are dropped with it rather than being merged into the final response,
which is the part that would be actively harmful.  Trailer fields are
dropped for the same reason (RFC 9110 §6.5.1 forbids merging them into
the header section).

An origin that accepts the connection but does not produce response
*headers* within `UPSTREAM_TIMEOUT` (60 s) yields `504 Gateway
Timeout`; one that cannot be reached at all yields `502 Bad Gateway`.
The distinction matters because caches and monitoring retry the former
and not the latter.  That clock is cancelled once the header section
ends, and it is the only one that can still produce a status code:
after it, the status is already on the wire.

There is deliberately no ceiling on how long a request may take — a
large download legitimately runs for as long as it runs.  What is
bounded instead is a *stall*: `UPSTREAM_STALL_TIMEOUT` (60 s) is how
long the origin may move no bytes in either direction.  The clock does
not run while Paivana is itself holding the origin back because the
client has not drained what has already arrived, so a client on a slow
link is never mistaken for a slow origin.

The MHD connection timeout does not cover any of this: a connection
waiting on the origin is suspended, and MHD drops suspended connections
from its timeout lists.

Once the response headers have gone out the status cannot be retracted,
so an origin that fails mid-body can only be reported as a framing
error: a declared `Content-Length` that is not met, or a chunked
response closed without its terminating chunk.  Both are required to be
treated as failures by RFC 9112 §8.1.2.  The exception is an HTTP/1.0
client receiving a response of unknown length, where the close *is* the
framing and truncation is indistinguishable from success.

`OPTIONS` carrying `Max-Forwards: 0` is answered by Paivana itself with
an `Allow` list, as RFC 9110 §7.6.2 requires of an intermediary; any
larger value is decremented before the request is passed on.


License
-------

GNU Affero General Public License version 3 or later.
See COPYING for the full text.


Bug reports
-----------

Please report bugs at https://bugs.taler.net/.
