commit 9c4af5d567298b58f659e32b3c31c6a71ef15d19
parent 5804dc623d51f887728f94325bd4322355b8a6ef
Author: Christian Grothoff <christian@grothoff.org>
Date: Wed, 1 Jul 2026 00:18:51 +0200
expand paivana documentation
Diffstat:
1 file changed, 254 insertions(+), 1 deletion(-)
diff --git a/frags/paivana-httpd-manual.rst b/frags/paivana-httpd-manual.rst
@@ -34,6 +34,90 @@ The full list of command-line options is documented in
documented in :manpage:`paivana.conf(5)`.
+.. _Paivana-Motivation:
+
+Motivation and use cases
+------------------------
+
+``paivana-httpd`` answers a single question for every incoming
+request — *"has the client paid for this?"* — but operators deploy it
+for two rather different reasons. Which one applies to you mostly
+changes how you *price* the templates (see :ref:`Paivana-Templates`),
+not how you install or run the daemon.
+
+Selling access: a paywall with revenue
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The classic use case is to charge money for access to digital content
+or services: a news article, a research dataset, an API call, a file
+download. Because Paivana builds on GNU Taler, this comes with
+properties that distinguish it from traditional paywalls:
+
+- **No accounts, no sign-up.** A visitor pays directly from a Taler
+ wallet; there is no registration, no login, and no credit-card form.
+ This removes the friction that kills conversion on per-article
+ purchases.
+- **Micropayments are practical.** Taler's costs per transaction are
+ low enough that charging fractions of a cent for a single page is
+ economically meaningful, enabling true pay-per-article or
+ pay-per-request business models.
+- **Privacy for the buyer.** Taler is privacy-preserving by design:
+ the seller (and Paivana) learns that *an* order was paid, not *who*
+ paid it. No tracking profile is required to gate the content.
+- **Subscriptions and discounts.** A template's ``choices`` can offer
+ a one-off payment *or* the purchase/redemption of a subscription
+ token, so returning subscribers pass the paywall automatically
+ (see :ref:`Paivana-Templates`).
+- **Agentic payments.** Every paywall response also carries a
+ machine-readable ``Paivana:`` HTTP header (see
+ :ref:`Paivana-Customizing`). An automated client — for example an
+ AI agent provisioned with a Taler wallet — can therefore pay and
+ fetch the resource without rendering the HTML page, while the
+ operator caps financial exposure to whatever was loaded into that
+ agent's wallet.
+
+Rate-limiting bots and abuse: DDoS and scraper protection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The second use case treats the payment not as a source of revenue but
+as a *cost imposed on the client* — economic rate limiting. Here you
+set the price very low (a fraction of a cent, effectively a "CAPTCHA
+token"): negligible for a human visitor reading a handful of pages,
+but ruinous for a botnet or scraper that wants to issue millions of
+requests. An attacker sending a million requests must spend a million
+micropayments; a legitimate user spends almost nothing.
+
+This is a deliberate alternative to two older defenses whose
+cost-asymmetry has eroded:
+
+- **Proof-of-work puzzles** (as used by tools such as Anubis) ask the
+ client to burn CPU before being served. But CPU is cheap and
+ abundant for a well-resourced attacker or botnet, and the puzzle
+ burdens low-powered legitimate devices (phones, assistive tech) the
+ most.
+- **CAPTCHAs** ask the client to solve a perception task. Modern
+ LLMs and automated solvers now clear text, image and even
+ "behavioral" CAPTCHAs cheaply and at scale, so they increasingly
+ inconvenience humans while barely slowing down the bots they were
+ meant to stop.
+
+A Taler payment sidesteps both problems: it does not rely on a human
+being slow, or on a machine being computationally weak. It relies on
+the one resource a flooding attacker genuinely cannot conjure for free
+— money. Bots *can* solve puzzles and CAPTCHAs; they cannot mint
+funds. For this use case you typically combine a low per-request
+price with a subscription-token ``choice`` so that a paying or
+authenticated user buys a token once and then sails through, while
+anonymous floods keep paying per request.
+
+.. note::
+
+ The two use cases are not mutually exclusive: the same deployment
+ can earn revenue on premium URLs and simultaneously throttle abuse
+ on cheap ones, simply by defining several templates with different
+ prices and ``website_regex`` scopes.
+
+
Architecture overview
---------------------
@@ -252,6 +336,100 @@ sent to the journal:
# journalctl -u paivana-httpd -f
+.. _Paivana-Whitelisting:
+
+Whitelisting unpaid resources
+-----------------------------
+
+Most sites contain resources that must remain freely accessible even
+to clients that have not paid: the stylesheets, fonts, logos and
+scripts referenced by the paywall page itself, a ``favicon.ico``, a
+``robots.txt``, or a health-check endpoint polled by a load balancer.
+If these were paywalled, the paywall page could not even render.
+
+The ``WHITELIST`` configuration key holds a single POSIX *extended*
+regular expression that is matched against the request path (which
+always begins with ``/``). When it matches, the request bypasses both
+the paywall and the access-cookie check and is proxied straight to the
+upstream:
+
+.. code-block:: ini
+
+ [paivana]
+ WHITELIST = ^/(favicon\.ico|robots\.txt|assets/.*|.*\.css|.*\.js)$
+
+A few details worth knowing:
+
+- The expression is matched against the URL path only, not against the
+ host or the request method. A match anywhere in the path counts, so
+ anchor with ``^`` and ``$`` if you mean "the whole path".
+- An invalid regular expression is a fatal configuration error:
+ ``paivana-httpd`` logs the problem and refuses to start.
+- The internal endpoints ``POST /.well-known/paivana`` (payment
+ callback) and ``GET /.well-known/paivana/templates/...`` (paywall
+ pages) are always handled by ``paivana-httpd`` itself and are not
+ affected by the whitelist.
+- Running with ``-n`` / ``--no-payment`` whitelists *everything*; the
+ daemon then behaves as a transparent reverse proxy and never
+ consults the merchant backend.
+
+.. note::
+
+ Whitelisting is coarse — it is keyed purely on the URL. Per-URL
+ *pricing* (charging different amounts for different paths) is done
+ with merchant templates and their ``website_regex``, described in
+ :ref:`Paivana-Templates`, not with ``WHITELIST``.
+
+
+.. _Paivana-AccessControl:
+
+Access control: per-page vs. site-wide payment
+----------------------------------------------
+
+Once a client has paid, ``paivana-httpd`` issues an HMAC-protected
+access cookie (``Paivana-Cookie``) instead of asking the merchant
+backend again on every subsequent request. The cookie is a keyed hash
+over *(expiration time, website, client address)*; it carries no
+server-side state, so any of the (possibly several) ``paivana-httpd``
+processes can validate it.
+
+Two aspects of this are operator-visible:
+
+Scope of a payment
+^^^^^^^^^^^^^^^^^^^
+
+By default a payment grants access to **the specific URL that was
+paid for**: the cookie's ``Path`` is set to that path and the website
+is bound into the hash. This is what you want when each page is sold
+individually.
+
+Passing ``-g`` / ``--global-payment`` changes this so that a single
+payment unlocks the **entire site**: the website component is dropped
+from the hash and the cookie is issued with ``Path=/``. Use this when
+one purchase (or one subscription) should cover everything behind the
+proxy, including for the abuse-mitigation use case where you simply
+want a paying client to stop hitting the paywall.
+
+Binding to the client address
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The cookie is tied to the client's network address so that it cannot
+simply be copied to another machine. When ``paivana-httpd`` runs
+behind a TLS-terminating reverse proxy, the socket peer is always that
+proxy, so **every** client would appear to share one address. Run the
+daemon with ``-f`` / ``--respect-forwarded-headers`` in that setup so
+the real client address is taken from ``X-Forwarded-For`` (only do this
+behind a proxy you trust to set that header — see
+:ref:`Paivana-ReverseProxy`).
+
+Cookie lifetime is derived from the order's ``max_pickup_delay``;
+configure it on the template (``max_pickup_duration``) to control how
+long a paid client may keep accessing the resource before paying
+again. Because the cookie is keyed by ``SECRET``, leaving ``SECRET``
+unset means a new random key is chosen at every startup and all
+outstanding cookies are invalidated on restart.
+
+
.. _Paivana-Templates:
Configuring Paivana templates
@@ -437,6 +615,81 @@ After any change, restart ``paivana-httpd`` so the new template
list takes effect.
+.. _Paivana-Customizing:
+
+Customizing the paywall page
+----------------------------
+
+The merchant templates of the previous section decide *which* URLs
+cost *how much*. This section is about the *look* of the page an
+unpaid visitor sees — the HTML the daemon returns with the
+``402 Payment Required`` status.
+
+How the page is rendered
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The paywall body is produced from a `Mustache
+<https://mustache.github.io/>`__ template named ``paywall`` that
+``paivana-httpd`` loads, via ``libtalertemplating``, from
+
+::
+
+ $PREFIX/share/paivana/templates/
+
+Templates are language-specific: the file is named
+``paywall.$LANG.must`` and the daemon picks a variant based on the
+client's ``Accept-Language`` header. The package ships an English
+default, ``paywall.en.must``. To offer the paywall in additional
+languages, drop further ``paywall.<lang>.must`` files into the same
+directory. Rendered responses are cached per language and per
+content-encoding (and carry ``Cache-Control: public`` and a matching
+``Vary`` header), which is what lets the paywall be served as a
+scalable, mostly-static page even under load.
+
+When rendering, ``paivana-httpd`` passes the following values into the
+template:
+
+======================== ====================================================
+Variable Meaning
+======================== ====================================================
+``merchant_backend`` Base URL of the merchant backend (the page
+ long-polls it for payment completion).
+``template_id`` ID of the matched merchant template.
+``summary`` Human-readable summary from the template,
+ if any.
+``choices`` Array of payment options (:ts:type:`OrderChoice`).
+``has_choices`` True if more than one payment option exists.
+``default_choice`` The first entry of ``choices``.
+``max_pickup_delay`` Seconds the order may be picked up within.
+======================== ====================================================
+
+The shipped page uses these to draw a ``taler://`` QR code and a
+"Pay now" link, long-poll the backend, and reload the original URL
+once the wallet confirms. The non-visual integration point — useful
+for automated and agentic clients — is the ``Paivana:`` HTTP response
+header, which carries the ``taler://pay-template/...`` URI directly, so
+a client need not parse the HTML to find out how to pay.
+
+Editing the appearance
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+For small tweaks — colours, wording, your logo, a link back to your
+site — you can edit the installed ``paywall.en.must`` directly; it is
+ordinary HTML/CSS with Mustache ``{{ ... }}`` placeholders for the
+variables above. Keep the ``<script>`` block intact: it is what
+performs the long-poll and the post-payment reload.
+
+The shipped ``paywall.en.must`` is in fact *generated* at build time
+from a Jinja2 source, ``src/frontend/paywall.en.must.j2``, which inlines
+the client-side logic from ``src/frontend/paywall.js`` and a bundled QR
+library. (To avoid clashing with Mustache's ``{{ }}``, the Jinja2
+source uses ``@@ ... @@`` and ``@< ... >@`` delimiters.) If you want to
+change the JavaScript behaviour rather than just the styling, edit the
+``.j2`` source and rebuild — the Meson ``custom_target`` regenerates the
+``.must`` file and installs it into the templates directory shown
+above.
+
+
.. _Paivana-ReverseProxy:
Reverse proxy configuration
@@ -560,7 +813,7 @@ a paywalled URL with ``curl``:
$ curl -i https://example.com/some-article
An unpaid request will first redirect (302 Found) to the static
-paywall page at `/.well-known/paivana/templates/{id}#{base64_website}`
+paywall page at ``/.well-known/paivana/templates/{id}#{base64_website}``
which will then return a ``HTTP/1.1 402 Payment Required`` together
with a Taler-formatted paywall body containing the ``taler://pay/...`` URI of
the freshly created order. Paying that order with any GNU Taler wallet (see
