commit 9023ae3c4dc7aabc7e7327b481cfdc60d7947d35
parent f064277e00790ca3309e843d6d7e2c23c3d96aec
Author: Christian Grothoff <christian@grothoff.org>
Date: Sat, 21 Nov 2026 22:53:20 +0100
add DD for beekeeper
Diffstat:
2 files changed, 350 insertions(+), 0 deletions(-)
diff --git a/design-documents/107-beekeeper.rst b/design-documents/107-beekeeper.rst
@@ -0,0 +1,349 @@
+DD 107: Beekeeper: Resumable Forms and KYC Media Collection
+###########################################################
+
+:Design status: Draft
+:Implementation status: untested
+:DD shepherd: Christian Grothoff
+:Historical contributors: Christian Grothoff
+:First published: 2026-09-29
+:Last substantive change: 2026-11-21
+
+Summary
+=======
+
+Beekeeper is a new, standalone service that collects structured information
+from users through forms defined in JSON. Forms have several pages and
+conditional questions, and users can fill them in over several sessions.
+Besides ordinary fields, Beekeeper can collect KYC-grade evidence: HD selfie
+videos with a liveness challenge, videos of directed movements, identity
+documents (image, MRZ and optionally the ePassport chip), business
+documentation (share registers, company registration data) and phone numbers
+and postal addresses that are verified by SMS and by letter. A single JSON
+REST API serves a single-page web app and native iOS and Android apps.
+Administrators manage forms, invite users, review submissions and send them
+back for correction.
+
+Beekeeper is a standalone product that can also act as an OAuth2 KYC
+provider for the Taler exchange, as the existing KYC providers do.
+
+Motivation
+==========
+
+Taler's KYC processes rely on external providers or on hand-built
+AML/KYC forms. Operators need a way to collect customer information that
+they can host themselves:
+
+* Requirements change with jurisdictions and regulators. So the questions
+ should be data, not code, and changing them should not require an app
+ release.
+* KYC evidence (identity documents, liveness videos, verified addresses) is
+ highly sensitive. Sending it to a third party is a privacy and compliance
+ risk, and many deployments would rather keep it in-house.
+* Users rarely finish a long KYC process in one sitting. They need to leave,
+ come back from another device and go back to earlier pages, and they need to
+ be told when a document must be submitted again.
+
+The same machinery is useful beyond KYC, for example for customer surveys or
+movement assessments. So Beekeeper is designed as a general form service,
+with KYC as its most demanding use case.
+
+Requirements
+============
+
+* Forms are JSON documents: questions, answer types, validation, a sequence
+ of pages, and questions that are shown only under conditions. Uploading a
+ form rejects inconsistent definitions up front.
+* Conditions and validators give the same result on the server, in the
+ browser and in both native apps.
+* Users fill in forms incrementally and resumably. They see how many steps
+ remain and can go back.
+* Each user has a personal link. A form can allow recovering a lost link
+ through a verified contact e-mail address, and can require an e-mail
+ one-time code in addition to the link.
+* Public forms are listed on the start page; if there is only one, a single
+ "start" button is shown.
+* Administrators can upload, inspect, publish, unpublish and retire forms;
+ create invitation links with or without an e-mail address; assign a form to
+ a user, or replace a user's form with another one; review, accept, reject
+ or return submissions; and export them.
+* Answers are stored in PostgreSQL as JSON; media are stored as large objects
+ keyed by their SHA-256 hash. Administrators can list media sorted by date or
+ size, view them and delete them in bulk.
+* Upload types: the usual HTML fields; HD selfie videos with liveness
+ detection; videos of directed movements; identity documents with MRZ;
+ phone numbers verified by SMS; postal addresses verified by letter. The
+ user is told when evidence has expired or failed and must be submitted
+ again.
+* The personal data must be protected in line with KYC regulations: encrypted
+ at rest, with a retention period per form and with administrator access
+ recorded in an audit log.
+
+Proposed Solution
+=================
+
+The detailed specifications live in the Beekeeper repository (``docs/``).
+This DD summarises the design and the reasons behind it.
+
+Components
+----------
+
+Beekeeper is written in Rust, with the crates used by ``taler-rust`` (axum,
+hyper, sqlx, tokio, clap, tracing). It consists of these parts:
+
+* ``beekeeper-httpd`` serves the REST API, including resumable uploads, and
+ the static web front ends. It is the only process that holds the master
+ key.
+* ``beekeeper-analyzer`` workers analyse media. They take jobs from a queue
+ in PostgreSQL (``FOR UPDATE SKIP LOCKED``, woken by ``LISTEN/NOTIFY``).
+ Because they decode untrusted video with FFmpeg, they run sandboxed:
+ landlock and seccomp, no network access apart from the database socket, and
+ a database role that can read only its own job. Analyzers can run on
+ separate GPU machines.
+* ``beekeeper-admin`` is a command-line tool for database setup, the first
+ administrator account, key rotation, garbage collection and retention.
+* Messages are delivered by helper programs for e-mail, SMS and letters.
+ They use the same calling convention as challenger's ``challenger-send-*``
+ scripts, so the existing provider integrations can be reused.
+
+The SPA and the admin UI live in the Beekeeper repository itself (one
+package, two bundles), using established libraries for resumable uploads
+(tus-js-client), the proof of work (ALTCHA) and live camera guidance
+(MediaPipe tasks-vision). The native apps are written in Swift and Kotlin,
+mainly because reading the ePassport chip over NFC needs NFCPassportReader on
+iOS and JMRTD on Android. They live in the same repository (``apps/``), so
+that an API or expression change lands in one commit across server, SPA and
+apps. Each has a platform-independent core (evaluator, form model, API and
+tus client) that is tested on Linux; the Android app uses Jetpack Compose and
+CameraX and no Google Play services, so that it can be built for F-Droid.
+
+The server builds on ``taler-common`` from ``taler-rust`` (configuration,
+database migrations, logging) and on a vendored copy of ``taler-api``, whose
+changes are kept as a series of patches meant for upstream. Its error codes
+are registered in GANA, in a registry of their own
+(``beekeeper-error-codes``, values 100000 to 110000), and returned in the
+usual ``ErrorDetail`` format.
+
+The analyzer uses Debian's builds of ONNX Runtime, FFmpeg and Tesseract. The
+models are distributed in a separate ``beekeeper-models`` repository with
+their provenance and conversion scripts.
+
+Form language
+-------------
+
+A form is a JSON document with pages, and each page has a list of typed
+questions: ``info``, ``text``, ``number``, ``boolean``, ``choice``,
+``date``, ``email``, ``phone``, ``postal_address``, ``file``, ``video`` and
+``identity_document``. Labels can be given per language. Each question can
+have ``required`` and ``visible_if`` conditions and a list of validators.
+Options per form cover manual or automatic acceptance, the e-mail OTP,
+whether users may go back, the retention period, and when evidence must be
+resubmitted.
+
+Conditions use JSONLogic *syntax* with a small set of operators (comparison,
+arithmetic, string operations, a portable regular-expression subset, date
+arithmetic and ``verified``). Their *semantics* are defined strictly rather
+than by "whatever JavaScript does":
+
+* Equality is strict and deep.
+* Type errors are errors, not coercions.
+* String lengths and comparisons are defined over Unicode scalar values.
+* ``\d``, ``\w`` and ``\s`` mean their ASCII classes.
+
+Only this makes four independent implementations agree. A shared set of
+conformance test vectors, currently 215, keeps them in step. A form is
+rejected if an expression refers to a later question, so visibility is
+computed in one pass in document order, and the progress indicator stays
+stable while the user fills in later pages.
+
+Users and authentication
+------------------------
+
+Each user has a personal link. Its secret sits in the URL fragment, so it
+never reaches server logs or ``Referer`` headers, and the client exchanges it
+for a session.
+
+* **One-time code**: a form may require an e-mail one-time code on top of
+ the link.
+* **Recovery**: if the user has a verified contact address, they can request
+ a new link by e-mail. The old link stays valid until the new one is first
+ used, so a third party who requests a recovery cannot lock the user out.
+* **Replaced forms**: when a user's form is replaced with another, answers
+ are carried over by question id and type. A user can have several open
+ assignments, in an order the administrator chooses.
+
+Administrators have database accounts with the roles reviewer, manager,
+superuser and service, and log in with a password and TOTP. Every
+administrator action is written to an audit log.
+
+Verification
+------------
+
+E-mail addresses and phone numbers are verified with codes while the user is
+on the page. A postal address is verified by a letter with a code. The form
+can be submitted while the letter is in transit. If the code is not entered
+before the deadline, or a later media analysis fails, the assignment goes back
+to the user in the ``returned`` state, with the affected pages marked.
+
+Media, liveness and identity documents
+--------------------------------------
+
+Uploads use the tus 1.0 protocol (with the creation and checksum
+extensions), which lets large videos survive unreliable mobile connections.
+A user can refer only to blobs that they uploaded themselves; knowing a
+SHA-256 hash is not enough.
+
+For a liveness video, the server issues a random sequence of actions (for
+example "turn head left, blink, smile") shortly before recording, and the
+video must show them in the right order and time windows. The analyzer runs
+open models through ONNX Runtime (``ort``):
+
+* YuNet for face detection;
+* the MediaPipe face and hand landmark models for eye, mouth and hand actions;
+* head pose from landmarks;
+* RTMPose or BlazePose for arm and body movements;
+* MiniFASNet for passive anti-spoofing;
+* SFace to compare the face with the document photo.
+
+The MRZ is read with Tesseract using the OCR-B training data, and its check
+digits are verified. For NFC passports, the server checks the chip's
+security object against the CSCA master list with RustCrypto ``cms`` and
+``x509-cert``.
+
+Each check yields a score. Clear passes and clear failures are decided
+automatically; everything in between goes to a human reviewer, who sees the
+per-action timings and key frames. Vendor services can be added later
+through a trait that takes a blob and a challenge and returns the same result
+structure. InsightFace is not used because its models are licensed for
+non-commercial use only.
+
+Encryption at rest
+------------------
+
+Envelope encryption protects the data:
+
+* **Keys**: a master key file wraps a random data key per user and per blob.
+* **Answers and addresses** are encrypted with XChaCha20-Poly1305. The
+ row's identifiers are bound in as associated data, so a ciphertext cannot
+ be moved to another row.
+* **Blobs** are encrypted in 64 KiB chunks (STREAM), so ranged reads and
+ resumed uploads work.
+* **Deletion** removes the data key first ("crypto-shredding"), which makes
+ the data unreadable in backups too.
+* **Left unencrypted** are identifiers, states, timestamps, sizes and hashes,
+ so that the admin UI can list and sort. Contact addresses are found through
+ an HMAC of the normalised address.
+* **The analyzer** never sees the master key. Each job carries the blob key
+ wrapped for that analyzer's X25519 key.
+
+Output
+------
+
+Administrators get submissions through the admin UI, as a JSON or ZIP export,
+and through a change feed with long polling. There are deliberately no
+webhooks: consumers pull, so there is no outbound delivery to retry and no
+receiver to authenticate.
+
+Taler integration
+-----------------
+
+Beekeeper implements the protocol of the exchange's ``oauth2`` KYC logic in
+the same shape as challenger: ``POST /oauth2/setup/$CLIENT_ID`` (with the
+client secret; the measure's ``initial_address`` pre-fills answers of the
+same name), ``GET /oauth2/authorize/$NONCE``, ``POST /oauth2/token`` and
+``GET /oauth2/info``. A client (registered with ``beekeeper-admin
+oauth-client add``) names the forms each process assigns; the process is one
+Beekeeper user, so administrators can assign further forms or return answers
+as usual.
+
+As the exchange takes the answer of ``/oauth2/info`` as final, the user is
+handed back only once every assignment of the process is accepted,
+automatically or by a reviewer; until then the web app shows that the answers
+are being checked, and opening the exchange's KYC link again resumes the
+process. A rejected or expired assignment hands the user back with
+``error=access_denied``.
+
+``/oauth2/info`` returns the visible answers, the verified questions and the
+analysis outcomes. Files are inlined as the exchange's ``{FILENAME,
+MIME_TYPE, CONTENTS}`` objects for the question types the client chose and up
+to its size limit, and given as metadata (SHA-256, type, size) otherwise.
+The converter ``taler-exchange-kyc-beekeeper-converter`` maps question ids to
+attribute names by upper-casing them (``full_name`` gives ``FULL_NAME``).
+
+Test Plan
+=========
+
+* **Conformance**: the expression vectors run in the Rust test suite and, as
+ they are written, in the TypeScript, Swift and Kotlin test suites.
+* **Static checks**: unit tests, and every example form in the repository
+ must pass them.
+* **Server**: integration tests against a real PostgreSQL database for page
+ saves, resumption, submission, carry-over, returns, recovery and OTP.
+ Also: token revocation, and that users cannot access other users' blobs.
+* **Uploads**: tus conformance tests, including interrupted and resumed
+ uploads and checksum mismatches.
+* **Verification**: the delivery helpers are replaced by scripts that record
+ the message, so tests can read the code.
+* **Analyzer**: a corpus of recorded videos (genuine; photo, replay and mask
+ attacks; wrong action order) with expected outcomes. Thresholds are tuned
+ on it, and regressions are measured against it.
+* **Sandbox**: tests that the analyzer cannot open network sockets or files
+ outside its model directory.
+
+Definition of Done
+==================
+
+* Forms can be uploaded, checked, published and retired; users can be invited
+ and can fill in, resume, go back and submit on the SPA and on both native
+ apps.
+* E-mail, SMS and letter verification work through challenger-compatible
+ helper programs.
+* Liveness, directed movement, MRZ and face-match analysis run in the
+ sandboxed analyzer, with a human review queue.
+* All personal data is encrypted at rest; retention and crypto-shredding are
+ implemented; administrator actions are audited.
+* Export and the change feed are available and documented in the API
+ reference.
+* The exchange can use Beekeeper as an OAuth2 KYC provider; the round trip
+ through the exchange's ``oauth2`` plugin (``taler-exchange-kyc-tester``)
+ yields the expected attributes.
+
+Alternatives
+============
+
+* **A commercial KYC vendor for liveness** would give certified
+ presentation-attack detection (ISO/IEC 30107-3), but would send the most
+ sensitive data to a third party and bind deployments to its pricing. It
+ remains possible through the analysis trait.
+* **Extending challenger** would reuse its address verification, but
+ challenger verifies one address per process, and multi-page forms, media and
+ review do not fit its model. Beekeeper reuses its helper programs instead.
+* **An existing form standard** (JSON Schema with a UI schema, SurveyJS,
+ FHIR Questionnaire) would bring existing tooling. None of them covers the
+ media and verification types, and their condition languages lack
+ semantics that can be reproduced exactly in four languages.
+* **Plain JSONLogic** has libraries for all four platforms, but these
+ libraries disagree on coercion, equality and string handling. Conditions
+ that behave differently on the server and in the app would be a source of
+ bugs and of rejected submissions.
+* **A TypeScript or Kotlin backend** would share code with a client. Rust
+ fits the existing ``taler-rust`` code base and its deployment, and is
+ memory-safe near FFmpeg and the ONNX models.
+* **Webhooks** would push results to consumers, but they need retries,
+ signing and a receiver. Long polling gives similar latency without these.
+
+Drawbacks
+=========
+
+* **Own pipeline code**: without MediaPipe's runtime and OpenCV, we have to
+ write the model pre- and post-processing ourselves. This is most of the
+ analyzer work.
+* **Four evaluators**: the condition language has to be implemented four
+ times. The conformance suite reduces this risk but does not remove it.
+* **Consequences of encryption**: the database cannot search inside answers,
+ and losing the master key loses all data.
+
+Discussion / Q&A
+================
+
+* Open: the expected scale (number of users and media volume), which decides
+ whether large objects in the database remain adequate for media storage.
diff --git a/design-documents/index.rst b/design-documents/index.rst
@@ -119,4 +119,5 @@ documents. The lifecycle metadata in each document is authoritative. See
103-https-wallet-links
104-wire-gateway-management-API
106-currency-conversion
+ 107-beekeeper
999-template