107-beekeeper.rst (17042B)
1 DD 107: Beekeeper: Resumable Forms and KYC Media Collection 2 ########################################################### 3 4 :Design status: Draft 5 :Implementation status: untested 6 :DD shepherd: Christian Grothoff 7 :Historical contributors: Christian Grothoff 8 :First published: 2026-09-29 9 :Last substantive change: 2026-11-21 10 11 Summary 12 ======= 13 14 Beekeeper is a new, standalone service that collects structured information 15 from users through forms defined in JSON. Forms have several pages and 16 conditional questions, and users can fill them in over several sessions. 17 Besides ordinary fields, Beekeeper can collect KYC-grade evidence: HD selfie 18 videos with a liveness challenge, videos of directed movements, identity 19 documents (image, MRZ and optionally the ePassport chip), business 20 documentation (share registers, company registration data) and phone numbers 21 and postal addresses that are verified by SMS and by letter. A single JSON 22 REST API serves a single-page web app and native iOS and Android apps. 23 Administrators manage forms, invite users, review submissions and send them 24 back for correction. 25 26 Beekeeper is a standalone product that can also act as an OAuth2 KYC 27 provider for the Taler exchange, as the existing KYC providers do. 28 29 Motivation 30 ========== 31 32 Taler's KYC processes rely on external providers or on hand-built 33 AML/KYC forms. Operators need a way to collect customer information that 34 they can host themselves: 35 36 * Requirements change with jurisdictions and regulators. So the questions 37 should be data, not code, and changing them should not require an app 38 release. 39 * KYC evidence (identity documents, liveness videos, verified addresses) is 40 highly sensitive. Sending it to a third party is a privacy and compliance 41 risk, and many deployments would rather keep it in-house. 42 * Users rarely finish a long KYC process in one sitting. They need to leave, 43 come back from another device and go back to earlier pages, and they need to 44 be told when a document must be submitted again. 45 46 The same machinery is useful beyond KYC, for example for customer surveys or 47 movement assessments. So Beekeeper is designed as a general form service, 48 with KYC as its most demanding use case. 49 50 Requirements 51 ============ 52 53 * Forms are JSON documents: questions, answer types, validation, a sequence 54 of pages, and questions that are shown only under conditions. Uploading a 55 form rejects inconsistent definitions up front. 56 * Conditions and validators give the same result on the server, in the 57 browser and in both native apps. 58 * Users fill in forms incrementally and resumably. They see how many steps 59 remain and can go back. 60 * Each user has a personal link. A form can allow recovering a lost link 61 through a verified contact e-mail address, and can require an e-mail 62 one-time code in addition to the link. 63 * Public forms are listed on the start page; if there is only one, a single 64 "start" button is shown. 65 * Administrators can upload, inspect, publish, unpublish and retire forms; 66 create invitation links with or without an e-mail address; assign a form to 67 a user, or replace a user's form with another one; review, accept, reject 68 or return submissions; and export them. 69 * Answers are stored in PostgreSQL as JSON; media are stored as large objects 70 keyed by their SHA-256 hash. Administrators can list media sorted by date or 71 size, view them and delete them in bulk. 72 * Upload types: the usual HTML fields; HD selfie videos with liveness 73 detection; videos of directed movements; identity documents with MRZ; 74 phone numbers verified by SMS; postal addresses verified by letter. The 75 user is told when evidence has expired or failed and must be submitted 76 again. 77 * The personal data must be protected in line with KYC regulations: encrypted 78 at rest, with a retention period per form and with administrator access 79 recorded in an audit log. 80 81 Proposed Solution 82 ================= 83 84 The detailed specifications live in the Beekeeper repository (``docs/``). 85 This DD summarises the design and the reasons behind it. 86 87 Components 88 ---------- 89 90 Beekeeper is written in Rust, with the crates used by ``taler-rust`` (axum, 91 hyper, sqlx, tokio, clap, tracing). It consists of these parts: 92 93 * ``beekeeper-httpd`` serves the REST API, including resumable uploads, and 94 the static web front ends. It is the only process that holds the master 95 key. 96 * ``beekeeper-analyzer`` workers analyse media. They take jobs from a queue 97 in PostgreSQL (``FOR UPDATE SKIP LOCKED``, woken by ``LISTEN/NOTIFY``). 98 Because they decode untrusted video with FFmpeg, they run sandboxed: 99 landlock and seccomp, no network access apart from the database socket, and 100 a database role that can read only its own job. Analyzers can run on 101 separate GPU machines. 102 * ``beekeeper-admin`` is a command-line tool for database setup, the first 103 administrator account, key rotation, garbage collection and retention. 104 * Messages are delivered by helper programs for e-mail, SMS and letters. 105 They use the same calling convention as challenger's ``challenger-send-*`` 106 scripts, so the existing provider integrations can be reused. 107 108 The SPA and the admin UI live in the Beekeeper repository itself (one 109 package, two bundles), using established libraries for resumable uploads 110 (tus-js-client), the proof of work (ALTCHA) and live camera guidance 111 (MediaPipe tasks-vision). The native apps are written in Swift and Kotlin, 112 mainly because reading the ePassport chip over NFC needs NFCPassportReader on 113 iOS and JMRTD on Android. They live in the same repository (``apps/``), so 114 that an API or expression change lands in one commit across server, SPA and 115 apps. Each has a platform-independent core (evaluator, form model, API and 116 tus client) that is tested on Linux; the Android app uses Jetpack Compose and 117 CameraX and no Google Play services, so that it can be built for F-Droid. 118 119 The server builds on ``taler-common`` from ``taler-rust`` (configuration, 120 database migrations, logging) and on a vendored copy of ``taler-api``, whose 121 changes are kept as a series of patches meant for upstream. Its error codes 122 are registered in GANA, in a registry of their own 123 (``beekeeper-error-codes``, values 100000 to 110000), and returned in the 124 usual ``ErrorDetail`` format. 125 126 The analyzer uses Debian's builds of ONNX Runtime, FFmpeg and Tesseract. The 127 models are distributed in a separate ``beekeeper-models`` repository with 128 their provenance and conversion scripts. 129 130 Form language 131 ------------- 132 133 A form is a JSON document with pages, and each page has a list of typed 134 questions: ``info``, ``text``, ``number``, ``boolean``, ``choice``, 135 ``date``, ``email``, ``phone``, ``postal_address``, ``file``, ``video`` and 136 ``identity_document``. Labels can be given per language. Each question can 137 have ``required`` and ``visible_if`` conditions and a list of validators. 138 Options per form cover manual or automatic acceptance, the e-mail OTP, 139 whether users may go back, the retention period, and when evidence must be 140 resubmitted. 141 142 Conditions use JSONLogic *syntax* with a small set of operators (comparison, 143 arithmetic, string operations, a portable regular-expression subset, date 144 arithmetic and ``verified``). Their *semantics* are defined strictly rather 145 than by "whatever JavaScript does": 146 147 * Equality is strict and deep. 148 * Type errors are errors, not coercions. 149 * String lengths and comparisons are defined over Unicode scalar values. 150 * ``\d``, ``\w`` and ``\s`` mean their ASCII classes. 151 152 Only this makes four independent implementations agree. A shared set of 153 conformance test vectors, currently 215, keeps them in step. A form is 154 rejected if an expression refers to a later question, so visibility is 155 computed in one pass in document order, and the progress indicator stays 156 stable while the user fills in later pages. 157 158 Users and authentication 159 ------------------------ 160 161 Each user has a personal link. Its secret sits in the URL fragment, so it 162 never reaches server logs or ``Referer`` headers, and the client exchanges it 163 for a session. 164 165 * **One-time code**: a form may require an e-mail one-time code on top of 166 the link. 167 * **Recovery**: if the user has a verified contact address, they can request 168 a new link by e-mail. The old link stays valid until the new one is first 169 used, so a third party who requests a recovery cannot lock the user out. 170 * **Replaced forms**: when a user's form is replaced with another, answers 171 are carried over by question id and type. A user can have several open 172 assignments, in an order the administrator chooses. 173 174 Administrators have database accounts with the roles reviewer, manager, 175 superuser and service, and log in with a password and TOTP. Every 176 administrator action is written to an audit log. 177 178 Verification 179 ------------ 180 181 E-mail addresses and phone numbers are verified with codes while the user is 182 on the page. A postal address is verified by a letter with a code. The form 183 can be submitted while the letter is in transit. If the code is not entered 184 before the deadline, or a later media analysis fails, the assignment goes back 185 to the user in the ``returned`` state, with the affected pages marked. 186 187 Media, liveness and identity documents 188 -------------------------------------- 189 190 Uploads use the tus 1.0 protocol (with the creation and checksum 191 extensions), which lets large videos survive unreliable mobile connections. 192 A user can refer only to blobs that they uploaded themselves; knowing a 193 SHA-256 hash is not enough. 194 195 For a liveness video, the server issues a random sequence of actions (for 196 example "turn head left, blink, smile") shortly before recording, and the 197 video must show them in the right order and time windows. The analyzer runs 198 open models through ONNX Runtime (``ort``): 199 200 * YuNet for face detection; 201 * the MediaPipe face and hand landmark models for eye, mouth and hand actions; 202 * head pose from landmarks; 203 * RTMPose or BlazePose for arm and body movements; 204 * MiniFASNet for passive anti-spoofing; 205 * SFace to compare the face with the document photo. 206 207 The MRZ is read with Tesseract using the OCR-B training data, and its check 208 digits are verified. For NFC passports, the server checks the chip's 209 security object against the CSCA master list with RustCrypto ``cms`` and 210 ``x509-cert``. 211 212 Each check yields a score. Clear passes and clear failures are decided 213 automatically; everything in between goes to a human reviewer, who sees the 214 per-action timings and key frames. Vendor services can be added later 215 through a trait that takes a blob and a challenge and returns the same result 216 structure. InsightFace is not used because its models are licensed for 217 non-commercial use only. 218 219 Encryption at rest 220 ------------------ 221 222 Envelope encryption protects the data: 223 224 * **Keys**: a master key file wraps a random data key per user and per blob. 225 * **Answers and addresses** are encrypted with XChaCha20-Poly1305. The 226 row's identifiers are bound in as associated data, so a ciphertext cannot 227 be moved to another row. 228 * **Blobs** are encrypted in 64 KiB chunks (STREAM), so ranged reads and 229 resumed uploads work. 230 * **Deletion** removes the data key first ("crypto-shredding"), which makes 231 the data unreadable in backups too. 232 * **Left unencrypted** are identifiers, states, timestamps, sizes and hashes, 233 so that the admin UI can list and sort. Contact addresses are found through 234 an HMAC of the normalised address. 235 * **The analyzer** never sees the master key. Each job carries the blob key 236 wrapped for that analyzer's X25519 key. 237 238 Output 239 ------ 240 241 Administrators get submissions through the admin UI, as a JSON or ZIP export, 242 and through a change feed with long polling. There are deliberately no 243 webhooks: consumers pull, so there is no outbound delivery to retry and no 244 receiver to authenticate. 245 246 Taler integration 247 ----------------- 248 249 Beekeeper implements the protocol of the exchange's ``oauth2`` KYC logic in 250 the same shape as challenger: ``POST /oauth2/setup/$CLIENT_ID`` (with the 251 client secret; the measure's ``initial_address`` pre-fills answers of the 252 same name), ``GET /oauth2/authorize/$NONCE``, ``POST /oauth2/token`` and 253 ``GET /oauth2/info``. A client (registered with ``beekeeper-admin 254 oauth-client add``) names the forms each process assigns; the process is one 255 Beekeeper user, so administrators can assign further forms or return answers 256 as usual. 257 258 As the exchange takes the answer of ``/oauth2/info`` as final, the user is 259 handed back only once every assignment of the process is accepted, 260 automatically or by a reviewer; until then the web app shows that the answers 261 are being checked, and opening the exchange's KYC link again resumes the 262 process. A rejected or expired assignment hands the user back with 263 ``error=access_denied``. 264 265 ``/oauth2/info`` returns the visible answers, the verified questions and the 266 analysis outcomes. Files are inlined as the exchange's ``{FILENAME, 267 MIME_TYPE, CONTENTS}`` objects for the question types the client chose and up 268 to its size limit, and given as metadata (SHA-256, type, size) otherwise. 269 The converter ``taler-exchange-kyc-beekeeper-converter`` maps question ids to 270 attribute names by upper-casing them (``full_name`` gives ``FULL_NAME``). 271 272 Test Plan 273 ========= 274 275 * **Conformance**: the expression vectors run in the Rust test suite and, as 276 they are written, in the TypeScript, Swift and Kotlin test suites. 277 * **Static checks**: unit tests, and every example form in the repository 278 must pass them. 279 * **Server**: integration tests against a real PostgreSQL database for page 280 saves, resumption, submission, carry-over, returns, recovery and OTP. 281 Also: token revocation, and that users cannot access other users' blobs. 282 * **Uploads**: tus conformance tests, including interrupted and resumed 283 uploads and checksum mismatches. 284 * **Verification**: the delivery helpers are replaced by scripts that record 285 the message, so tests can read the code. 286 * **Analyzer**: a corpus of recorded videos (genuine; photo, replay and mask 287 attacks; wrong action order) with expected outcomes. Thresholds are tuned 288 on it, and regressions are measured against it. 289 * **Sandbox**: tests that the analyzer cannot open network sockets or files 290 outside its model directory. 291 292 Definition of Done 293 ================== 294 295 * Forms can be uploaded, checked, published and retired; users can be invited 296 and can fill in, resume, go back and submit on the SPA and on both native 297 apps. 298 * E-mail, SMS and letter verification work through challenger-compatible 299 helper programs. 300 * Liveness, directed movement, MRZ and face-match analysis run in the 301 sandboxed analyzer, with a human review queue. 302 * All personal data is encrypted at rest; retention and crypto-shredding are 303 implemented; administrator actions are audited. 304 * Export and the change feed are available and documented in the API 305 reference. 306 * The exchange can use Beekeeper as an OAuth2 KYC provider; the round trip 307 through the exchange's ``oauth2`` plugin (``taler-exchange-kyc-tester``) 308 yields the expected attributes. 309 310 Alternatives 311 ============ 312 313 * **A commercial KYC vendor for liveness** would give certified 314 presentation-attack detection (ISO/IEC 30107-3), but would send the most 315 sensitive data to a third party and bind deployments to its pricing. It 316 remains possible through the analysis trait. 317 * **Extending challenger** would reuse its address verification, but 318 challenger verifies one address per process, and multi-page forms, media and 319 review do not fit its model. Beekeeper reuses its helper programs instead. 320 * **An existing form standard** (JSON Schema with a UI schema, SurveyJS, 321 FHIR Questionnaire) would bring existing tooling. None of them covers the 322 media and verification types, and their condition languages lack 323 semantics that can be reproduced exactly in four languages. 324 * **Plain JSONLogic** has libraries for all four platforms, but these 325 libraries disagree on coercion, equality and string handling. Conditions 326 that behave differently on the server and in the app would be a source of 327 bugs and of rejected submissions. 328 * **A TypeScript or Kotlin backend** would share code with a client. Rust 329 fits the existing ``taler-rust`` code base and its deployment, and is 330 memory-safe near FFmpeg and the ONNX models. 331 * **Webhooks** would push results to consumers, but they need retries, 332 signing and a receiver. Long polling gives similar latency without these. 333 334 Drawbacks 335 ========= 336 337 * **Own pipeline code**: without MediaPipe's runtime and OpenCV, we have to 338 write the model pre- and post-processing ourselves. This is most of the 339 analyzer work. 340 * **Four evaluators**: the condition language has to be implemented four 341 times. The conformance suite reduces this risk but does not remove it. 342 * **Consequences of encryption**: the database cannot search inside answers, 343 and losing the master key loses all data. 344 345 Discussion / Q&A 346 ================ 347 348 * Open: the expected scale (number of users and media volume), which decides 349 whether large objects in the database remain adequate for media storage.