commit ac4d9ab35788e12d6b7bd517c41c149a89d45298
parent 399483707af4911397c26c0622b38ff22e617aa5
Author: Antoine A <>
Date: Fri, 6 Nov 2026 16:27:40 +0100
DD104 new proposal
Diffstat:
1 file changed, 197 insertions(+), 170 deletions(-)
diff --git a/design-documents/104-wire-gateway-management-API.rst b/design-documents/104-wire-gateway-management-API.rst
@@ -6,233 +6,261 @@ DD 104: Wire Gateway Management API
:DD shepherd: Antoine d'Aligny
:Historical contributors: Antoine d'Aligny
:First published: 2026-09-18
-:Last substantive change: 2026-09-18
+:Last substantive change: 2026-11-06
Summary
=======
-This design document presents the challenges we currently face when handling
-transaction failures in wire gateway adapters and proposes a solution for
-addressing them.
+Extend the Wire Gateway API so administrators can inspect raw transactions
+and follow outgoing operations, including failures. Transfers, bounces and
+ordinary payments share one operation resource.
+
+This is a working design document. It describes the new resources and their
+relationships, detailed protocol specification will follow later.
Motivation
==========
-In the current design and implementations, both incoming and outgoing
-transactions can fail without administrators being notified or having the
-ability to take corrective action.
+Some incoming payments never reach Taler history, and some outgoing payments
+fail without enough information being available for reconciliation. A bounce
+may also be skipped or blocked before any outgoing payment exists.
+
+Listing operations by creation ID is insufficient for monitoring: older
+operations can change status after newer ones. We need current records and a
+history of operation statuses.
Requirements
============
-* The solution must allow administrators to see failures, understand their
- causes, and possibly fix them.
-* The solution must be lightweight to implement in wire gateway adapters, as
- the goal of this API is to provide a thin wrapper around how each
- implementation works. Common logic that can be extracted into another
- component should be.
+* Expose failures, their reasons and the identifiers needed for reconciliation.
+* Include incomplete transactions and decisions not to bounce.
+* Share the management model across adapters while keeping their native
+ execution mechanisms and storage models.
+* Reuse existing Wire Gateway conventions, identifiers and stored fields.
Proposed Solution
=================
-Terminology
------------
-
-* `in_tx`: final incoming transaction
-* `out_tx`: final outgoing transaction
-* `talerable_in_tx`: talerable final incoming transaction (RESERVE | KYC | MAP)
-* `talerable_out_tx`: talerable final outgoing transaction (deposit)
-* `transfer`: initiated talerable outgoing transaction
-* `initiated`: initiated outgoing transaction
-* `bounce`: bounced invalid outgoing transaction
-
-Current API State
------------------
-
-We already have endpoints for querying Talerable transactions:
-`talerable_in_tx` with `GET /history/incoming` and `talerable_out_tx` with
-`GET /history/outgoing`.
-
-We also already have endpoints for querying `transfer` with `GET /transfers`
-and `GET /transfers/$ROW_ID`.
+API placement
+-------------
+
+Add read-only endpoints under ``/management`` in the Wire Gateway API. Reuse
+its account scope, authentication, types and response conventions.
+TODO: should we add a new management API with different auth ?
+
+All lists use the shared :ref:`row-id-pagination` convention. Operation status
+history also supports the existing :ref:`long-polling` convention. Only the new
+fields and filters are described below.
+
+.. list-table:: Proposed endpoints
+ :header-rows: 1
+ :widths: 55 45
+
+ * - Endpoint
+ - Purpose
+ * - ``GET /management/operations``
+ - List operations, filter by kind or status.
+ * - ``GET /management/operations/$OPERATION_ID``
+ - Inspect an operation.
+ * - ``GET /management/operations/status-history``
+ - Follow status and message changes, filter by operation ID.
+ * - ``GET /management/incoming``
+ - List raw incoming records, filter by kind.
+ * - ``GET /management/incoming/$ROW_ID``
+ - Inspect an incoming record.
+ * - ``GET /management/outgoing``
+ - List raw outgoing records.
+ * - ``GET /management/outgoing/$ROW_ID``
+ - Inspect an outgoing record.
+
+Operation lists paginate by ``operation_id``, transaction lists and status
+history paginate by their respective ``row_id``. Status-history IDs are
+independent of operation IDs. Reuse existing local record IDs where possible.
+Links to transfers and Taler history use the existing APIs' IDs.
+
+Incoming and outgoing endpoints expose current records. Tracking later
+completion or reconciliation requires re-reading these records, their change
+feeds are deferred.
+
+Outgoing operations
+-------------------
+
+An operation represents an outgoing intent, even if the adapter implements it
+as a native chargeback. A bounce stored as an initiated payment appears once.
+Submission retries retain the same operation ID.
+
+``transfer`` originates from the Wire Gateway transfer API. ``bounce`` returns
+an unsuitable incoming payment. ``payment`` is an ordinary payment initiated
+through an adapter-specific tool, such as ``libeufin-nexus initiate-payment``.
+This management API does not create payments.
+
+A bank payment observed without a local intent belongs in ``outgoing`` and
+has no operation.
+
+``pending``, ``transient_failure``, ``permanent_failure`` and ``success``
+follow the existing transfer model. Submission or batch acceptance alone
+does not mean success: execution must be confirmed.
+``late_failure`` records a reversal or failure after success. These management
+statuses do not change the existing ``/transfers`` response types.
+
+.. ts:def:: ManagementOperation
+
+ type ManagementOperation = {
+ operation_id: SafeUint64;
+ status: "pending" | "transient_failure" | "permanent_failure"
+ | "success" | "late_failure";
+ status_msg?: string;
-What is missing is access to raw (incompled, malformed, etc) transactions, `in_tx` and `out_tx`, as well
-as initiated and bounced transactions, `initiated` and `bounce`.
+ amount?: Amount;
+ credit_account?: string;
+ subject?: string;
-Generic API Design
-------------------
+ // Existing link to the reconciled outgoing transaction, when known.
+ outgoing_transaction_id?: SafeUint64;
-Each adapter works differently. In most adapters, `transfer` and `bounce`
-produce an `initiated` operation, while other payment systems, such as
-Cyclos, provide a proper way to perform a bounce idempotently and have a
-specific bounce operation.
+ // Initiation time
+ created_at: Timestamp;
+ details: { [name: string]: string };
+ } & (
+ | { kind: "transfer"; transfer_id: SafeUint64 }
+ | { kind: "bounce"; incoming_transaction_id: SafeUint64 }
+ | { kind: "payment" }
+ );
-Each adapter also has different internal unique identifiers. For example,
-`libeufin_nexus` uses three of them while `depolymerizer-bitcoin` has a
-single transaction ID.
+``kind`` and its associated ID are top-level fields. Transfers require
+``transfer_id``, bounces require ``incoming_transaction_id``, and ordinary
+payments have neither. The two source IDs are mutually exclusive.
-We need an API that can expose enough detail for manual reconciliation while
-remaining generic enough to reuse the same logic across all adapters.
+The adapters store one reconciled outgoing transaction per operation.
+Additional native references can be exposed in ``details``. Status-change
+times belong to status-history entries, no separate current-status timestamp
+needs to be maintained.
-For `transfer`, we already have the paginated `GET /transfers` endpoint and
-the single-entry `GET /transfers/$ROW_ID` endpoint. The problem with this
-design is that it is not well suited to following the progress of transfers.
-Transfers do not progress or reach finality in the order in which they are
-created.
+Unknown payment fields may be omitted. Available
+failure explanations are exposed in ``status_msg``. An uncertain outcome,
+such as a submission timeout, must not be reported as a confirmed rejection.
+Status alone does not indicate whether submitting again would be safe.
-I propose adding a new status-history endpoint,
-`GET /transfers/status-history`. This will allow administrators to understand
-the lifecycle of a transaction and track the progress of all transfers in
-real time.
+Reasons and adapter details
+---------------------------
-We also need new endpoints for `incoming` and `outgoing` transactions,
-including paginated and single-entry endpoints, so that administrators can
-inspect all transactions, including incomplete or invalid ones. This is
-necessary to understand and debug why an incoming transaction never reaches
-the Talerable history.
+``status_msg`` exposes the adapter's existing human-readable explanation.
+On incoming records it explains processing or a decision not to bounce,
+on operations it explains execution status. Messages are unstructured and
+clients should not parse them to determine status.
-Finally, we need a way to track both `initiated` and `bounce` transactions
-using new endpoints providing paginated listings, status histories and
-single-entry views.
+``details`` holds adapter-specific information, especially native transaction,
+submission and batch identifiers needed for manual reconciliation. Native
+reason codes may be included when available, no common reason-code mapping
+is required. Generic clients use ``kind`` and operation ``status``.
-We could potentially remove all `transfer` endpoints, since transfers are a
-subset of `initiated` operatuibs. Whether to do so depends on whether the
-transfer-specific API remains useful as a higher-level abstraction.
+Incoming and outgoing records
+-----------------------------
-We could also enforce a single `initiated` operation concept, event if underneath
-bounces works differently. As long as doing this it not too messy for the
-database layer we should pursue this simplification.
+Expose stored transactions even when incomplete or unsuitable for Taler.
+Common payment fields retain their existing Wire Gateway meanings, unknown
+values are omitted. Bank documents that cannot be parsed into a transaction
+remain outside these resources.
-.. ts:def:: InTx
+.. ts:def:: ManagementInTx
- interface InTx {
- // Opaque identifier of the returned record.
+ interface ManagementInTx {
row_id: SafeUint64;
-
- // Unstructured implementation specific fields that must be shown.
- // Contains at least the unique identifiers to be used for manual reconciliation.
- details: {string: string};
-
- // Date of the transaction.
- date: Timestamp;
-
- // Amount received before credit_fee.
+ date?: Timestamp;
amount?: Amount;
-
- // Unstructured transaction subject
subject?: string;
-
- // Fee paid by the creditor.
- // If not null, creditor actually received amount - credit_fee
credit_fee?: Amount;
-
- // Full payto URI to identify the sender of funds.
debit_account?: string;
- // ID of the bounce operation if bounced
- bounce_id?: SafeUint64;
- }
+ kind: "incomplete" | "pending" | "rejected" | "reserve" | "kycauth" | "wad";
+ status_msg?: string;
-.. ts:def:: OutTx
+ // Present once a bounce operation has been initiated.
+ bounce_operation_id?: SafeUint64;
+ taler_history_id?: SafeUint64;
+ details: { [name: string]: string };
+ }
- interface OutTx {
- // Opaque identifier of the returned record.
- row_id: SafeUint64;
+``kind`` is ``incomplete`` when transaction information is missing, ``pending``
+when the transaction is complete but awaiting a mapping, or ``rejected`` when
+the payment cannot be accepted for Taler processing. Available explanations
+are exposed in ``status_msg``. ``reserve``, ``kycauth`` and ``wad`` identify
+accepted payments exposed in Taler history, with ``taler_history_id`` linking
+to the corresponding entry.
- // Unstructured implementation specific fields that must be shown.
- // Contains at least the unique identifiers to be used for manual reconciliation.
- details: {string: string};
+``bounce_operation_id`` is present if a bounce has been initiated and absent
+otherwise. Before initiation, the incoming record's ``status_msg`` explains
+why a return is pending, skipped or blocked. Once initiated, the link remains
+regardless of the outcome, execution status and reasons belong to the bounce
+operation.
- // Date of the transaction.
- date: Timestamp;
+.. ts:def:: ManagementOutTx
- // Amount received before credit_fee.
+ interface ManagementOutTx {
+ row_id: SafeUint64;
+ date?: Timestamp;
amount?: Amount;
-
- // Unstructured transaction subject
subject?: string;
-
- // Fee paid by the debtor.
- // If not null, debtor actually paid amount + debit_fee
debit_fee?: Amount;
-
- // Full payto URI to identify the receiver of funds.
credit_account?: string;
- }
-
-.. ts:def:: InitiatedTx
-
- interface InitiatedTx {
- // Opaque ID of this operation.
- operation_id: SafeUint64;
-
- // Amount to transfer.
- amount: Amount;
- // The recipient's account identifier as a full payto:// URI.
- credit_account: string;
-
- // Optional ID of the wire transfer initiation if operation is a transfer
- transfer_id?: SafeUint64;
-
- // Optional ID of the bounced incoming transaction if operation is a bounce
- bounced_id?: SafeUint64;
-
- // Current status
- // pending: in progress
- // transient_failure: has failed but may succeed later
- // permanent_failure: has failed permanently and will never reach finality
- // success: has succeeded and reached finality
- // late_failure: has failed permanently after reaching the success status
- status: "pending" | "transient_failure" | "permanent_failure" | "success" | "late_failure";
-
- // Timestamp that indicates when this status was reached.
- timestamp: Timestamp;
+ operation_id?: SafeUint64;
+ taler_history_id?: SafeUint64;
+ details: { [name: string]: string };
}
-.. ts:def:: InitiatedStatus
+Operation status history
+------------------------
- interface InitiatedStatus {
- // Opaque ID of this operation.
- operation_id: SafeUint64;
+Record operation creation and changes to ``status`` or ``status_msg`` in a
+small append-only history. Each entry stores the resulting status and message,
+clients fetch the operation when they need payment or reconciliation details.
+Changes to those details alone do not require a history entry.
- // Opaque ID of this status, used for pagination.
- row_id: SafeUint64;
-
- // pending: in progress
- // transient_failure: has failed but may succeed later
- // permanent_failure: has failed permanently and will never reach finality
- // success: has succeeded and reached finality
- // late_failure: has failed permanently after reaching the success status
- status: "pending" | "transient_failure" | "permanent_failure" | "success" | "late_failure";
+.. ts:def:: ManagementStatusChange
- // Optional unstructured messages about the status. Can be used to document the reasons for failure or the state of progress.
+ interface ManagementStatusChange {
+ row_id: SafeUint64;
+ operation_id: SafeUint64;
+ status: "pending" | "transient_failure" | "permanent_failure"
+ | "success" | "late_failure";
status_msg?: string;
-
- // Timestamp that indicates when this status was reached.
+ // When the adapter recorded this status, not the bank execution time.
timestamp: Timestamp;
}
-Management logic
-----------------
-
-On top of this API, we could build a Management SPA to support administrative operations.
-The UI would show recent failures and track whether someone has reviewed them and made a decision on how to handle them.
-We could also extend the API with administrative operations, such as manually requesting a retry.
+History starts when the feature is enabled. Existing operations remain
+available through the current list/detail endpoints without reconstructing
+past transitions.
-Test Plan
-=========
+Adapter implementation
+----------------------
+Derive operation ``kind`` by joining the existing transfer and bounce tables,
+use ``payment`` for other initiated payments. Derive incoming ``kind`` from
+existing information: a Taler history entry supplies its accepted type, a
+pending mapping record supplies ``pending``, and missing required transaction
+information supplies ``incomplete``. A complete record that the adapter has
+rejected for Taler processing is ``rejected``.
+Magnet and Nexus can reuse initiation IDs as operation IDs, their stored
+status/message and their existing outgoing link. Map Nexus ``unsubmitted`` to
+``pending`` in the API.
-Definition of Done
-==================
+Cyclos keeps its native chargeback path. Its current ``bounced`` table already
+allows a missing ``chargeback_id``. It can hold the bounce intent before the
+native call, with status/message fields added for execution tracking and a
+small ID mapping to expose both bounces and initiated payments as operations.
+Registration and recovery must update the same intent, including when the
+chargeback result is initially unknown.
-Alternatives
-============
+Reuse stored diagnostic messages and identifiers. Where a skipped or blocked
+bounce is currently only logged, persist that explanation on the incoming
+record. Older records may omit unavailable messages.
-Drawbacks
-=========
+Management logic
+----------------
-Discussion / Q&A
-================
-\ No newline at end of file
+A separate service can consume status history, notify administrators and keep
+track of reviewed failures. Adapters remain responsible for execution and
+reconciliation. Retry and other corrective commands are outside this proposal.