commit 2186f753d98149e1e10bc226b759c02ab2694c95
parent ca5a81e94c7c148f55841c38fcb69a3008310ec2
Author: Christian Grothoff <christian@grothoff.org>
Date: Mon, 17 Aug 2026 20:16:54 +0200
dd92: update to latest wallet-core API
Diffstat:
1 file changed, 580 insertions(+), 120 deletions(-)
diff --git a/design-documents/092-incremental-backup-sync.rst b/design-documents/092-incremental-backup-sync.rst
@@ -759,6 +759,25 @@ backup with matching primary keys, a state-based CRDT “merge” strategy was
carefuly devised for every top-level operation type in the block, so that
wallets can deterministically agree on a consistent global state.
+One rule cuts across all of the transaction families: **a transaction only
+ever moves towards its end.** The wallets of a group work on the same
+transactions at the same time, so an increment that would take a record
+back to a state it has already moved past is describing an older view of
+it, and only its origin block is recorded. The terminal states are ranked
+rather than simply frozen, so that two wallets which reached *different*
+ones both settle on the same one:
+
+.. code-block:: text
+
+ done > failed > aborted > expired > (not terminal)
+
+Preferring ``done`` is deterministic, which is what convergence needs, and
+it is also the truthful answer: a transaction that finished actually moved
+the money. Without the rule, a wallet that completed a withdrawal would
+pull in the abort another device had issued against the copy it restored,
+and end up showing an abandoned transaction while holding the coins it
+produced.
+
Add or update an exchange
~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -897,6 +916,20 @@ before a cycle ran restores in its spent state.
Restoring the coin also recomputes the wallet's *coin availability* rows
(the counts the balance reads) from the restored coins, so a restored
wallet shows the same balance as the wallet that made the backup. The
+counts are always derived and never carried, which is what makes the
+restore idempotent; only a coin that is spendable (status ``fresh``)
+counts, matching what the wallet's own bookkeeping does with a suspended
+one.
+
+For the two balances to agree, *every* change to whether a coin counts has
+to reach the other wallets, not only spending: a coin melted into a
+refresh, recouped from a revoked denomination, written off with its
+denomination, or suspended by the user is reported with a ``spend-coin``
+increment carrying its new status. The section is the coin's terminal
+update, whatever brought it about. A change that is not reported is the
+one way the two devices can end up disagreeing about how much money the
+user has, since a coin that is already backed up is never offered again by
+the full collection pass. The
reserves (and with them the ability to recoup a restored coin) are backed
up by the ``add-reserve`` family, and the withdrawal family
(``withdrawal-start`` / ``withdrawal-abort`` / ``withdrawal-done`` /
@@ -906,11 +939,18 @@ up by the ``add-reserve`` family, and the withdrawal family
the withdrawal transactions themselves and lets a restored wallet
continue a pending one -- the bank's operation is keyed by that URI, and
the reserve key pair and the coin seed are in the backup too; only an
-expired bank operation cannot be resumed. The refresh groups are
-still not backed up, so a restored refreshed coin cannot be
-recouped-refreshed until the refresh family lands (recoup of a withdrawn
-coin works,
-see the recoup discussion under "Add a reserve").
+expired bank operation cannot be resumed. A refreshed coin's melt is
+backed up by the refresh family below, so a restored coin can be
+recouped-refreshed as well as recouped (see the recoup discussion under
+"Add a reserve").
+
+``exchangeWithdrawValues`` carries the blinding values the exchange
+contributed to the withdraw, which a recoup has to replay. For an RSA
+coin they are the constant ``{"cipher": "RSA"}``; for a Clause-Schnorr
+coin they are the R-values, which nothing can re-derive, so they have to
+travel in the increment. The field is optional because it was added
+after the increment was first released: a coin from a wallet that
+predates it is treated as RSA.
.. ts:def:: AddCoinInc
@@ -930,6 +970,7 @@ see the recoup discussion under "Add a reserve").
visible?: number;
maxAge: number;
ageCommitmentProof?: AgeCommitmentProof;
+ exchangeWithdrawValues?: ExchangeWithdrawValue;
}
.. ts:def:: CoinSource
@@ -947,7 +988,13 @@ see the recoup discussion under "Add a reserve").
reservePub: string;
}
-.. TODO: RefreshCoinSource (backup refresh groups?)
+.. ts:def:: RefreshCoinSource
+
+ interface RefreshCoinSource {
+ type: "refresh";
+ refreshGroupId: string;
+ oldCoinPub: string;
+ }
* **Primary key:** ``[coinPub]``
* **Deletion groups:** ``[coins]``
@@ -981,6 +1028,7 @@ A signed coin is spent by the user.
visible?: number;
maxAge: number;
ageCommitmentProof?: AgeCommitmentProof;
+ exchangeWithdrawValues?: ExchangeWithdrawValue;
}
* **Primary key:** ``[coinPub]``
@@ -989,19 +1037,58 @@ A signed coin is spent by the user.
Add a token
~~~~~~~~~~~
-A token is generated by the wallet but not yet signed by the merchant.
+A token is generated by the wallet but not yet signed by the merchant (the
+wallet database calls this a *slate*).
+
+Like coins, tokens were originally designed as seed-derived: the increment
+carried ``[secretSeed, choiceIndex, outputIndex]`` and the wallet
+re-derived the key pair from it. The wallet database stores per-token key
+material instead, so the increments carry the token as it stands, and the
+token's *use* public key is the primary key of the family. The three
+increments share one body, ``TokenIncBase``:
+
+.. ts:def:: TokenIncBase
+
+ interface TokenIncBase {
+ // Purchase the token belongs to, and the position within its
+ // contract that produced it.
+ purchaseId: string;
+ transactionId?: string;
+ choiceIndex?: number;
+ outputIndex?: number;
+ repeatIndex?: number;
+
+ merchantBaseUrl: string;
+ kind: MerchantContractTokenKind;
+ slug: string;
+ name: string;
+ description: string;
+ descriptionI18n?: InternationalizedString;
+ extraData: MerchantContractTokenDetails;
+
+ tokenIssuePub: TokenIssuePublicKey;
+ tokenIssuePubHash: string;
+ tokenFamilyHash?: string;
+ validAfter: TalerProtocolTimestamp;
+ validBefore: TalerProtocolTimestamp;
+
+ // The key material the wallet holds for this token. Nothing can
+ // reconstruct it, so it travels in the increment.
+ tokenUsePub: string;
+ tokenUsePriv: string;
+ tokenUseSig?: TokenUseSig;
+ tokenEv: TokenEnvelope;
+ tokenEvHash: string;
+ blindingKey: string;
+ }
.. ts:def:: AddTokenInc
- interface AddTokenInc {
+ interface AddTokenInc extends TokenIncBase {
type: "add-token";
- secretSeed: string;
- choiceIndex: number;
- outputIndex: number;
- contractTermsHash: HashCode; // blob
}
-* **Primary key:** ``[secretSeed, choiceIndex, outputIndex]``
+* **Primary key:** ``[tokenUsePub]``
* **Deletion groups:** ``[tokens]``
Merge strategy
@@ -1012,20 +1099,18 @@ No merge is required, new tokens are unique.
Sign a token
~~~~~~~~~~~~
-A token is signed by the merchant.
+A token is signed by the merchant. Applying this increment also removes
+the slate the token was issued from, the same way the wallet's own
+issuance flow does.
.. ts:def:: SignTokenInc
- interface SignTokenInc {
+ interface SignTokenInc extends TokenIncBase {
type: "sign-token";
- secretSeed: string;
- choiceIndex: number;
- outputIndex: number;
- contractTermsHash: HashCode; // blob
tokenIssueSig: UnblindedDenominationSignature;
}
-* **Primary key:** ``[secretSeed, choiceIndex, outputIndex]``
+* **Primary key:** ``[tokenUsePub]``
* **Deletion groups:** ``[tokens]``
Merge strategy
@@ -1037,18 +1122,20 @@ the merchant, further attempts to sign it will fail.
Spend a token
~~~~~~~~~~~~~
-A signed token is spent by the user.
+A signed token is spent by the user. Only the fields the spend changes
+travel; the increment updates a token that is already there and is skipped
+when it is not.
.. ts:def:: SpendTokenInc
interface SpendTokenInc {
type: "spend-token";
- secretSeed: string;
- choiceIndex: number;
- outputIndex: number;
+ tokenUsePub: string;
+ transactionId?: string;
+ tokenUseSig?: TokenUseSig;
}
-* **Primary key:** ``[secretSeed, choiceIndex, outputIndex]``
+* **Primary key:** ``[tokenUsePub]``
* **Deletion groups:** ``[tokens]``
Merge strategy
@@ -1296,10 +1383,8 @@ entries (``currentMergeReserveRowId``) and the peer-pull-credit records
starts without one; the merge reserve remains findable by its public
key, and re-linking the pointer on restore is a follow-up.
-The increment types that do **not** depend on the reserves store -- the
-deposit, merchant-payment, peer-push-credit, peer-push-debit and
-peer-pull-debit families -- are implemented; see the "Definition of done"
-section.
+Every increment family in this document is implemented; see the
+"Definition of done" section for what remains.
Start a deposit
~~~~~~~~~~~~~~~
@@ -1891,6 +1976,241 @@ Merge strategy
Store all ``failReason`` in the database.
+Start a refresh
+~~~~~~~~~~~~~~~
+
+The wallet melts the remainder of one or more coins into fresh ones -- as
+change after a payment, or to renew a coin whose denomination is about to
+expire.
+
+The group carries the plan; how far it has got lives in the per-coin
+sessions below. A restored group is what lets a wallet that melted a coin
+and then lost the device still collect the change: the exchange holds the
+first melt commitment, and a wallet that re-melted with a fresh seed could
+not reveal against it.
+
+.. ts:def:: RefreshStartInc
+
+ interface RefreshStartInc {
+ type: "refresh-start";
+ refreshGroupId: string;
+ currency: string;
+ reason: string;
+ originatingTransactionId?: string;
+ oldCoinPubs: string[];
+ inputPerCoin: AmountString[];
+ expectedOutputPerCoin: AmountString[];
+ timestampCreated: TalerPreciseTimestamp;
+ }
+
+* **Primary key:** ``[refreshGroupId]``
+* **Deletion groups:** ``[refreshes]``
+
+Merge strategy
+++++++++++++++
+
+Last write wins: the plan of a refresh group never changes.
+
+Refresh session
+~~~~~~~~~~~~~~~
+
+The melt of one coin of a refresh group.
+
+Everything the reveal step needs -- the fresh coins' key material included
+-- is derived from ``sessionPublicSeed`` together with the old coin and the
+chosen denominations, all of which travel here, so this is the part of a
+refresh that has to be backed up.
+
+.. ts:def:: RefreshSessionInc
+
+ interface RefreshSessionInc {
+ type: "refresh-session";
+ refreshGroupId: string;
+ coinIndex: number;
+ sessionPublicSeed?: string;
+ refreshProtocolVersion?: number;
+ amountRefreshOutput: AmountString;
+ newDenoms: { denomPubHash: string; count: number }[];
+ norevealIndex?: number;
+ }
+
+* **Primary key:** ``[refreshGroupId, coinIndex]``
+* **Deletion groups:** ``[refreshes]``
+
+Merge strategy
+++++++++++++++
+
+Last write wins: the session is written once, when the coin is melted.
+
+Refresh done
+~~~~~~~~~~~~
+
+Every coin of the group has been melted and the fresh coins collected.
+
+.. ts:def:: RefreshDoneInc
+
+ interface RefreshDoneInc {
+ type: "refresh-done";
+ refreshGroupId: string;
+ timestampFinished: TalerPreciseTimestamp;
+ }
+
+* **Primary key:** ``[refreshGroupId]``
+* **Deletion groups:** ``[refreshes]``
+
+Refresh failed
+~~~~~~~~~~~~~~
+
+The refresh could not be completed.
+
+.. ts:def:: RefreshFailInc
+
+ interface RefreshFailInc {
+ type: "refresh-fail";
+ refreshGroupId: string;
+ failReason: TalerErrorDetail;
+ }
+
+* **Primary key:** ``[refreshGroupId]``
+* **Deletion groups:** ``[refreshes]``
+
+Derived operations: refunds, recoups and denomination losses
+------------------------------------------------------------
+
+The three families below differ from every other one in this document:
+the wallet does not start them, it *learns* about them. A refund is the
+merchant's answer to a refund query, a recoup is forced by an exchange
+revoking a denomination, and a denomination loss is what the wallet has to
+write off when a denomination expires or is withdrawn from circulation.
+
+Any wallet holding the coins can ask the same question and get the same
+answer, which is what decides how they are backed up: **only a finished
+one travels, and it restores as finished.** Backing up a pending one
+would hand the second device work on an operation it cannot see the whole
+of -- it would go and query a merchant about a refund that is already
+settled on the first device -- and would leave the user looking at an
+operation that is long over elsewhere but "pending" here. A pending one
+is simply not collected, and keeps no origin block, so a later pass offers
+it up once it has finished.
+
+Refund
+~~~~~~
+
+A refund the merchant granted, as it finally stood.
+
+The refund *items* (one per coin) are deliberately not carried: nothing
+outside the refund query itself reads them, the transaction is rendered
+entirely from the group, and their identity is the merchant's
+(``coin_pub``/``rtransaction_id``), so a wallet that does query gets the
+same ones back.
+
+.. ts:def:: RefundInc
+
+ interface RefundInc {
+ type: "refund";
+ refundGroupId: string;
+ // The purchase this refunds; restored as the transaction it points
+ // at, and not applied at all when that purchase is not there.
+ proposalId: string;
+ outcome: DerivedOutcome;
+ amountRaw: AmountString;
+ amountEffective: AmountString;
+ timestampCreated: TalerPreciseTimestamp;
+ }
+
+.. ts:def:: DerivedOutcome
+
+ // How one of the derived operations ended. A wire string rather than
+ // the wallet's numeric status enum, which is a database detail.
+ type DerivedOutcome = "done" | "failed" | "aborted" | "expired";
+
+* **Primary key:** ``[refundGroupId]``
+* **Deletion groups:** ``[refunds, payments]``
+
+Merge strategy
+++++++++++++++
+
+Last write wins: the increment describes one finished operation, and there
+is nothing to reconcile field by field.
+
+Recoup
+~~~~~~
+
+Coins reclaimed from an exchange that revoked their denomination.
+
+What the recoup *did* to the coins reaches the other wallets as coin
+increments; this is what makes the operation itself appear. Its per-coin
+progress is not carried -- it describes a run the other wallet did not
+make -- and a restored recoup is marked finished for every coin, so that
+the second device does not go and re-submit somebody else's recoup.
+
+.. ts:def:: RecoupInc
+
+ interface RecoupInc {
+ type: "recoup";
+ recoupGroupId: string;
+ exchangeBaseUrl: string;
+ outcome: DerivedOutcome;
+ // The coins that were recouped, in the order the group listed them.
+ coinPubs: string[];
+ timestampStarted: TalerPreciseTimestamp;
+ timestampFinished?: TalerPreciseTimestamp;
+ }
+
+* **Primary key:** ``[recoupGroupId]``
+* **Deletion groups:** ``[recoups, coins]``
+
+Merge strategy
+++++++++++++++
+
+Last write wins.
+
+Denomination loss
+~~~~~~~~~~~~~~~~~
+
+A denomination the wallet had to write off, with the coins it cost.
+
+Unlike the two above this one is not merely history: until the other
+wallets learn of it they keep the affected coins in their balance, and the
+two devices disagree about how much money the user has. The coins
+themselves carry the same news -- their status becomes ``denom-loss`` --
+and this is what makes the transaction appear.
+
+``denomLossEventId`` is **derived from the loss** rather than drawn at
+random. Both wallets notice the same expiry on their own, each updating
+the exchange and seeing the same denominations go; with random identifiers
+the user would end up with the same loss listed twice.
+
+.. code-block:: text
+
+ denom_loss_event_id = SHA512(exchange_base_url || 0 || event_type || 0 ||
+ sorted(denom_pub_hashes) each || 0)[0:32]
+
+.. ts:def:: DenomLossInc
+
+ interface DenomLossInc {
+ type: "denom-loss";
+ denomLossEventId: string;
+ currency: string;
+ exchangeBaseUrl: string;
+ denomPubHashes: string[];
+ // "denom-expired", "denom-vanished", "denom-revoked",
+ // "denom-unoffered".
+ eventType: string;
+ // "aborted" when the loss turned out to be reversible.
+ outcome: "done" | "aborted";
+ amount: AmountString;
+ timestampCreated: TalerPreciseTimestamp;
+ }
+
+* **Primary key:** ``[denomLossEventId]``
+* **Deletion groups:** ``[denom-losses, denominations]``
+
+Merge strategy
+++++++++++++++
+
+Last write wins.
+
Item deletion
-------------
@@ -1945,6 +2265,34 @@ Backup process
Collecting increments
~~~~~~~~~~~~~~~~~~~~~
+Recording runs inside the very transaction that performs the withdrawal,
+the payment or the deposit, which is what makes wallet state and backup
+state commit together -- and also means that anything the recording throws
+takes that operation down with it. It must therefore be impossible for
+the backup to fail an operation: the eager recording is an *optimisation*,
+not the guarantee. A record whose increment never made it keeps its
+``originBlocks`` unset, which is exactly what the full collection pass
+looks for, so a failure costs a delay and nothing else. Recording, waking
+the cycle and queueing a deletion all log and swallow; the critical-point
+hold fails open.
+
+The same applies to key material the wallet *derives* for an operation. A
+reserve key pair comes from the reserve seed, so a seed the wallet cannot
+decode would otherwise block every withdrawal, permanently, since the seed
+is stored. An unusable seed instead falls back to a random reserve key
+pair, which the backup carries as ``reservePriv`` the way it does for
+reserves that predate the seed, and the seed itself is left untouched --
+reserves already derived from it are named by their index, so replacing it
+would make them underivable elsewhere.
+
+Stored key material is checked before it is decoded, because the two
+Crockford base32 decoders a wallet may run on do not agree: the JavaScript
+one ignores trailing padding bits that are not zero, while the native
+(qtart) one rejects the string outright. A value decoded unchecked
+therefore works in a browser extension and throws on a phone. Re-encoding
+the decoded bytes and comparing settles it on either runtime, and is what
+the restore path uses to refuse a malformed seed rather than store one.
+
Wallet transactions record what they changed by appending increments to a
pending buffer, held in the wallet's backup configuration record. The
recording happens **within the same database transaction that performs the
@@ -2003,9 +2351,41 @@ longer reconstruct them.
A cycle is triggered after the recording transaction commits; if the wallet
stops before it runs, the increments simply stay pending until the next
-cycle. Independently, a periodic task runs a cycle at a fixed interval,
-covering increments whose trigger never fired, e.g. because the wallet was
-offline or the operation has no critical point.
+cycle. Independently, a periodic task runs a cycle every hour, covering
+increments whose trigger never fired, e.g. because the wallet was offline
+or the operation has no critical point. A cycle that could not reach the
+provider is retried after five minutes, and one that is waiting for the
+account payment to be prepared after thirty seconds -- the payment is what
+unlocks every upload, so it is worth retrying as soon as the provider's
+merchant backend recovers.
+
+Waking the cycle is not always enough. Past a critical point the wallet
+has already revealed key material to somebody else -- the exchange has
+signed the planchets, the purse exists and can be paid into -- and the
+cycle runs concurrently, so the operation would go ahead regardless.
+Those points therefore *hold*: the task returns to the scheduler and is
+retried, and only proceeds once the pending buffer has reached the
+provider. The hold is skipped when the account is unpaid, since no cycle
+can drain the buffer until the user pays and freezing every such
+transaction would be the worse failure.
+
+Each request for a cycle names how much is at stake, and the most urgent
+reason asked for since the last cycle that reached the provider is what
+decides how hard a *failing* cycle retries:
+
+* ``irrecoverable-secret`` -- key material a lost device would turn into
+ lost money. Retried after fifteen seconds: the transaction that
+ produced it is held until the buffer drains, so a longer wait is also
+ how long that transaction sits still.
+* ``transaction-milestone`` -- a state the user would notice losing, but
+ one that can be reconstructed.
+* ``account-payment`` -- the sync account's own payment moved; nothing of
+ the user's is at stake.
+
+The last two fall back to the ordinary five-minute retry. The urgency is
+not persisted: after a restart the pending increments are still there and
+the critical points ask again on their next retry, so it re-establishes
+itself rather than having to be reconstructed.
Full collection pass
~~~~~~~~~~~~~~~~~~~~
@@ -2081,22 +2461,36 @@ scheduling mechanisms described above drive them.
activate?: boolean;
}
-``addBackupProvider`` registers a sync server: it fetches the provider's
-``/config`` for its terms (protocol version, annual fee, storage limit),
-stores a provider record, and -- when ``activate`` is set -- makes it the
-active sync target and runs the first backup cycle. The cycle is what
-settles the account payment: a sync account only exists once it has been
-paid for, and the server rejects every upload (even at a zero annual fee)
-until then. A zero-fee account is paid automatically; any other account
-produces a payment transaction that the user confirms from the wallet.
-The response either confirms the provider is ready or hands back a
-``taler://pay/...`` URI for the account payment:
+The cycle never *waits* for the account payment. Downloading the
+provider's proposal and paying it are the purchase's own task, so the
+cycle only ever looks at where that purchase has got to -- confirming it
+when it is waiting for a decision, and otherwise leaving it alone -- and
+comes back when the purchase transitions, or on its retry interval. Every
+step is therefore idempotent and survives a wallet that stops in the
+middle.
+
+``addBackupProvider`` registers a sync server: it stores a provider record
+and -- when ``activate`` is set -- makes it the active sync target and
+wakes the backup cycle. The request itself does not talk to the provider
+and returns as soon as the record is written; an unreachable provider, or
+one that is not a sync server, therefore shows up as a failing (and
+retrying) cycle rather than as an error from this request.
+
+The first cycle is what learns the provider's terms (it fetches
+``/config`` and reports the result with the ``terms-fetched`` phase of the
+``backup-status`` notification) and what settles the account payment: a
+sync account only exists once it has been paid for, and the server rejects
+every upload (even at a zero annual fee) until then. A zero-fee account is
+paid automatically; any other account produces a payment transaction that
+the user confirms from the wallet, and the ``payment-required`` phase of
+the notification carries its ``taler://pay/...`` URI. Clients follow all
+of this through the notifications, not through this request's response:
.. ts:def:: AddBackupProviderResponse
- type AddBackupProviderResponse =
- | { status: "ok" }
- | { status: "payment-required"; talerUri?: string };
+ interface AddBackupProviderResponse {
+ status: "ok";
+ }
``removeBackupProvider`` takes a `RemoveBackupProviderRequest` naming the
provider by base URL and returns an empty object.
@@ -2128,12 +2522,26 @@ account on it:
backupProviderBaseUrl: string;
name: string;
terms?: BackupProviderTerms;
+
+ // Why the last cycle failed, when it did. Only for the active
+ // provider: the cycle statistics describe the wallet's last cycle,
+ // and that ran against the provider it syncs to.
lastError?: TalerErrorDetail;
lastSuccessfulBackupTimestamp?: TalerPreciseTimestamp;
lastAttemptedBackupTimestamp?: TalerPreciseTimestamp;
+
+ // Payment transactions opened for this account, most recent last.
+ paymentTransactionIds: string[];
+ // Deprecated alias of paymentTransactionIds, with the same contents,
+ // for user interfaces built against an older wallet-core.
paymentProposalIds: string[];
- backupProblem?: BackupProblem;
paymentStatus: ProviderPaymentStatus;
+
+ // What the provider reports it holds for the account, from the
+ // account status lookup. Absent until a cycle has managed to ask,
+ // and for providers older than sync protocol v4.
+ storageUsedBytes?: number;
+ blockCount?: number;
}
.. ts:def:: BackupProviderTerms
@@ -2171,37 +2579,66 @@ what the user backs up out of band, and what a restoring wallet is fed.
name: string;
url: string;
}[];
+
+ // The same data as a self-contained plain text, for writing down by
+ // hand or saving to a file. Produced here and *not* consumed by
+ // loadBackupRecovery, which reads the structured fields above.
+ paperKey?: string;
}
+The paper key is line-oriented, so that a line is the unit to copy, parse
+and transpose:
+
+.. code-block:: text
+
+ TALER-PAPERKEY:1
+ KEY: GXDG VQKT ... (the root key, grouped in fours)
+ CHECK: a1b2c3d4 (first 8 hex digits of SHA-512(root key))
+ PROVIDER: https://sync.example.com/
+ URI: taler://restore/... (the machine-readable form, LSD0006 5.7)
+
+The ``URI`` line is the canonical machine form: a device restoring from a
+scan or a file needs nothing but that line. The ``KEY`` / ``PROVIDER``
+lines are the human form, and the checksum catches a transcription error
+before it silently restores a different -- empty -- sync group.
+
``loadBackupRecovery`` feeds such a recovery document into a wallet, which
-is how a second (or replacing) device joins the sync group. With the
-default "theirs" strategy the wallet adopts the recovery's root key -- the
-key all per-provider account keys are derived from -- adds the recovery's
-providers, and drops its own sync pointers so that the next backup cycle
-re-pulls the whole linked list. With "ours" it keeps its own root key and only
-takes over the providers.
+is how a second (or replacing) device joins the sync group. The wallet
+adopts the recovery's root key -- the key every per-provider account key is
+derived from, so adopting it *is* what joining the group means -- and adds
+the recovery's providers. There is no "keep my own key" variant: a wallet
+that kept its own key would derive different account keys and so would not
+be in the group at all.
+
+Adopting another root key also detaches the wallet from the group it was
+in: the blocks it stored are encrypted under a key it no longer has, and
+the ``originBlocks`` lists that reference them are meaningless. Both are
+cleared. That deliberately leaves the wallet's own records looking "never
+backed up", which is what they are with respect to the group being joined:
+the full collection pass then offers them up, instead of the pull's
+"deleted iff absent from all origin blocks" sweep removing them for not
+appearing in the new group's linked list.
+
+The providers are registered but not activated; the client activates one
+with ``addBackupProvider`` (``activate: true``), and that is what starts
+the cycle which pulls the backup.
.. ts:def:: RecoveryLoadRequest
interface RecoveryLoadRequest {
recovery: BackupRecovery;
- strategy?: RecoveryMergeStrategy;
- }
-
-.. ts:def:: RecoveryMergeStrategy
-
- enum RecoveryMergeStrategy {
- // Keep the local wallet root key, import and take over providers.
- Ours = "ours";
- // Migrate to the wallet root key from the recovery information.
- Theirs = "theirs";
}
``runBackupCycle`` runs a backup cycle now, instead of waiting for the
periodic task. This is the dedicated "back up now" request; earlier
-implementations triggered a cycle by re-adding the active provider. The
-cycle runs synchronously and is serialized against any other cycle, and
-the response reports what it did.
+implementations triggered a cycle by re-adding the active provider.
+
+The request only *wakes* the cycle and returns an empty object
+immediately: the cycle runs asynchronously (and is serialized against any
+other cycle), reports its progress and outcome through the
+``backup-status`` notifications, and persists its statistics for
+``getBackupDiagnostics``. Clients track the cycle through those, not
+through this request's response.
.. ts:def:: RunBackupCycleRequest
@@ -2213,13 +2650,7 @@ the response reports what it did.
force?: boolean;
}
-.. ts:def:: RunBackupCycleResponse
-
- interface RunBackupCycleResponse {
- stats: BackupCycleStats;
- }
-
-The same statistics are persisted by the wallet after every cycle,
+The statistics are persisted by the wallet after every cycle,
whatever triggered it, and are reported by ``getBackupDiagnostics`` as
the "last cycle" outcome. The ``outcome`` field says how the cycle
ended: ``"ok"`` (including idle cycles with nothing to push),
@@ -2231,6 +2662,9 @@ ended: ``"ok"`` (including idle cycles with nothing to push),
timestamp: TalerPreciseTimestamp;
// How the cycle ended: "ok", "payment-required" or "error".
outcome: "ok" | "payment-required" | "error";
+ // Why it failed, when the outcome is "error"; the same detail the
+ // notification carried, kept for a client that was not listening.
+ lastError?: TalerErrorDetail;
// What the cycle pushed to the provider.
pushed: {
@@ -2310,10 +2744,10 @@ is in and, on the terminal phases, the outcome and the relevant counters.
interface BackupStatusNotification {
type: "backup-status";
providerBaseUrl: string;
- // "started", "pulling" or "pushing" for the progress phases; the
- // cycle ends in exactly one of "completed", "error" and
+ // "started", "pulling", "pushing" and "terms-fetched" are progress
+ // phases; the cycle ends in exactly one of "completed", "error" and
// "payment-required".
- phase: "started" | "pulling" | "pushing" |
+ phase: "started" | "pulling" | "pushing" | "terms-fetched" |
"completed" | "error" | "payment-required";
// Number of increments packed into the block being pushed
// (at "pushing").
@@ -2333,8 +2767,10 @@ is in and, on the terminal phases, the outcome and the relevant counters.
The wallet emits ``started`` when a cycle begins, ``pulling`` before the
linked list is fetched, ``pushing`` with the increment count before the
-packed block (and its blobs) is uploaded, and a terminal phase when the
-cycle ends:
+packed block (and its blobs) is uploaded, ``terms-fetched`` when it has
+read the provider's ``/config`` (which is where a newly added provider's
+terms come from, so a client showing them refreshes on it), and a terminal
+phase when the cycle ends:
* ``completed`` -- the cycle ran without error and without requiring
payment (``pulledBlocks`` / ``pushedBlockNonce`` carry the counters);
@@ -2342,7 +2778,16 @@ cycle ends:
may already have been prepared, and the UI should take the user to it;
* ``error`` -- the cycle failed (with ``error`` as the reason); the
wallet retries on its own schedule, so the notification is only for
- the user interface.
+ the user interface. The reason is also persisted, and reported by
+ ``getBackupInfo`` as the active provider's ``lastError``, so a client
+ that was not listening at the time still sees it.
+
+A cycle whose pull applied anything additionally emits a ``balance-change``
+notification. The apply path writes coins and transactions straight into
+the database, so none of the transaction state machines report them; the
+``backup-status`` notification says a cycle finished, not that the
+wallet's contents changed, and a client that refreshed on it alone would
+show a restoring wallet as empty until something else happened.
An earlier ``backup-error`` notification type (``BackupOperationError``)
was part of a legacy backup proof of concept and has been removed in
@@ -2355,53 +2800,68 @@ Definition of done
* [ ] Design incremental sync.
* [x] Design backup/restore schedules.
* [x] Design wallet-core API.
-* [ ] Wallet-core implementation (block and blob encoding, CRDT merge, sync
- protocol client and signatures, increment collection, the scheduled backup
- cycle with its pull/merge/apply half, the API request handlers, the account
- payment flow and item deletion (retro-redaction of the ``originBlocks`` plus
- the pull-side "deleted iff absent from all origin blocks" sweep) are done;
- of the increment types, the exchange, global-trust, bank-account, donau and
- denomination families as well as the deposit, merchant-payment,
- peer-push-credit, peer-push-debit and peer-pull-debit families are
- implemented. The contract terms of those transactions are backed up as
- blobs (uploaded ahead of the referencing blocks with their reference-count
- adjustments, and fetched and stored back into the contract-terms store on
- the pull side); a transaction whose terms are not available is shown in a
- reduced form instead of failing the transaction listing. The coin family
- (``add-coin`` / ``spend-coin``, carrying the per-coin key material the
- wallet database stores) is implemented, and restoring a coin recomputes
- the coin-availability rows so that the balance matches across wallets.
- The reserve family (``set-reserve-seed`` / ``add-reserve``, with the
+* [x] Wallet-core implementation. The machinery -- block and blob
+ encoding, CRDT merge, the sync protocol client and its signatures,
+ increment collection, the scheduled backup cycle with its
+ pull/merge/apply half, the API request handlers, the account payment
+ flow, and item deletion (retro-redaction of the ``originBlocks`` plus
+ the pull-side "deleted iff absent from all origin blocks" sweep) -- is
+ done, and so is **every increment family in this document**: the
+ exchange, global-trust, bank-account, donau and denomination entities;
+ the reserve family (``set-reserve-seed`` / ``add-reserve``, with the
seed-derived key pairs and the ``reservePriv`` fallback for reserves
- that predate the seed) is designed as described above; once implemented,
- it restores the reserves' key material, which is what makes a restored
- coin recoupable (see the recoup discussion under "Add a reserve").
- The withdrawal families are implemented
- (``withdrawal-start`` / ``withdrawal-abort`` / ``withdrawal-done`` /
- ``withdrawal-fail``, referencing the reserve by
- ``[exchangeBaseUrl, reservePub]`` and carrying the ``wgInfo`` with the
- ``taler://withdraw`` URI of the bank's operation); a restored wallet
- can continue a pending withdrawal (an expired bank operation is the
- only thing that cannot be resumed).
- The refresh groups are not backed up, so a restored refreshed coin
- cannot be recouped-refreshed; the token family is
- blocked on a schema redesign: the increments in this document model
- seed-derived tokens, while the wallet database now stores per-token key
- material, so restoring from the documented increments could not produce
- spendable tokens). The ``runBackupCycle`` and ``getBackupDiagnostics``
- request handlers, the per-cycle statistics (persisted for the diagnostics
- view), the forced full-collection pass and the ``backup-status``
- notifications are implemented. The native (sqlite) backend persists the
- ``originBlocks`` of every backup-managed record; earlier native databases
- are migrated on open).
+ that predate the seed), which is what makes a restored coin recoupable;
+ the withdrawal, deposit, merchant-payment, peer-push-credit,
+ peer-push-debit, peer-pull-debit and peer-pull-credit transaction
+ families; the refresh family, whose per-coin session seed lets a
+ restored wallet finish a melt instead of losing the change; and the coin
+ and token families, which carry the per-record key material the wallet
+ database stores (the seed-derived modelling of earlier drafts is gone
+ from both).
+
+ Contract terms travel as blobs -- uploaded ahead of the blocks that
+ reference them, with their reference counts adjusted, and fetched and
+ stored back into the contract-terms store on the pull side; a
+ transaction whose terms are not available is shown in a reduced form
+ instead of failing the transaction listing. Restoring a coin recomputes
+ the coin-availability rows, so a restored wallet shows the same balance
+ as the wallet that made the backup, and a restored wallet can continue a
+ pending withdrawal (only an expired bank operation cannot be resumed).
+ ``runBackupCycle`` and ``getBackupDiagnostics``, the per-cycle
+ statistics, the forced full-collection pass and the ``backup-status``
+ notifications are all in place, on both database backends: the native
+ (sqlite) schema stores the backup providers and blocks and the
+ ``originBlocks`` of every backup-managed record, and a wallet migrating
+ from the IndexedDB backend carries all three across.
+
+ The three *derived* families -- refund, recoup and denomination loss --
+ are implemented as finished facts, and every change to whether a coin
+ counts towards the balance (spend, refresh, recoup, denomination loss,
+ suspend) is reported as a coin increment, so two wallets converge on the
+ same balance rather than only on the same coins. A transaction can no
+ longer be taken back out of a terminal state by an increment describing
+ an older view of it.
+
+ Known gaps, none of which loses money: refund *items* are not carried
+ (nothing outside the refund query reads them, and the merchant hands
+ back the same ones); the exchange entries and peer-pull-credit records
+ do not restore their ``currentMergeReserveRowId`` pointer, since it is a
+ row id local to one database; recoup transactions are backed up and
+ restored but the wallet does not yet render them as transactions; and a
+ wallet cannot join a sync group written by a *newer* wallet -- it
+ refuses the blocks rather than re-uploading a truncated view of them.
* [x] Design sync API (+ auth).
* [ ] Server-side implementation (partial: block GET/POST/PUT/DELETE, object
store GET/POST with reference counting, /config and payments done;
reconciliation mechanism still missing).
-* [ ] UI/UX for backup and sync (done in the Android wallet: "back up now"
- uses ``runBackupCycle``, with a force-full-backup control and a backup
- diagnostics card exposed in developer mode, and a progress display driven
- by the ``backup-status`` notifications).
+* [x] UI/UX for backup and sync, in the Android wallet: adding and removing
+ a provider, the account payment prompt, the recovery as a QR code and as
+ a paper key (written down or saved to a file) with its import
+ counterpart, "back up now" through ``runBackupCycle`` with a
+ force-full-backup control and a diagnostics card in developer mode, and
+ a progress display driven by the ``backup-status`` notifications. The
+ web extension shows the cycle in its wallet-activity view, but has no
+ provider management user interface yet.
Alternatives
============
