commit d2010a8fe25921c6e467881ac3e3ca3683b141fa
parent 31771451cedb9784cce621d0af5028897546bebf
Author: Florian Dold <dold@taler.net>
Date: Sat, 5 Sep 2026 13:42:56 +0200
DD71: specify refresh scheduling and exchange key guidance
Define adaptive emergency thresholds, value-first output selection, and
persistent randomized refresh scheduling. Count all value loss when
checking for free refreshes and report expiration risk independently
of the selected rule.
Explain the 720-day spending and 30-day withdrawal guidance and derive
a conditional annual refresh-cost bound with worked acceptance cases.
Issue: https://bugs.taler.net/n/6563
Issue: https://bugs.taler.net/n/7847
Diffstat:
1 file changed, 282 insertions(+), 106 deletions(-)
diff --git a/design-documents/071-auto-refresh.rst b/design-documents/071-auto-refresh.rst
@@ -1,148 +1,324 @@
DD 71: Auto-refresh
###################
-:Design status: Accepted
+:Design status: Proposed
:Implementation status: Prototype
:DD shepherd: TBD
:Historical contributors: Christian Grothoff
:First published: 2025-10-23
-:Last substantive change: 2025-10-23
-:Implementation evidence: ``taler-typescript-core`` (2026-08-19, not merged into the reviewed HEAD)
+:Last substantive change: 2026-09-05
+:Implementation evidence: ``taler-typescript-core`` commit ``4889bee049023c59a125973532b847ec60ab6f1b`` (2026-08-19, feature-branch prototype of the earlier policy)
Summary
=======
This document describes when the wallet should automatically refresh
-non-dirty coins.
+non-dirty coins, how exchanges should configure denomination keys for this
+policy, and how the wallet can bound the cost of keeping a balance.
+
+It addresses `issue 6563 <https://bugs.gnunet.org/view.php?id=6563>`_,
+including predictable refresh costs, and its duplicate
+`issue 7847 <https://bugs.gnunet.org/view.php?id=7847>`_, about synchronized
+refresh traffic.
+
+.. note::
+
+ The prototype implements the earlier policy, not the revisions proposed here.
Motivation
==========
-The wallet must refresh non-dirty coins before they expire, least the
-user looses the money. However, this should not be done too early to
-avoid refresh fees and/or excessive load on the exchange. On the other
-hand, we need to be careful to not hold off for too long and risk
-the wallet not going online before the expiration time.
-
-We note that there obviously is no perfect solution, as at least in
-principle the user could always not restart the wallet until the
-expiration time.
+The wallet must refresh non-dirty coins before they expire, lest the user
+lose money. Refreshing too early causes avoidable fees and exchange load;
+waiting too long risks the wallet not going online before expiration.
+No policy can protect a wallet that remains offline beyond its coins'
+validity periods.
Requirements
============
-* Do not loose funds under most conditions
-* Do not cause clearly avoidable refresh operations
-* Try to educate the user if they are about to get into trouble
-* Independently of any specific approach, the wallet
- MUST spend those coins first that are earliest to
- their expiration time within their equivalence class
- as that is always the best way to avoid expiration.
-* While lifetimes of denominations are often identical,
- that may not always be the case. Theoretically, an
- exchange could significantly increase or decrease the
- deposit period at any time. The solution should
- take this into consideration.
+* Preserve funds under ordinary operating conditions, without clearly
+ avoidable refresh operations.
+* Spread automatic refresh traffic across wallets holding the same
+ denomination, including when an exchange publishes replacement keys.
+* Explain expiration risk and the conditions under which refresh-cost
+ estimates apply.
+* Within an equivalence class of otherwise interchangeable coins, the wallet
+ MUST spend those with the earliest deposit expiration first.
+* Account for different denomination lifetimes and changes in the exchange's
+ offerings. Operator recommendations are not assumptions that the wallet may
+ make about an arbitrary exchange.
Proposed Solution
=================
-1. For each denomination, consider if a refresh would
- lengthen the expiration date by more than a factor
- of four (4x), that is if the deposit expiration time
- of the denomination(s) we could currently withdraw
- is more than 4x as long as what remains for the
- denomination. If so, refresh
- all coins of that denomination.
-
-
- .. note::
-
- This basically suggests that a refresh
- would have a significant positive **impact**.
-
-2. For each denomination, consider if the remaining
- deposit period is less than **6** months, if the
- refresh fee would be zero, and if after refreshing
- the deposit expiration time would exceed **12** months,
- and if we are not on battery power. If so, refresh
- all coins of that denomination.
-
- .. note::
-
- This is again a significant impact, and it is basically
- gratis for the user.
-
-3. For each denomination, consider if the remaining
- deposit period is less than **3** months, and if
- after refreshing the deposit expiration time would
- be larger. If so, refresh all coins of that denomination.
- If afterwards the expiration time exceeds **12** months,
- show the user a warning:
-
- "This wallet was offline for too long. Make sure to
- start it at least every **3 months** to avoid the
- risk of loosing funds to expiration."
-
- .. note::
-
- This is basically a last-minute effort (unless we have
- an exchange with extremely short expiration periods).
- We do not like getting into this situation, so it is
- time to educate the user.
-
-4. Explicitly show a warning in the balances list of
- the respective currency if the remaining deposit
- period for any coin drops below 90 days.
- Distinguish in the warning key causes:
-
- 1. "We are offline and cannot expand the validity period."
- 2. "The payment service provider does not offer longer
- validity periods."
-
- .. note::
-
- This should prevent us from getting into trouble if
- e-cash is lost anyway.
+Lifetimes and output selection
+------------------------------
+
+A day means 24 hours. The thresholds below use fixed durations of 90, 180,
+and 360 days, not calendar months. For an input denomination define::
+
+ D = stamp_expire_deposit
+ W = stamp_expire_withdraw
+ G = D - W
+ R = D - now
+ E = min(90 days, G / 4)
+
+``G`` is the deposit period after withdrawals close; ``E`` is the adaptive
+emergency threshold. This policy requires finite timestamps with ``G > 0``.
+An invalid lifetime must be reported rather than used to schedule refreshes.
+Already-expired coins follow existing expiry/recovery handling; they must not
+be treated as ordinary auto-refresh candidates.
+
+For each eligible input coin, consider feasible, nonempty output selections
+from currently withdrawable, verified, non-revoked, non-lost denomination
+keys at its exchange. Respect currency, cipher, age restrictions, and refresh
+protocol limits. The input must be fresh and available, rather than reserved
+for a payment or already participating in a refresh.
+
+Selections must fit the input value after melting and output withdrawal fees,
+as specified by ``/melt`` in the :doc:`exchange API <../core/api-exchange>`::
+
+ recovered_value = sum of output coin face values
+ total_cost = input_value - recovered_value
+ output_expiration = earliest deposit expiration among selected outputs
+ output_remaining = output_expiration - now
+
+``total_cost`` includes melting fees, output withdrawal fees, and value that
+cannot be represented by the selected denominations. A refresh is free only
+if ``total_cost`` is zero, not merely when ``fee_refresh`` is zero.
+
+Select outputs in this order, before evaluating the refresh rules:
+
+1. Maximize recovered value.
+2. Among equally valuable selections, maximize ``output_expiration``.
+3. Among remaining ties, minimize the number of output coins.
+4. Break remaining ties by the lexicographic order of sorted denomination
+ hash/count pairs.
+
+If the chosen selection cannot extend validity or recover positive value,
+do not melt; wait and report expiration risk when applicable.
+
+The decision and execution MUST use the same selected outputs. Revalidate
+input availability, keys, and rule eligibility before melting. If the plan
+has become invalid, recompute it rather than silently executing a different
+selection. Once a melt has succeeded, finish that refresh using its committed
+outputs through the existing refresh lifecycle.
+
+Refresh rules
+-------------
+
+Evaluate the following rules in priority order using the chosen output
+selection. Any matching rule makes the input eligible for scheduled refresh:
+
+1. ``output_remaining > 4 * R``: the refresh provides a substantial extension.
+2. ``R < 180 days``, ``output_remaining > 360 days``, ``total_cost == 0``, and
+ confirmed external power: opportunistically refresh without losing value.
+ Unknown power state does not qualify for this rule.
+3. ``R < E`` and ``output_expiration > D``: attempt an emergency extension.
+
+All inequalities are strict. Equality alone does not satisfy a rule; schedule
+its next evaluation for the first representable time after the boundary.
+Battery state gates only rule 2, not necessary refreshes under rules 1 or 3.
+Apply the decision to all available coins of the input denomination with the
+same value and compatibility constraints.
+
+A currently withdrawable output has at least its own ``G`` remaining, well
+above its emergency threshold of at most ``G / 4``. This prevents repeated
+emergency refreshes merely because slightly newer short-lived keys appear.
+
+Scheduling and retries
+----------------------
+
+On first observing eligibility, choose an independent uniform random delay
+per wallet and input denomination, using private wallet randomness::
+
+ urgent_boundary = D - E / 10
+ delay_cap = min(1 day, E / 10, (urgent_boundary - now) / 2)
+
+Before the urgent boundary, draw a delay from zero to ``delay_cap`` and
+persist the execution deadline. At or beyond that boundary, attempt refresh
+immediately. This spreads traffic across wallets, including when new keys
+are published, while retaining time for retries.
+
+Restarts, checks, and key updates MUST NOT redraw or postpone the deadline.
+A missed deadline is due immediately if still eligible. Otherwise retain it
+until a valid plan is available, without repeatedly executing an ineligible
+task. Move it earlier if needed for urgency; clear it when no applicable fresh
+coins remain. Coins of the same denomination may share a wallet's deadline.
+
+Reevaluate on startup, relevant key or coin changes, external-power changes,
+and the next known rule boundary. Revalidate before execution and persist
+in-progress state to prevent duplicate refresh groups. Use randomized retry
+backoff capped by the time to the urgent boundary; once urgent, cap waits by
+remaining deposit lifetime without an additional DD71 delay. Avoid busy loops.
+Outages can still prevent successful refresh.
+
+Warnings
+--------
+
+Evaluate warnings independently of the selected rule. For ``0 < R < E``, show
+expiration risk in the currency balance with the exchange, affected amount,
+and earliest expiration. Pending or failing refreshes retain that indication
+until successful completion. Distinguish connectivity failures, exchange
+errors, unavailable suitable replacements according to fresh key information,
+and a pending refresh; do not infer device-wide offline status.
+
+After successfully refreshing coins whose pre-melt ``R`` was below ``E``,
+provide one recovery notice per refresh group, even if rule 1 matched. State
+that coins close to expiration were renewed; do not assert that the wallet
+was offline. Show the next relevant date, since safe reopening intervals
+depend on denomination lifetimes. Already-expired funds use the existing
+expiry indication.
+
+Exchange Configuration Recommendations
+======================================
+
+For a general-purpose exchange, start with this guidance for every relevant
+denomination family::
+
+ DURATION_SPEND = 720 days
+ DURATION_WITHDRAW = 30 days
+
+These are recommendations, not protocol requirements or universal optima.
+In the exchange implementation, ``DURATION_SPEND`` runs from withdrawal
+expiration to deposit expiration: it is ``G``, not the interval from key
+validity start to deposit expiration. With this configuration, a coin acquired
+during its withdrawal window has between 720 and 750 days left to deposit.
+
+Rationale
+---------
+
+* Replacement validity of 720--750 days makes rule 1 eligible with roughly
+ 180--187.5 days left, leaving substantial margin before the 90-day emergency
+ threshold. Under stable offerings, paid refreshes are separated by at least
+ ``720 - 750 / 4 = 532.5 days`` along each sequence of replacement coins.
+* The 30-day issuance window limits replacement-lifetime variation to about
+ four percent and limits how long a key is used for issuing coins. It does
+ not eliminate compromise risk during the longer spending period.
+* Longer spending periods require more coin-state storage; shorter issuance
+ windows require more keys and key-management work. These values balance
+ those costs with occasionally opened wallets, rather than representing a
+ measured optimum. The 720 days are two fixed 360-day periods, not calendar
+ years.
+
+Keep spending lifetimes comparable across denominations, including affordable
+change, and fees low enough to permit positive refresh outputs. Maintain
+continuous replacement-key coverage for all required values, ciphers, and age
+restrictions. Shorter spending periods require more frequent refreshes and
+shorter offline intervals; the adaptive rule remains a fallback.
+
+Follow the :ref:`exchange's denomination-key guidance <Coins-denomination-keys>`:
+withdrawal overlap must be below ``DURATION_WITHDRAW``; ``LOOKAHEAD_SIGN`` should
+be at least one year, with offline signing at least every six months. Keys
+must be signed and published, and are usable only once their validity starts.
+For lifetime or fee changes, introduce new configuration sections as the
+manual requires, preserving suitable replacements for existing coins.
+
+Conditional Annual Refresh-Cost Bound
+=====================================
+
+The fee summary should distinguish a conservative upper bound from an expected
+charge. The following bound covers keeping the current balance for 365 days
+without payments or additional withdrawals. It excludes initial withdrawal
+fees, payment-related refreshes, and losses caused by expiration or an outage.
+Those risks must not be presented as covered by the estimate.
+
+Assumptions
+-----------
+
+For this metric, a reasonable denomination/fee structure satisfies:
+
+* Continuous compatible offerings with stable values, fees, and lifetime
+ bounds. Unpublished future offerings are projection assumptions, not signed
+ commitments.
+* Every reachable denomination, including small change, admits positive
+ value-first outputs. Refresh completes while output keys remain withdrawable;
+ a delayed reveal outside that window invalidates the estimate.
+* ``Gmin > 0`` is a lower bound on remaining lifetime at output acquisition;
+ ``Hmax`` bounds all obtainable output lifetimes from above. Full key lifetime
+ from validity start to deposit expiration suffices for ``Hmax``.
+* ``Delta = Gmin - Hmax / 4 > 0``.
+* ``0 <= p < 1`` bounds total loss divided by input value for every paid refresh
+ of every reachable denomination and projected selection, including output
+ fees and rounding.
+
+If these assumptions cannot be established, report the bound as unavailable
+and explain why. Recompute or withdraw it when assumptions fail; do not ignore
+unrefreshable dust or assume unknown future fees are zero.
+
+Calculation and justification
+-----------------------------
+
+For current balance ``B`` and horizon ``T = 365 days``, compute::
+
+ Delta = Gmin - Hmax / 4
+ N = 1 + floor(T / Delta)
+ annual_loss_bound = B * (1 - (1 - p) ** N)
+
+Round the monetary bound upward to the currency's supported amount precision.
+If computing separate bounds per exchange or compatibility group, sum their
+rounded bounds. The fee summary must show the horizon and assumptions, and
+label the result a conditional upper bound, not a predicted debit or a
+promise about future exchange offerings.
+
+A new coin starts with at least ``Gmin`` remaining. Neither paid rule can
+match above ``Hmax / 4``: rule 1 requires a fourfold extension, and rule 3 has
+``E <= G / 4 <= Hmax / 4``. Paid refreshes are therefore separated by at least
+``Delta`` along each sequence of replacement coins. Free refreshes cannot
+shorten that interval; scheduling delays only lengthen it. The initial ``1``
+allows an immediately due refresh of existing holdings.
+
+Each paid refresh preserves at least ``1 - p`` of its input, including when
+it splits into multiple outputs. Since every resulting sequence has at most
+``N`` paid refreshes, at least ``B * (1 - p) ** N`` remains. If ``p == 0``, the
+bound is zero.
+
+With the recommended configuration, ``Gmin = 720 days``, ``Hmax = 750 days``,
+and ``Delta = 532.5 days``, giving ``N = 1``. If the total loss per paid refresh
+is bounded by one percent across all reachable denominations, a balance of
+100 currency units therefore has a conditional annual bound of 1 unit. The
+one-percent premise must be checked; the key durations alone do not imply it.
Definition of Done
==================
-* [x] Prototype implemented in a wallet-core feature branch
-* [ ] Prototype merged into the main branch
-* [ ] Changes to interactions for signalling warnings to GUIs
-* [ ] Dev experiments exist to trigger special alerts to users
-* [ ] GUIs have been designed and tested
-
+* [x] Earlier policy prototyped in a wallet-core feature branch
+* [ ] Revised policy reviewed, implemented, and merged
+* [ ] Tests cover adaptive and strict thresholds, mixed/empty outputs, fees and
+ rounding, power state, persistent scheduling, retries, and warning overlap
+* [ ] Fee-bound tests cover zero cost, split outputs, initially due refreshes,
+ and unavailable estimates
+* [ ] Exchange guidance checked against supported key rotation
+* [ ] GUI warnings and fee summaries designed and tested with dev experiments
+* [ ] Implemented behavior documented in the wallet reference
Alternatives
============
-The wallet currently implements simple rules for auto-refresh:
+The older wallet policy checked an exchange after 75% of the period from
+withdrawal expiration to deposit expiration had passed, refreshing coins more
+than 50% into that period. Grouping these operations reduced separate checks,
+but did not account for replacement lifetimes or absolute offline margins.
-1. After 75% of a denomination's "deposit lifespan" has passed,
- we do "auto-refresh check" for all coins of the exchange
-
-2. During this auto-refresh check, all coins that are >50% into
- their deposit lifespan are auto-refreshed.
-
-This is risky as it does not consider absolute lifespans or user
-behavior.
+The earlier DD71 policy accepted any extension below a fixed 90-day threshold.
+That could repeatedly refresh short-lived coins whenever slightly newer keys
+appeared. The adaptive threshold retains an emergency path without treating
+all short-lived coins as perpetual emergencies.
Drawbacks
=========
-* This approach does not (yet) consider user behavior. We could
- theoretically learn from that.
-
-
-
-Discussion / Q&A
-================
-
-(This should be filled in with results from discussions on mailing lists / personal communication.)
+* This policy does not learn individual wallet usage patterns.
+* Value-first selection can defer a lifetime extension that would cost more.
+* The annual bound can be conservative or unavailable, especially with changing
+ offerings or unrefreshable small denominations.
+* Independent scheduling reduces synchronized traffic; overdue startup checks,
+ widespread outages, and urgent retries can still create bursts.
+* Longer spending lifetimes increase exchange storage obligations. Operator
+ recommendations must balance those costs against wallet availability.
