taler-docs

Documentation for GNU Taler components, APIs and protocols
Log | Files | Refs | README | LICENSE

taler-merchant-manual.rst (81748B)


      1 ..
      2   This file is part of GNU TALER.
      3 
      4   Copyright (C) 2014-2023 Taler Systems SA
      5 
      6   TALER is free software; you can redistribute it and/or modify it under the
      7   terms of the GNU Affero General Public License as published by the Free Software
      8   Foundation; either version 3.0, or (at your option) any later version.
      9 
     10   TALER is distributed in the hope that it will be useful, but WITHOUT ANY
     11   WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
     12   A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more details.
     13 
     14   You should have received a copy of the GNU Affero General Public License along with
     15   TALER; see the file COPYING.  If not, see <http://www.gnu.org/licenses/>
     16 
     17   @author Christian Grothoff
     18 
     19 .. _taler-merchant-backend-operator-manual:
     20 
     21 Merchant Backend Operator Manual
     22 ################################
     23 
     24 
     25 Introduction
     26 ============
     27 
     28 About GNU Taler
     29 ---------------
     30 
     31 .. include:: frags/about-taler.rst
     32 
     33 
     34 .. _About-this-manual:
     35 
     36 About this manual
     37 -----------------
     38 
     39 This manual targets system administrators who want to install a GNU
     40 Taler merchant *backend*.
     41 
     42 We expect some moderate familiarity with the compilation and
     43 installation of Free Software packages. An understanding of cryptography
     44 is not required.
     45 
     46 .. _Architecture-overview:
     47 
     48 Architecture overview
     49 ---------------------
     50 
     51 .. index:: crypto-currency
     52 .. index:: KUDOS
     53 
     54 Taler is a pure payment system, not a new crypto-currency. As such, it
     55 operates in a traditional banking context. In particular, this means that in
     56 order to receive funds via Taler, the merchant must have a regular bank
     57 account, and payments can be executed in ordinary currencies such as USD or
     58 EUR.  Taler can also be used as a regional currency; for such scenarios, the
     59 Taler system also includes its own stand-alone bank.
     60 
     61 .. index:: frontend
     62 .. index:: back-office
     63 .. index:: backend
     64 .. index:: DBMS
     65 .. index:: PostgreSQL
     66 
     67 The Taler software stack for a merchant consists of four main components:
     68 
     69 -  A *frontend* which interacts with the customer’s browser. The frontend
     70    enables the customer to build a shopping cart and place an order.
     71    Upon payment, it triggers the respective business logic to satisfy
     72    the order. This component is not included with Taler, but rather
     73    assumed to exist at the merchant.
     74    The `Merchant API Tutorial <https://tutorials.taler.net/dev/merchant-api/>`_ gives an
     75    introduction for how to integrate Taler with Web shop frontends.
     76 -  A *back-office* application that enables the shop operators to view
     77    customer orders, match them to financial transfers, and possibly
     78    approve refunds if an order cannot be satisfied. This component is
     79    not included with Taler, but rather assumed to exist at the
     80    merchant. The :ref:`Merchant Backend API <merchant-api>` provides
     81    the API specification that should be reviewed to integrate such a
     82    back-office with the Taler backend.
     83 -  A Taler-specific payment *backend* which makes it easy for the frontend
     84    to process financial transactions with Taler. This manual primarily
     85    describes how to install and configure this backend.
     86 -  A *DBMS* which stores the transaction history for the Taler backend.
     87    For now, the GNU Taler reference implementation only supports
     88    PostgreSQL, but the code could be easily extended to support another
     89    DBMS.  Please review the PostgreSQL documentation for details on
     90    how to configure the database.
     91 
     92 The following image illustrates the various interactions of these key
     93 components:
     94 
     95 .. image:: images/arch-api.png
     96 
     97 .. index:: RESTful
     98 
     99 Basically, the backend provides the cryptographic protocol support, stores
    100 Taler-specific financial information in a DBMS and communicates with the GNU
    101 Taler exchange over the Internet. The frontend accesses the backend via a
    102 RESTful API. As a result, the frontend never has to directly communicate with
    103 the exchange, and also does not deal with sensitive data. In particular, the
    104 merchant’s signing keys and bank account information are encapsulated within
    105 the Taler merchant backend.
    106 
    107 A typical deployment will additionally include a full-blown Web server (like
    108 Apache or Nginx). Such a Web server would be responsible for TLS termination and
    109 access control to the ``/private/`` and ``/management/`` API endpoints of the
    110 merchant backend. Please carefully review the section on :ref:`secure setup
    111 <Secure-setup>` before deploying a Taler merchant backend into production.
    112 
    113 
    114 Terminology
    115 ===========
    116 
    117 This chapter describes some of the key concepts used throughout the manual.
    118 
    119 Instances
    120 ---------
    121 
    122 .. index:: instance
    123 
    124 The backend allows a single HTTP server to support multiple independent shops
    125 with distinct business entities sharing a single backend.  An *instance* is
    126 the name or identifier that allows the single HTTP server to determine which
    127 shop a request is intended for.  Each instance has its own base URL in the
    128 REST API of the merchant backend (``/instances/$INSTANCE/``).  Each instance
    129 can use its own bank accounts and keys for signing contracts. All major
    130 accounting functionality is separate per instance.  Access to each instance is
    131 controlled via a bearer token (to be set in the HTTP "Authorization" header).
    132 All instances share the same *database*, top-level HTTP(S) address and the
    133 main Taler configuration (especially the accepted *currency* and *exchanges*).
    134 
    135   .. note::
    136 
    137     This documentation does not use the term "user" or "username" in
    138     conjunction with instances as that might create confusion between
    139     instances with paying customers using the system.  We also do not use the
    140     term "account" in conjunction with instances, as that might cause
    141     confusion with bank accounts.  That said, conceptually it is of course
    142     acceptable to consider instances to be the "users" or "accounts" of a
    143     merchant backend and the bearer token is equivalent to a passphrase.
    144 
    145 .. _instance-bank-account:
    146 
    147 Instance Bank Accounts
    148 ----------------------
    149 
    150 .. index:: Bank account
    151 
    152 To receive payments, an instance must have configured one or more bank
    153 *accounts*.  When configuring the bank account of an instance, one should
    154 ideally also provide the address and credentials of an HTTP service
    155 implementing the :ref:`Taler Bank Revenue HTTP API
    156 <taler-bank-merchant-http-api>`.  Given such a service, the GNU Taler merchant
    157 backend can automatically reconcile wire transfers from the exchange to the
    158 merchant's bank account with the orders that are being settled.
    159 
    160 This documentation exclusively uses the term *account* for the bank
    161 accounts of a merchant or shop that may be associated with an instance.
    162 
    163 .. note::
    164 
    165    When an instance has **more than one** bank account configured, there is
    166    no notion of a "default" or "primary" account: *any* of the configured
    167    accounts may receive the funds for a given transaction, and which one is
    168    used is **not predictable from the merchant UI**.  The account is chosen
    169    per transaction based on what the settling exchange supports — i.e. the
    170    wire method and account restrictions of that exchange.  For example, if an
    171    instance offers both a CH-IBAN and a DE-IBAN account, one exchange may
    172    settle to the CH account while another settles to the DE account; adding an
    173    ``x-taler-bank`` account introduces yet another possibility.  If you need
    174    funds to arrive in a specific account, configure only that account.
    175 
    176 Inventory
    177 ---------
    178 
    179 .. index:: inventory
    180 .. index:: product
    181 .. index:: lock
    182 .. index:: unit
    183 .. index:: order
    184 
    185 The Taler backend offers inventory management as an optional function.
    186 Inventory is tracked per instance and consists of *products* sold in
    187 *units*. Inventory can be finite (physical stock) or infinite (for digital
    188 products).  Products may include previews (images) to be shown to the user as
    189 well as other meta-data. Inventory management allows the frontend to *lock*
    190 products, reserving a number of units from stock for a particular (unpaid)
    191 *order*. The backend can keep track of how many units of a product remain in
    192 stock and ensure that the number of units sold does not exceed the number of
    193 units in stock.
    194 
    195 Inventory management is optional, and it is possible for the frontend to
    196 include products in orders that are not in the inventory. The frontend
    197 can also override prices of products in the inventory or set a total price
    198 for an order that is different from the price of the sum of the products
    199 in the order.
    200 
    201 
    202 Orders and Contracts
    203 --------------------
    204 
    205 .. index:: order
    206 .. index:: terms
    207 .. index:: contract
    208 .. index:: claim
    209 .. index:: pay
    210 .. index:: refund
    211 .. index:: wire deadline
    212 .. index:: lock
    213 .. index:: legal expiration
    214 
    215 In Taler, users pay merchants for *orders*. An order is first created by the
    216 merchant. To create an order, the merchant must specify the specific *terms*
    217 of the order.  Order *terms* include details such as the total amount to be
    218 paid, payment fees the merchant is willing to cover, the set of products to
    219 deliver, a delivery location and many other details.  The :ref:`merchant API
    220 specification <contract-terms>` specifies the full set of possible order
    221 terms.
    222 
    223 After an order is created, it is *claimed* by a wallet. Once an order is
    224 claimed by a specific wallet, only that wallet will be able to pay for this
    225 order, to the exclusion of other wallets even if they see the same order URL.
    226 Sharing order URLs is explicitly allowed: if a user shares an order URL
    227 with another user, that other user should be given the opportunity to
    228 purchase the same product.
    229 
    230 To prevent unauthorized wallets from claiming an order, merchants can specify
    231 that claims require authorization in the form of a *claim token*. This is
    232 useful in case the order ID is predictable (say because an existing order ID
    233 scheme with predictable order IDs from the merchant frontend is used) and at
    234 the same time malicious actors claiming orders is problematic (say because of
    235 limited stocks). The use of claim tokens is optional, but if a claim token is
    236 used, it must be provided to the wallet as part of the order URI.
    237 
    238 Additionally, when stocks are limited, you can configure Taler to set a
    239 *product lock* on items (say, while composing the shopping cart).  These
    240 locks will ensure that the limited stock is respected when making offers
    241 to consumers.
    242 
    243 A wallet may *pay* for a claimed order, at which point the order turns into a
    244 (paid) *contract*.  Orders have a configurable expiration date (the
    245 ``pay_deadline``) after which the commercial offer expires and any stock of
    246 products *locked* by the order will be automatically released, allowing the
    247 stock to be sold in other orders.  When an unpaid order expires, the customer
    248 must request a fresh order if they still want to make a purchase.
    249 
    250 Once a contract has been paid, the merchant should fulfill the contract.  It
    251 is possible for the merchant to *refund* a contract order, for example if the
    252 contract cannot be fulfilled after all. Refunds are only possible after the
    253 customer paid and before the exchange has *wired* the payment to the
    254 merchant. Once the funds have been wired, refunds are no longer allowed by the
    255 Taler exchange.  The *wire deadline* specifies the latest point in time by
    256 which an exchange must wire the funds, while the (earlier) *refund deadline*
    257 specifies the earliest point in time when an exchange may wire the funds.
    258 Thus, refunds are always possible between the time of purchase and the
    259 refund deadline, but may remain possible until the wire deadline.
    260 
    261 Contract information is kept for legal reasons in the merchant database.  The
    262 main legal reason is typically to provide tax records in case of a tax audit.
    263 After the *legal expiration* (by default: a decade), contract information is
    264 deleted when running the garbage collector using ``taler-merchant-dbinit``.
    265 
    266 
    267 .. _template:
    268 
    269 Templates
    270 ---------
    271 
    272 .. index:: Template
    273 
    274 Usually, a merchant must use an authenticated endpoint to create an order and
    275 then share the link to the order with a wallet. Templates are a mechanism that
    276 allows wallets to create their own orders directly, using a public endpoint.
    277 The template fixes some properties of the contracts created from it, while
    278 other details may be left for the customer to provide.  Templates are useful
    279 in cases where the point-of-sale of a merchant is offline (and thus cannot
    280 setup an order), or even in cases where a simple static QR code is desired to
    281 accept payments or donations.
    282 
    283 When generating a template, the "summary" text of the contract and the
    284 "amount" to be paid by the customer can be fixed or left for the customer to
    285 specify.  If the customer is expected to provide either or both of these
    286 values, the template link (or QR code) can specify a default value. For
    287 example, a cafeteria with a fixed price lunch may use a "lunch" template with
    288 both values fixed to the lunch price and the "lunch" product, a bakery might
    289 fix the summary to "baked goods" but allow the customer to enter the amount
    290 based on the total price of the items being bought, and a charity may allow
    291 donating an arbitrary amount and summary message while also suggesting default
    292 values.
    293 
    294 If an offline merchant wants to confirm that a customer did actually pay the
    295 agreed amount using an order derived from a template, they can associate an
    296 OTP device with the template.
    297 
    298 
    299 .. _otp-device:
    300 
    301 OTP Devices
    302 -----------
    303 
    304 .. index:: OTP
    305 .. index:: TOTP
    306 
    307 A One-Time-Password (OTP) generator is a device or application that generates
    308 a 4 to 8 digit code typically used for authentication. The widely used TOTP
    309 standard is described in `RFC 6238 <https://www.rfc-editor.org/rfc/rfc6238>`_.
    310 For GNU Taler merchant backends, OTP devices are used as a way to assure a
    311 merchant without network connectivity that a customer made a digital
    312 payment. The idea is described in depth in our `SUERF Policy Brief
    313 <https://www.suerf.org/suer-policy-brief/69851/practical-offline-payments-using-one-time-passcodes>`_.
    314 To use this method, a merchant must configure the OTP device's shared secret
    315 in the merchant backend, and then associate the OTP device with a
    316 :ref:`template`.  Once the customer has paid, they are given a list of OTP
    317 codes which must be shown to the merchant who can check that at least one of
    318 the codes matches their OTP device, proving that the customer made the
    319 payment.
    320 
    321 
    322 Transfers
    323 ---------
    324 
    325 .. index:: transfer
    326 .. index:: wire transfer
    327 
    328 The Taler backend can be used to verify that the exchange correctly wired all
    329 of the funds to the merchant. However, if no :ref:`Taler Bank Revenue HTTP API
    330 <taler-bank-merchant-http-api>` was provided for the respective bank account,
    331 the backend does not have access to the incoming wire transfers of the
    332 merchant's bank account. In this case, merchants should manually provide the
    333 backend with wire *transfer* data that specifies the *wire transfer subject*
    334 and the amount that was received. Given this information, the backend can
    335 detect and report any irregularities that might arise.
    336 
    337 
    338 Webhooks
    339 --------
    340 
    341 .. index:: webhook
    342 
    343 A webhook is a pre-defined HTTP request that the GNU Taler merchant backend
    344 will make upon certain events, such as an order being paid or refunded. When
    345 the configured event happens, the merchant backend will make an HTTP request
    346 to the endpoint configured in the webhook configuration, possibly sending
    347 selected data about the event to the respective Web service. Webhooks can be
    348 used to trigger additional business logic outside of the GNU Taler merchant
    349 backend.
    350 
    351 
    352 Installation
    353 ============
    354 
    355 This chapter describes how to install the GNU Taler merchant backend.
    356 
    357 .. _Generic-instructions:
    358 
    359 Installing the GNU Taler binary packages on Debian
    360 --------------------------------------------------
    361 
    362 .. include:: frags/installing-debian.rst
    363 
    364 .. include:: frags/apt-install-taler-merchant.rst
    365 
    366 
    367 Installing the GNU Taler binary packages on Ubuntu
    368 --------------------------------------------------
    369 
    370 .. include:: frags/installing-ubuntu.rst
    371 
    372 .. include:: frags/apt-install-taler-merchant.rst
    373 
    374 
    375 Installing from source
    376 ----------------------
    377 
    378 The following instructions will show how to install a GNU Taler
    379 merchant backend from source.
    380 
    381 The package sources can be found in our
    382 `download directory <http://ftpmirror.gnu.org/taler/>`__.
    383 
    384 .. include:: frags/semver.rst
    385 
    386 First, the following packages need to be installed before we can compile the
    387 backend:
    388 
    389 .. include:: frags/list-of-dependencies.rst
    390 
    391 .. include:: frags/installing-gnunet.rst
    392 
    393 .. include:: frags/install-before-check.rst
    394 
    395 There is no need to actually run a GNUnet peer to use the Taler merchant
    396 backend -- all the merchant needs from GNUnet is a number of headers and
    397 libraries!
    398 
    399 .. include:: frags/installing-taler-exchange.rst
    400 
    401 There is no need to actually run a Taler exchange to use the Taler merchant
    402 backend -- all the merchant needs from the Taler exchange is a few headers and
    403 libraries!
    404 
    405 .. note::
    406 
    407   There is an additional **optional** dependency that you could install to
    408   obtain support for tax-deductible donations. This is only useful for
    409   charities and only in countries with tax authorities that operate a Donau to
    410   register charities and accept Taler-style digitally signed donation
    411   statements.  As of right now, we are pretty sure that list is right now
    412   empty. But, if you want to experiment with Taler-style donation statements,
    413   you need to install Donau after the exchange and before the merchant.
    414 
    415 .. include:: frags/install-before-check.rst
    416 
    417 .. include:: frags/installing-taler-merchant.rst
    418 
    419 .. include:: frags/install-before-check.rst
    420 
    421 
    422 
    423 How to configure the merchant backend
    424 =====================================
    425 
    426 .. index:: taler-merchant.conf
    427 
    428 The installation already provides reasonable defaults for most of the
    429 configuration options. However, some must be provided, in particular the
    430 database that the backend should use. By default, the file
    431 ``$HOME/.config/taler-merchant.conf`` is where the Web shop administrator specifies
    432 configuration values that augment or override the defaults.
    433 Note that when using our binary packages, the systemd service files
    434 force the use of ``/etc/taler-merchant/taler-merchant.conf`` as the main configuration file.
    435 
    436 
    437 .. include:: frags/configuration-format.rst
    438 
    439 
    440 .. _Backend-options:
    441 
    442 Backend options
    443 ---------------
    444 
    445 .. index:: DBMS
    446 .. index:: PostgreSQL
    447 .. index:: UNIX domain socket
    448 .. index:: TCP
    449 .. index:: port
    450 .. index:: currency
    451 .. index:: KUDOS
    452 .. index:: exchange
    453 .. index:: instance
    454 .. index:: wire format
    455 
    456 The following table describes the options that commonly need to be
    457 modified. Here, the notation ``[$SECTION]/$OPTION`` denotes the option
    458 ``$OPTION`` under the section ``[$SECTION]`` in the configuration file.
    459 
    460 
    461 Service address
    462 ^^^^^^^^^^^^^^^
    463 
    464 The service address specifies where the taler-merchant-httpd should listen for
    465 requests. When using the Debian/Ubuntu packages, these options will already be
    466 configured correctly for the included Nginx and Apache configurations and will
    467 not need any changes.
    468 
    469 The following option sets the transport protocol used by the merchant backend:
    470 
    471 .. code-block:: ini
    472 
    473       [MERCHANT]
    474       SERVE = unix # or tcp
    475 
    476 If this option is set to
    477 
    478 -  ``tcp`` then we need to set the TCP port in ``[MERCHANT]/PORT``;
    479 
    480 -  ``unix`` then we need to set the unix domain socket path and mode
    481    in ``[MERCHANT]/UNIXPATH`` and ``[MERCHANT]/UNIXPATH_MODE``. The
    482    latter takes the usual permission mask given as a number, e.g. 660
    483    for user/group read-write access.
    484 
    485 The frontend can then connect to the backend over HTTP using the specified
    486 address. If frontend and backend run within the same operating system, the
    487 use of a UNIX domain socket is recommended to avoid accidentally exposing
    488 the backend to the network.
    489 
    490 To run the Taler backend on TCP port 9966 (the default), use:
    491 
    492 .. code-block:: ini
    493 
    494    [MERCHANT]
    495    SERVE = tcp
    496    PORT = 9966
    497 
    498 .. note::
    499 
    500    If you need to change where the taler-merchant-httpd listens for requests,
    501    you should edit ``/etc/taler-merchant/merchant-overrides.conf``.  By default, the
    502    Taler merchant package will use a UNIX domain socket at
    503    ``/run/taler-merchant/merchant-httpd/merchant-http.sock``. For the best possible
    504    security it is recommended to leave this in place and configure a reverse
    505    proxy (Nginx or Apache) as described below.
    506 
    507    When using the Debian/Ubuntu packages, the use of a UNIX domain socket
    508    is already pre-configured in the ``/etc/taler-merchant/conf.d/merchant.conf``
    509    configuration file.  Suitable reverse proxy configuration
    510    file templates (``taler-merchant``) are be installed in the
    511    respective ``sites-available`` directories of Apache and Nginx.
    512 
    513 
    514 
    515 Currency
    516 ^^^^^^^^
    517 
    518 Which currency the SPA uses by default is
    519 specified using the option:
    520 
    521 .. code-block:: ini
    522 
    523       [MERCHANT]
    524       CURRENCY = EUR # or USD, ...
    525 
    526 When testing with the Taler demonstration exchange at
    527 https://exchange.demo.taler.net/ you probably want to set this
    528 value to ``KUDOS``:
    529 
    530 .. code-block:: ini
    531 
    532        [MERCHANT]
    533        CURRENCY = KUDOS
    534 
    535 The merchant backend is already multi-currency capable, and will allow you to
    536 create orders in all currencies for which an exchange is configured, not just
    537 the default currency.  However, the Web interface does not yet offer
    538 multi-currency support and often only supports using the default currency.
    539 
    540 .. note::
    541 
    542    When using the Debian/Ubuntu packages, these options should be
    543    configured in the ``/etc/taler-merchant/taler-merchant.conf`` configuration file
    544    (alternatively, you can also edit ``/etc/taler-merchant/merchant-overrides.conf``).
    545    However, you must edit the ``taler-merchant.conf`` file manually and **must not**
    546    use ``taler-merchant-config`` to do this, as that would inline the include
    547    directives and destroy the carefully setup path structure.
    548 
    549 
    550 Database
    551 ^^^^^^^^
    552 
    553 In principle it is possible for the backend to support different DBMSs.
    554 The option
    555 
    556 .. code-block:: ini
    557 
    558       [MERCHANT]
    559       DB = postgres
    560 
    561 specifies which DBMS is to be used. However, currently only the value
    562 ``postgres`` is supported. This is also the default.
    563 
    564 In addition to selecting the DBMS software, the backend requires
    565 DBMS-specific options to access the database.
    566 
    567 .. note::
    568 
    569    The **taler-merchant-dbconfig** tool can be used to automate the database
    570    setup. When using the Debian/Ubuntu packages, the user should already have
    571    been created, so you can just run the tool without any arguments and should
    572    have a working database configuration.
    573 
    574    The tool preserves an existing database unless explicitly invoked with
    575    ``-r``. Existing databases must have the expected ownership and migration
    576    privileges. It provisions only the default local PostgreSQL cluster; for
    577    remote databases, provision the database separately and use
    578    ``taler-merchant-dbinit`` with the appropriate database identity.
    579 
    580 
    581 For the ``postgres`` backend, you need to specify:
    582 
    583 .. code-block:: ini
    584 
    585       [merchantdb-postgres]
    586       CONFIG = "postgres:///taler-merchant"
    587 
    588 This option specifies a PostgreSQL access path, typically using the format
    589 ``postgres:///$DBNAME``, where ``$DBNAME`` is the name of the PostgreSQL
    590 database you want to use (here, ``taler-merchant`` on the local machine).
    591 Suppose ``$USER`` is the name of the user who will run the backend process
    592 (usually ``taler-merchant-httpd``). Then, you need to first run:
    593 
    594 .. code-block:: console
    595 
    596    $ sudo -u postgres createuser -d $USER
    597 
    598 as the PostgreSQL database administrator (usually ``postgres``) to
    599 grant ``$USER`` the ability to create new databases. Next, you should
    600 as ``$USER`` run:
    601 
    602 .. code-block:: console
    603 
    604    $ createdb $DBNAME
    605 
    606 to create the backend’s database. Here, ``$DBNAME`` must match the
    607 database name given in the configuration file.
    608 
    609 Now you should be able to create the tables and indices. To do this, run as
    610 ``$USER`` (usually ``taler-merchant-httpd``):
    611 
    612 .. code-block:: console
    613 
    614    $ taler-merchant-dbinit
    615 
    616 You may improve your security posture if you now REVOKE the rights to CREATE,
    617 DROP or ALTER tables from ``$USER``. However, if you do so, please be aware
    618 that you may have to temporarily GRANT those rights again when you update the
    619 merchant backend.  For details on how to REVOKE or GRANT these rights, consult
    620 the PostgreSQL documentation.
    621 
    622 .. include:: frags/db-stores-sensitive-data.rst
    623 
    624 
    625 .. index: MASTER_KEY
    626 
    627 Exchange
    628 ^^^^^^^^
    629 
    630 To add an exchange to the list of trusted payment service providers, you
    631 create a section with a name that starts with “MERCHANT-EXCHANGE-”. In that
    632 section, the following options need to be configured:
    633 
    634 -  The ``EXCHANGE_BASE_URL`` option specifies the exchange’s base URL.
    635    For example, to use the Taler demonstrator, specify:
    636 
    637    .. code-block:: ini
    638 
    639       [merchant-exchange-kudos]
    640       EXCHANGE_BASE_URL = "https://exchange.demo.taler.net/"
    641 
    642 -  The KUDOS demo exchange and others are usually enabled by default
    643    in the distribution so you can easily test your backend. If you want
    644    to disable KUDOS, for example because you only want to support
    645    CHF or EUR, do not delete the "kudos.conf", but instead disable
    646    it explicitly in your main taler-merchant.conf configuration
    647    file using:
    648 
    649    .. code-block:: ini
    650 
    651       [merchant-exchange-kudos]
    652       DISABLED = YES
    653 
    654    This is also the preferred way to disable any other exchange that
    655    may be enabled by default. You can get a list of all of these
    656    exchange configuration sections using
    657    ``taler-merchant-config -S | grep merchant-exchange-``.
    658 
    659 -  The ``MASTER_KEY`` option specifies the exchange’s master public key
    660    in base32 encoding. For the Taler demonstrator, use:
    661 
    662    .. code-block:: ini
    663 
    664       [merchant-exchange-kudos]
    665       MASTER_KEY = "GNRJCH0HYKN59939JC0CJ2JDC7ZNEBSATJFF00CVS3WPG4TQEA7G"
    666 
    667   You can find out this key by running ``curl https://exchange.demo.taler.net/keys | jq .master_public_key``.
    668 
    669 -  The ``CURRENCY`` option specifies the exchange’s currency.
    670    For the Taler demonstrator, use:
    671 
    672    .. code-block:: ini
    673 
    674       [merchant-exchange-kudos]
    675       CURRENCY = "KUDOS"
    676 
    677 
    678 Note that multiple exchanges can be added to the system by using different
    679 identifiers in place of ``KUDOS`` in the example above. One exchange will only
    680 ever support a single currency; thus, if you need support for multiple
    681 currencies, you must add multiple exchanges.
    682 
    683 The merchant already ships with a default configuration that contains the
    684 ``merchant-exchange-kudos`` section from above.
    685 
    686 .. note::
    687 
    688    Manually setting up exchanges is only recommended under special
    689    circumstances. In general, GNU Taler distributions will include trustworthy
    690    exchanges (for each currency) in the default configuration, and there is
    691    rarely a good reason for trusting an exchange that has no relationship
    692    with the GNU Taler development team.
    693 
    694 
    695 .. _Sample-backend-configuration:
    696 
    697 Sample backend configuration
    698 ----------------------------
    699 
    700 .. index:: configuration
    701 
    702 The following is an example for a complete backend configuration:
    703 
    704 .. code-block:: ini
    705 
    706    [merchant]
    707    # Default currency; note that multi-currency support is still
    708    # a bit experimental...
    709    CURRENCY = KUDOS
    710    SERVE = TCP
    711    PORT = 9966
    712    DATABASE = postgres
    713 
    714    [merchantdb-postgres]
    715    CONFIG = postgres:///taler-merchant
    716 
    717    [merchant-exchange-kudos]
    718    EXCHANGE_BASE_URL = https://exchange.demo.taler.net/
    719    MASTER_KEY = FH1Y8ZMHCTPQ0YFSZECDH8C9407JR3YN0MF1706PTG24Q4NEWGV0
    720    CURRENCY = KUDOS
    721 
    722 Given the above configuration, the backend will use a PostgreSQL database
    723 named ``donations`` running on the same host.
    724 
    725 The backend will deposit the coins it receives to the exchange at
    726 https://exchange.demo.taler.net/, which has the master key
    727 ``FH1Y8ZMHCTPQ0YFSZECDH8C9407JR3YN0MF1706PTG24Q4NEWGV0``.
    728 
    729 
    730 .. _Launching-the-backend:
    731 
    732 Launching the backend
    733 ---------------------
    734 
    735 .. index:: backend
    736 .. index:: taler-merchant-httpd
    737 
    738 Assuming you have configured everything correctly, you can launch the
    739 merchant backend as ``$USER`` using (to provide a trivial example):
    740 
    741 .. code-block:: console
    742 
    743    $ taler-merchant-httpd &
    744    $ taler-merchant-webhook &
    745    $ taler-merchant-kyccheck &
    746    $ taler-merchant-wirewatch &
    747    $ taler-merchant-depositcheck &
    748    $ taler-merchant-exchangekeyupdate &
    749    $ taler-merchant-reconciliation &
    750 
    751 .. note::
    752 
    753   If you compiled the merchant backend with support for donation
    754   statements via Donau, you need to additionally launch
    755   ``taler-merchant-donaukeyupdate``.
    756 
    757 To ensure these processes run always in the background and also after
    758 rebooting, you should use systemd, cron or some other init system of your
    759 operating system to launch the process.  You should also periodically re-start
    760 these services to prevent them from exhausting the memory utilization of the
    761 PostgreSQL database.  Consult the documentation of your operating system for
    762 how to start and stop daemons.
    763 
    764 .. note::
    765 
    766    When using the Debian/Ubuntu packages, the systemd configuration
    767    will already exist. You only need to enable and start the service
    768    using ``systemctl enable taler-merchant.target`` and
    769    ``systemctl start taler-merchant.target``. Additionally, you should
    770    review the ``/etc/apache2/sites-available/taler-merchant.conf``
    771    or ``/etc/nginx/sites-available/taler-merchant`` (these files
    772    contain additional instructions to follow), symlink it to
    773    ``sites-enabled/`` and restart your HTTP server. After that, you
    774    should be able to visit the merchant backend at the respective
    775    HTTP(S) endpoint.
    776 
    777 If everything worked as expected, the command
    778 
    779 .. code-block:: console
    780 
    781    $ wget -O - http://localhost:9966/config
    782 
    783 should return some basic configuration status data about the service.
    784 
    785 Please note that your backend might then be globally reachable without
    786 any access control.  You can either:
    787 
    788   * Use the ``--auth=$TOKEN`` command-line option to **taler-merchant-httpd** to set an access token to be provided in an ``Authorization: Bearer $TOKEN`` HTTP header. Note that this can be used at anytime to override access control, but remains only in effect until a first instance is created or an existing instance authentication setting is modified.
    789   * Set the ``TALER_MERCHANT_TOKEN`` environment variable to ``$TOKEN`` for the same effect. This method has the advantage of ``$TOKEN`` not being visible as a command-line interface to other local users on the same machine.
    790   * Set up an instance with an authentication token before some unauthorized person has a chance to access the backend.  As the backend is useless without any instance and the chances of remote attackers during the initial configuration is low, this is probably sufficient for most use-cases. Still, keep the first two scenarios in mind in case you ever forget your access token!
    791 
    792 Production systems should additionally be configured to bind to a UNIX domain socket
    793 and use TLS for improved network privacy, see :ref:`Secure setup <Secure-setup>`.
    794 
    795 
    796 Multi-factor authentication
    797 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    798 
    799 The merchant backend supports multi-factor authentication for critical
    800 endpoints, including issuing access token, changing login credentials,
    801 modifying the bank account and deleting the instance. To use
    802 multi-factor authentication, the merchant backend must be configured
    803 with helper programs that can be used to send e-mails or SMS messages.
    804 
    805 
    806 .. code-block:: ini
    807 
    808       [MERCHANT]
    809       HELPER_SMS = sms_helper.sh
    810       HELPER_EMAIL = email_helper.sh
    811 
    812 These scripts will be called by the merchant backend with the
    813 first argument being the phone number or the e-mail address, and
    814 with the message with the TAN code to send to the user on standard
    815 input. The shell scripts must exit with a status code of 0 on
    816 success.  A trivial e-mail helper would look like this:
    817 
    818 .. code-block:: sh
    819 
    820       #!/bin/sh
    821       exec mail --subject="TAN code for the Taler merchant backend" "$1"
    822 
    823 Here, ``mail`` refers to the classic UNIX mail command.
    824 Example scripts can also be found in the `GNU Anastasis Git
    825 <https://git.taler.net/anastasis.git/tree/src/authorization>`_.
    826 
    827 
    828 Self-provisioning
    829 ^^^^^^^^^^^^^^^^^
    830 
    831 Self-provisioning allows anyone to create a merchant instance. This
    832 is useful if a merchant backend is offered as a public service. In this
    833 case, you should set:
    834 
    835 .. code-block:: ini
    836 
    837       [MERCHANT]
    838       ENABLE_SELF_PROVISIONING = YES
    839 
    840 
    841 To enable self-provisioned users to reset their passwords if they
    842 forgot them, the merchant backend requires two other authentication
    843 methods to be available and validated when the instance is
    844 self-provisioned.  This can be enabled using:
    845 
    846 .. code-block:: ini
    847 
    848       [MERCHANT]
    849       MANDATORY_TAN_CHANNELS = sms email
    850 
    851 With this setting, users that self-provision an instance must first
    852 demonstrate control over the e-mail address and phone number provided,
    853 and can thus later reset their password without needing support from
    854 the administrator.
    855 
    856 To also require one of these channels as a second factor when a merchant
    857 changes its password, set:
    858 
    859 .. code-block:: ini
    860 
    861       [MERCHANT]
    862       PASSWORD_CHANGE_MFA = YES
    863 
    864 The current password is still required.  When both SMS and e-mail are usable,
    865 the merchant may complete either challenge.
    866 
    867 
    868 .. index:: instance
    869 .. _Instance-setup:
    870 
    871 
    872 Instance setup
    873 ==============
    874 
    875 We recommend the use of the single-page administration application (SPA) that
    876 is served by default at the base URL of the merchant backend.  You can use it
    877 to perform all steps described in this section (and more!), using a simple Web
    878 interface. Alternatively, you can also use the ``wget`` commands given below.
    879 
    880 Regardless of which approach you use, the first step for using the backend
    881 involves the creation of the ``admin`` instance. The ``admin`` instance can
    882 also create, configure or delete other instances, similar to the ``root``
    883 account on UNIX.  When no instance exists, then the backend is reachable
    884 without any access control (unless you configured some in the reverse proxy).
    885 
    886 .. note::
    887 
    888   If you created a non-admin instance first, you cannot create an ``admin``
    889   instance via the SPA anymore. In this case, you can only create an
    890   administrative account by using the command-line. By invoking
    891   ``taler-merchant-passwd --instance=admin $PASSWORD`` you can set both the
    892   password and create an ``admin`` instance if it does not yet exist.
    893   However, for non-admin instances, you can only set the password with this tool.
    894 
    895 The following documentation shows how to handle any instance. Thus, if you
    896 want to have multiple instances, you may need to perform the steps multiple
    897 times, once for each instance.
    898 
    899 .. note::
    900 
    901   A potential security concern is that normal API usage leaks instance existence.
    902   This means unauthorized users can distinguish between the case where the
    903   instance does not exist (HTTP 404) and the case where access is denied
    904   (HTTP 403).
    905   This concern can be addressed using a properly configured
    906   :ref:`reverse proxy <reverse-proxy-configuration>`.
    907 
    908 
    909 Instance setup with the SPA
    910 ---------------------------
    911 
    912 In order to setup an instance, you need the merchant backend to already be
    913 running, and you must either have the credentials for the ``admin`` instance,
    914 or no instance must be configured at all yet.
    915 
    916 To start, point your browser to ``$PROTO://backend.$DOMAIN_NAME/``, replacing
    917 "$PROTO" with "https" or (rarely) "http" and "$DOMAIN_NAME" with your
    918 organization's DNS domain or subdomain.
    919 
    920 .. note::
    921 
    922    The label "backend" here is also just a suggestion, your administrator
    923    can in principle choose any name.
    924 
    925 You should be welcomed by the following merchant backoffice page:
    926 
    927 .. image:: screenshots/merchant_first_login.png
    928 
    929 After supplying the required fields, primarily the name of your organization
    930 and the desired access token, click ``confirm``.  You can change the instance
    931 settings later via the ``Settings`` entry in the menu on the left.
    932 
    933 
    934 Instance settings
    935 -----------------
    936 
    937 The settings dialog allows you to select an image to be used as a logo
    938 for your shop. Wallets may use that logo when showing contracts to
    939 highlight to customers which shop they are buying from.
    940 
    941 The settings dialog allows you to specify the address of your business
    942 and the jurisdiction the shop is under. Both will be embedded into the
    943 contracts and may be shown by the Taler wallet to customers that want
    944 to know these details.
    945 
    946 You must also configure whether you intend to pay transaction fees,
    947 or whether the customer is required to pay for any payment fees. If
    948 you do not cover the fees, the fees will be shown separately to the
    949 customer and added to the total of each order, which may discourage
    950 consumers from using the Taler payment method.  The specific
    951 magnitude of the fees cannot be configured here, as it depends on
    952 the amount of the order and is dynamically computed. Regardless of
    953 what you specify here, the front-end can override the acceptable
    954 fee amount for each order it creates.
    955 
    956 .. note::
    957 
    958   Details on the acceptable fee calculation
    959   are described in the Taler design document 47.
    960 
    961 Finally, you need to specify several settings relating to default
    962 deadlines.
    963 
    964   (1) The "Default payment delay" specifies when an offer expires.  The
    965   customer basically has this amount of time to pay, or the backend will
    966   refuse the payment and require the customer to get a new quote.
    967 
    968   (2) The "Default refund delay" specifies how long the customer may receive
    969   refunds. The refund period is cumulative on top of the "Default payment
    970   delay". Thus, the refund period ends independently of when the customer
    971   actually paid for the order. The exchange will **not** wire the funds to the
    972   merchant before the refund deadline lapses, as after the funds have been
    973   wired refunds using Taler are no longer possible.
    974 
    975   (3) The "Default wire transfer delay" specifies how soon the exchange
    976   **must** wire the funds **after** the refund deadline.  The delay is again
    977   cumulative on top of the "Default payment delay" and the "Default refund
    978   delay".  However, the resulting time is still not the actual wire
    979   deadline, as first the "Default wire rounding interval" is also considered.
    980 
    981   (4) The "Default wire rounding interval" specifies to what period the
    982   wire deadline should be rounded up to. The ultimate wire deadline is
    983   computed by adding the default payment, rounding and wire delays to
    984   the current time and rounding the resulting timestamp to the
    985   "Default wire rounding interval". Typical values include
    986   end-of-day, end-of-week, end-of-month, end-of-quarter or end-of-year.
    987 
    988 .. note::
    989 
    990   The wire deadline is rounded using the local timezone of the Taler merchant
    991   backend server, so if you want end-of-day payments make sure to run your
    992   merchant backend in your own timezone.
    993 
    994 Specifying larger values for the wire transfer delay and the wire rounding
    995 interval allows the exchange to aggregate more payments into larger wire
    996 transfers. The exchange is required by the protocol to initiate the wire
    997 transfer **before** the wire transfer deadline.
    998 
    999 All of the computed deadlines (payment, refund and wire transfer)
   1000 are just defaults and can be modified by frontends for any
   1001 specific order.
   1002 
   1003 
   1004 Instance setup without the Web interface
   1005 ----------------------------------------
   1006 
   1007 Instances can be created by POSTing a request to ``/management/instances``
   1008 without using the Web interface.  This could be useful if you want to create
   1009 many instances programmatically.  To create an instance without the Web
   1010 interface create a file ``instance.json`` with an
   1011 `InstanceConfigurationRequest`:
   1012 
   1013 .. code-block:: json
   1014 
   1015    {
   1016      "id" : "admin",
   1017      "name": "Example Inc.",
   1018      "address": { "country" : "zz" },
   1019      "auth": { "method" : "external"} ,
   1020      "jurisdiction": { "country" : "zz" },
   1021      "use_stefan": true,
   1022      "default_pay_delay": { "d_ms" : 1209600000 }
   1023      "default_refund_delay": { "d_ms" : 1209600000 }
   1024      "default_wire_transfer_delay": { "d_ms" : 1209600000 },
   1025    }
   1026 
   1027 The ``name`` field will be shown as the name of your shop. The ``address``
   1028 field is expected to contain your shop's physical address. The various
   1029 ``default_*`` values specify defaults for transaction fees your shop is
   1030 willing to cover, how long offers made to the customer are valid, and how long
   1031 the exchange has before it must wire the funds to your bank account. Those
   1032 defaults can be modified for individual orders.  For details, see the
   1033 :ref:`contract terms <contract-terms>` specification.
   1034 
   1035 You can then create the instance using:
   1036 
   1037 .. code-block:: console
   1038 
   1039    $ wget --post-file=instance.json http://localhost:9966/management/instances
   1040 
   1041 The base URL for the instance will then be ``http://localhost:9966/``.  You
   1042 can create additional instances, which will then be reachable under
   1043 ``http://localhost:9966/instances/$ID`` where ``$ID`` needs to be changed to
   1044 the identifier value of the respective instance.
   1045 
   1046 Endpoints to modify (reconfigure), permanently disable (while keeping the data)
   1047 or purge (deleting all associated data) instances exist as well and are documented
   1048 in the :ref:`Merchant Backend API documentation <merchant-api>`.
   1049 
   1050 .. _instance-account-setup:
   1051 
   1052 Instance bank account setup
   1053 ===========================
   1054 
   1055 Before you can use an instance productively, you need to configure one or more
   1056 bank accounts. These bank accounts will be provided to the Taler exchange
   1057 operator to tell it where to wire the income from your sales. Every bank
   1058 account has an associated *wire method* which determines how an exchange can
   1059 transfer the funds.  The most commonly supported wire method is *iban*, which
   1060 implies that bank accounts are identified by IBAN numbers and wire transfers
   1061 are to be executed between IBAN accounts.  For regional currency setups, the
   1062 wire method could also be *x-taler-bank*.
   1063 
   1064 .. note::
   1065 
   1066    When using a regional currency, you need to first create a bank account at
   1067    the regional bank. You may need to contact the respective administrator who
   1068    can set one up.  After being able to login to the new bank account, you can
   1069    see your bank account number by clicking on the ``Welcome, $USERNAME``
   1070    message in the profile page.  Next to the bank account number, you can find
   1071    a convenient button to copy the number to the clipboard.
   1072 
   1073 Not every exchange will support every *wire method*, and if you do not add a
   1074 bank account with a wire method that is supported by a particular exchange,
   1075 then you will not be able to receive payments via that exchange even if you
   1076 configured the merchant backend to trust that exchange.
   1077 
   1078 The simplest way to configure an account is to use the Web interface which has
   1079 specific forms for different wire methods.  First, select ``Bank account`` at
   1080 the left of the page.  The following page should be shown:
   1081 
   1082 .. image:: screenshots/no_default_account_yet.png
   1083 
   1084 Click on the blue "+" sign on the top right of the page to add a new
   1085 bank account. The following page should appear:
   1086 
   1087 .. image:: screenshots/enter_instance_details.png
   1088 
   1089 First, you should select the wire method, after which the dialog will show you
   1090 additional fields specific to the wire method. For example, if you choose
   1091 ``iban`` as the account type, the following page should appear:
   1092 
   1093 .. image:: screenshots/instance_iban_config.png
   1094 
   1095 Specifying the revenue gateway with username and password is optional and
   1096 discussed in section :ref:`automatic-settlement-data-import` below.
   1097 
   1098 After providing the details and confirming, the shop is ready to generate orders
   1099 and accept payments.
   1100 
   1101 
   1102 
   1103 Detecting Settlement: Manually Adding Transfers
   1104 -----------------------------------------------
   1105 
   1106 The exchange may aggregate many small amounts into one larger wire transfer.
   1107 If you want to safely determine which orders have been settled (final
   1108 payment from the exchange has been received), the backend must learn about the
   1109 wire transfers made into your bank account. Basically, as a secure system, we
   1110 do not simply trust a claim by the exchange that it would transfer the money,
   1111 but we allow each merchant to check settlements.
   1112 
   1113 An easy (but somewhat tedious) way to check settlements is to manually add
   1114 every wire transfer that a merchant bank account has received from the
   1115 exchange with the total amount and the wire transfer subject. Given this
   1116 information, the merchant backend will inquire with the exchange which
   1117 individual payments were aggregated, check that the total amount is correct,
   1118 and will then flag the respective contracts as wired.
   1119 
   1120 You can manually enter wire transfers under ``Transfers``. However, this is
   1121 tedious, and so if your banking setup supports it, we highly recommend
   1122 using the automatic settlement data import.
   1123 
   1124 .. _automatic-settlement-data-import:
   1125 
   1126 Automatic Settlement Data Import
   1127 --------------------------------
   1128 
   1129 To automatically import settlement data, you need to provide the merchant
   1130 backend with the address and access credentials of a
   1131 :ref:`taler-bank-merchant-http-api` for each bank account of an instance.  The
   1132 revenue API endpoint will allow the merchant backend to obtain a list of all
   1133 incoming wire transfers into your bank account and automatically import them
   1134 into the list of confirmed wire transfers.
   1135 
   1136 Note that setting up a revenue API endpoint will usually require you to first
   1137 ask your bank for EBICS access and to set up :ref:`libeufin-nexus` to provide
   1138 the revenue API endpoint. The :ref:`libeufin-bank` used by regional currency
   1139 setups also provides a revenue API endpoint at
   1140 ``$BANK_URL/accounts/$ACCOUNT_NAME/taler-revenue/``.  Thus, when using a
   1141 regional currency setup, simply use the ``$BANK_URL`` of your bank and specify
   1142 your bank login name and password in the :ref:`instance-account-setup` dialog.
   1143 
   1144 
   1145 Manually creating an order using the SPA
   1146 ========================================
   1147 
   1148 Arbitrary orders can also be created manually using the Web interface of
   1149 the GNU Taler merchant backend. For this, log into the single page app (SPA)
   1150 of the merchant backend using the authorization token of the respective
   1151 instance you want to use.
   1152 
   1153 Click on ``Orders`` at the top left corner of the merchant backoffice page; the
   1154 following page should appear
   1155 
   1156 .. image:: screenshots/create_orders.png
   1157 
   1158 You can then set up orders by providing all of the required fields of an
   1159 order, in particular an order summary and a price. You can also set various
   1160 optional fields or override instance default settings.
   1161 
   1162 After this the interface should show the following page with the related links
   1163 to check the status of the order and let wallet pay for it.
   1164 
   1165 .. image:: screenshots/payment_links.png
   1166 
   1167 The order status page also shows you the progress of the order, including when
   1168 a wallet has made the payment. You can also use the backend to approve refunds.
   1169 
   1170 In order to test the setup, it should be now possible to use the command line wallet
   1171 to withdraw Taler coins and spend them to pay for the order we just created.
   1172 
   1173 In practice, you will rarely if ever setup orders manually like this. Instead,
   1174 a `GNU Taler e-commerce front-end
   1175 <https://taler.net/en/docs.html#extensions>`_ or the
   1176 :ref:`taler-merchant-pos-app` will do this on-demand. Here, you will only need
   1177 to provide the respective front-ends with the URL of your instance
   1178 (e.g. ``https://backend.$DOMAIN/instances/$NAME``) and your access token.
   1179 
   1180 
   1181 Paying an order
   1182 ===============
   1183 
   1184 The payer simply scans the (dynamic) QR code to initiate the payment.  If a
   1185 website is interacting with a WebExtension wallet, it may also directly
   1186 trigger the GNU Taler wallet without requiring the user to explicitly scan the
   1187 QR code.  The payer should now review the contract terms and applicable fees.
   1188 Selecting "pay" will complete the transaction.  Typically, the wallet will
   1189 then redirect the user to the fulfillment page where they can track the order
   1190 or directly view the digital product that they purchased.
   1191 
   1192 
   1193 Setting up a template
   1194 =====================
   1195 
   1196 A template provides all or part of the information needed to setup an order
   1197 and allows GNU Taler wallets to create an order.  Usually, the creation of
   1198 orders is a privileged process that requires knowledge of the authorization
   1199 code for the respective instance. With templates, a customer's wallet can
   1200 directly create an order on-demand.  The information of a template can be
   1201 partial, in which case the customer is expected to provide the remaining
   1202 details, typically the summary and/or amount of the order.
   1203 
   1204 When setting up a template you need to specify all of the fixed inputs that
   1205 the customer cannot change.  You can then generate a template QR code where
   1206 you may additionally specify editable defaults for the order, such as a
   1207 default summary or a default amount which may still be changed by the wallet.
   1208 The resulting template QR code encodes the specific merchant backend, instance
   1209 and template ID as well as the (editable) default values.  The resulting
   1210 static QR code can then be printed and put on display.
   1211 
   1212 Customers can scan the QR code with their GNU Taler wallet, complete the
   1213 missing details or edit the defaults (if any), and pay the resulting order.
   1214 
   1215 To secure template-based payments, you may specify a TOTP secret as part of
   1216 the template. In this case, the merchant backend will send a set of TOTP
   1217 payment confirmation codes to the GNU Taler wallet upon receiving a payment
   1218 for an order created based on the template.  If the point-of-sale has a TOTP
   1219 generator with the same secret, they can compare their TOTP code with the
   1220 codes shown by the customer on their wallet. This provides additional
   1221 assurance that the customer actually made the payment instead of just showing
   1222 a fake confirmation screen.
   1223 
   1224 
   1225 Paying with static QR codes
   1226 ===========================
   1227 
   1228 The payer simply scans the (static) QR code to initiate the payment.  If the
   1229 template does not specify a fixed amount, the payer will be prompted to enter
   1230 the amount to be paid (and possibly given the opportunity to specify or alter
   1231 the summary).  Selecting "pay" will complete the transaction.  If payment
   1232 confirmations are configured by the merchant backend, the wallet will then
   1233 display a TOTP confirmation code that can be shown to the merchant as a proof
   1234 of payment.
   1235 
   1236 
   1237 
   1238 Setting up a webhook
   1239 ====================
   1240 
   1241 To receive notifications when a purchase has been made or a refund was given
   1242 to a wallet, you can set up webhooks in the GNU Taler merchant backend.
   1243 Webhooks allow you to trigger HTTP(S) requests based on certain events.  A
   1244 webhook is thus simply an HTTP request that the GNU Taler merchant backend
   1245 will make when a certain event (such as a payment) happens.
   1246 
   1247 There are various providers that can send an SMS to a phone number based on an
   1248 HTTP request.  Thus, by configuring such a provider in a webhook you can
   1249 receive an SMS notification whenever a customer makes a payment.
   1250 
   1251 Webhooks are configured per instance.  In the Webhook configuration,
   1252 you can specify which URL, which HTTP headers, which HTTP method and what HTTP
   1253 body to send to the Webhook.  Webhooks are automatically retried (with
   1254 increasing delays) when the target server returns a temporary error.
   1255 
   1256 `Mustach templates <https://mustache.github.io/mustache.5.html>`__ and a limited
   1257 version of it is used when defining the contents of Webhooks.
   1258 Depending on the triggering event, the templates will be expanded with event-specific
   1259 data. Limited in this case means that only a specific string is being replaced
   1260 with the event-specific data, no support for parsing conditions or nested structures
   1261 is provided.
   1262 
   1263 
   1264 Order created events
   1265 --------------------
   1266 
   1267 For "order_created" events, the backend will provide the following
   1268 information to the Mustache templating engine:
   1269 
   1270 * webhook_type: "order_created".
   1271 * order_id: the identifier of the newly created order.
   1272 * contract: the full JSON contract for the order (see :ref:`contract_terms <contract-terms>` for the structure).
   1273 * instance_id: the merchant instance identifier that created the order.
   1274 
   1275 
   1276 Order pay events
   1277 ----------------
   1278 
   1279 For "pay" events, the backend will provide the following
   1280 information to the Mustache templating engine:
   1281 
   1282 * webhook_type: "pay".
   1283 * :ref:`contract_terms <contract-terms>`: the contract terms of the paid order.
   1284 * order_id: the ID of the order that received the refund.
   1285 
   1286 
   1287 Order refund events
   1288 -------------------
   1289 
   1290 For "refund" events which are triggered when a refund is
   1291 approved, the backend will provide the following information to the
   1292 Mustache templating engine:
   1293 
   1294 * webhook_type: "refund".
   1295 * timestamp: time of the refund (using a `Timestamp` with the time in seconds since the UNIX epoch).
   1296 * order_id: the ID of the order that received the refund.
   1297 * :ref:`contract_terms <contract-terms>`: the full JSON of the contract terms of the refunded order.
   1298 * refund_amount: the amount that was being refunded.
   1299 * reason: the reason entered by the merchant staff for granting the refund;
   1300   be careful, you probably want to inform your staff if a webhook may expose
   1301   this information to the consumer.
   1302 
   1303 
   1304 Order settled events
   1305 --------------------
   1306 
   1307 For "order_settled" events which are triggered when the
   1308 **taler-merchant-reconciliation** service is able to map
   1309 an incoming wire transfer from the exchange to a paid
   1310 order, the backend will provide the following information to the limited
   1311 Mustache templating engine:
   1312 
   1313 * webhook_type: "order_settled".
   1314 * order_id: The unique identifier of the order that has been fully settled (all payments completed and wired to the merchant).
   1315 * wtid: The wire transfer ID of the settlement.
   1316 
   1317 Category added events
   1318 ---------------------
   1319 
   1320 For "category_added" events, the backend will provide the following information to the limited
   1321 Mustache templating engine:
   1322 
   1323 * webhook_type: "category_added".
   1324 * category_serial: The unique identifier of the newly added category.
   1325 * category_name: The name of the newly added category.
   1326 * merchant_serial: The unique identifier of the merchant associated with the category.
   1327 
   1328 
   1329 Category updated events
   1330 -----------------------
   1331 
   1332 For "category_updated" events, the backend will provide the following information to the limited
   1333 Mustache templating engine:
   1334 
   1335 * webhook_type: "category_updated".
   1336 * category_serial: The unique identifier of the updated category.
   1337 * old_category_name: The name of the category before the update.
   1338 * category_name: The name of the category after the update.
   1339 * category_name_i18n: The internationalized name of the category after the update.
   1340 * old_category_name_i18n: The internationalized name of the category before the update.
   1341 
   1342 
   1343 Category deleted events
   1344 -----------------------
   1345 
   1346 For "category_deleted" events, the backend will provide the following information to the limited
   1347 Mustache templating engine:
   1348 
   1349 * webhook_type: "category_deleted".
   1350 * category_serial: The unique identifier of the deleted category.
   1351 * category_name: The name of the deleted category.
   1352 
   1353 
   1354 Inventory added events
   1355 ----------------------
   1356 
   1357 For "inventory_added" events, the backend will provide the following information to the limited
   1358 Mustache templating engine:
   1359 
   1360 * webhook_type: "inventory_added".
   1361 * product_serial: The unique identifier of the newly added product.
   1362 * product_id: The ID of the newly added product.
   1363 * description: The description of the newly added product.
   1364 * description_i18n: The internationalized description of the newly added product.
   1365 * unit: The unit of the newly added product.
   1366 * image: The image of the newly added product.
   1367 * taxes: The taxes of the newly added product.
   1368 * price: The price of the newly added product.
   1369 * total_stock: The total stock of the newly added product.
   1370 * total_sold: The total sold of the newly added product.
   1371 * total_lost: The total lost of the newly added product.
   1372 * address: The address of the newly added product.
   1373 * next_restock: The next restock of the newly added product.
   1374 * minimum_age: The minimum age for buying the newly added product.
   1375 
   1376 Inventory updated events
   1377 ------------------------
   1378 
   1379 For "inventory_updated" events, the backend will provide the following information to the limited
   1380 Mustache templating engine:
   1381 
   1382 * webhook_type: "inventory_updated".
   1383 * product_serial: The unique identifier of the updated product.
   1384 * product_id: The ID of the product.
   1385 * old_description: The description of the product before the update.
   1386 * description: The description of the product after the update.
   1387 * old_description_i18n: The internationalized description of the product before the update.
   1388 * description_i18n: The internationalized description of the product after the update.
   1389 * old_unit: The unit of the product before the update.
   1390 * unit: The unit of the product after the update.
   1391 * old_image: The image of the product before the update.
   1392 * image: The image of the product after the update.
   1393 * old_taxes: The taxes of the product before the update.
   1394 * taxes: The taxes of the product after the update.
   1395 * old_price: The price of the product before the update.
   1396 * price: The price of the product after the update.
   1397 * old_total_stock: The total stock of the product before the update.
   1398 * total_stock: The total stock of the product after the update.
   1399 * old_total_sold: The total sold of the product before the update.
   1400 * total_sold: The total sold of the product after the update.
   1401 * old_total_lost: The total lost of the product before the update.
   1402 * total_lost: The total lost of the product after the update.
   1403 * old_address: The address of the product before the update.
   1404 * address: The address of the product after the update.
   1405 * old_next_restock: The next restock of the product before the update.
   1406 * next_restock: The next restock of the product after the update.
   1407 * old_minimum_age: The minimum age for buying the product before the update.
   1408 * minimum_age: The minimum age for buying the product after the update.
   1409 
   1410 
   1411 Inventory deleted events
   1412 ------------------------
   1413 
   1414 For "inventory_deleted" events, the backend will provide the following information to the limited
   1415 Mustache templating engine:
   1416 
   1417 * webhook_type: "inventory_deleted".
   1418 * product_serial: The unique identifier of the deleted product.
   1419 * product_id: The ID of the deleted product.
   1420 * description: The description of the deleted product.
   1421 * description_i18n: The internationalized description of the deleted product.
   1422 * unit: The unit of the deleted product.
   1423 * image: The image of the deleted product.
   1424 * taxes: The taxes of the deleted product.
   1425 * price: The price of the deleted product.
   1426 * total_stock: The total stock of the deleted product.
   1427 * total_sold: The total sold of the deleted product.
   1428 * total_lost: The total lost of the deleted product.
   1429 * address: The address of the deleted product.
   1430 * next_restock: The next restock of the deleted product.
   1431 * minimum_age: The minimum age for buying the deleted product.
   1432 
   1433 
   1434 .. _Secure-setup:
   1435 
   1436 Secure setup
   1437 ============
   1438 
   1439 .. index:: security
   1440 .. index:: TLS
   1441 
   1442 The Taler backend is deliberately simple in terms of support for access
   1443 control or transport layer security (TLS).  Thus, production setups **must**
   1444 deploy the Taler backend behind an HTTP(S) server that acts as a *reverse
   1445 proxy*, performs TLS termination and authentication and then forwards requests
   1446 to the backend.
   1447 
   1448 
   1449 Using UNIX domain sockets
   1450 -------------------------
   1451 
   1452 To ensure that the merchant backend is not exposed directly to the network,
   1453 you *should* bind the backend to a UNIX domain socket:
   1454 
   1455 .. code-block:: ini
   1456 
   1457    [MERCHANT]
   1458    SERVE = unix
   1459    UNIXPATH = "/some/path/here.sock"
   1460 
   1461 Do not use a UNIX domain socket path in "/tmp": systemd (or other init
   1462 systems) may give Web servers a private "/tmp" thereby hiding UNIX domain
   1463 sockets created by other users/processes in "/tmp".
   1464 
   1465 If UNIX domain sockets are for some reason not possible, you *may* use a
   1466 host-based firewall to block access to the TCP port of the merchant backend,
   1467 but this is *not recommended*.  If you do need a TCP socket, you should
   1468 instead strongly consider using the "BIND_TO" option to at least bind it only
   1469 to "localhost".
   1470 
   1471 .. _reverse-proxy-configuration:
   1472 
   1473 Reverse proxy configuration
   1474 ---------------------------
   1475 
   1476 Nginx
   1477 ^^^^^
   1478 
   1479 For Nginx, a possible basic reverse proxy configuration would be:
   1480 
   1481 .. code-block:: nginx
   1482 
   1483       proxy_pass http://unix:/some/path/here.sock;
   1484       proxy_redirect off;
   1485       proxy_set_header Host $host;
   1486       proxy_set_header X-Forwarded-Host "example.com";
   1487       proxy_set_header X-Forwarded-Proto "https";
   1488 
   1489 Note that the above assumes your domain name is ``example.com`` and that you
   1490 have TLS configured.  Leave out the last line if your Nginx reverse proxy does
   1491 not have HTTPS enabled.  Make sure to restart the ``taler-merchant-httpd``
   1492 process after changing the ``SERVE`` configuration.
   1493 
   1494 Apache
   1495 ^^^^^^
   1496 
   1497 In Apache, make sure you have ``mod_proxy``, ``mod_proxy_http`` and
   1498 ``mod_headers`` enabled:
   1499 
   1500 .. code-block:: console
   1501 
   1502    $ a2enmod proxy
   1503    $ a2enmod proxy_http
   1504    $ a2enmod headers
   1505 
   1506 Then configure your Apache reverse proxy like this (you may change the
   1507 endpoint):
   1508 
   1509 .. code-block:: apacheconf
   1510 
   1511        <Location "/">
   1512        ProxyPass "unix:/some/path/here.sock|http://example.com/"
   1513        RequestHeader add "X-Forwarded-Proto" "https"
   1514        </Location>
   1515 
   1516 Note that the above again assumes your domain name is ``example.com`` and that
   1517 you have TLS configured.  Note that you must add the ``https`` header unless
   1518 your site is not available via TLS.
   1519 
   1520 .. _auto-reverse-proxy-configuration:
   1521 
   1522 Automated Reverse Proxy Configuration
   1523 -------------------------------------
   1524 
   1525 GNU Taler provides a helper script, ``taler-merchant-rproxy-setup``, that
   1526 automates the most common reverse-proxy configuration tasks for both Nginx
   1527 and Apache:
   1528 
   1529 -   Detects (or lets you choose) Nginx vs. Apache
   1530 -   Verifies and enables required modules and packages
   1531 -   Obtains (or skips) TLS certificates via Certbot (unless ``--httponly``)
   1532 -   Backs up and populates the stock configuration with your domain
   1533 -   Optionally forces HTTP → HTTPS redirection (``--httpsonly``)
   1534 -   Enables the site and reloads the web server
   1535 
   1536 Usage
   1537 ^^^^^
   1538 
   1539 .. code-block:: console
   1540 
   1541    taler-merchant-rproxy-setup \
   1542      --domain <example.com> [--nginx | --apache] [--httponly | --httpsonly]
   1543 
   1544 Options
   1545 ^^^^^^^
   1546 
   1547 ``--domain <name>``
   1548     (Required) The public domain name to configure.
   1549 
   1550 ``--nginx``
   1551     Force use of Nginx (overriding auto-detection).
   1552 
   1553 ``--apache``
   1554     Force use of Apache2 (overriding auto-detection).
   1555 
   1556 ``--httponly``
   1557     Only configure HTTP (no TLS). Skips Certbot entirely.
   1558 
   1559 ``--httpsonly``
   1560     Enable HTTPS and add an HTTP→HTTPS redirect.
   1561 
   1562 ``-h, --help``
   1563     Show this help message and exit.
   1564 
   1565 .. note::
   1566 
   1567    This script must be run as root (for example via ``sudo``) and requires
   1568    that either Nginx or Apache2 (and Certbot for non-HTTP-only modes) be
   1569    installed on the system.
   1570 
   1571 
   1572 Access control
   1573 --------------
   1574 
   1575 All endpoints with ``/private/`` in the URL must be restricted to authorized
   1576 users of the respective instance.  Specifically, the HTTP server must be
   1577 configured to only allow access to ``$BASE_URL/private/`` to the authorized
   1578 users of the ``admin`` instance, and to ``$BASE_URL/instances/$ID/private/`` to
   1579 the authorized users of the instance ``$ID``.
   1580 
   1581 By default, the GNU Taler merchant backend simply requires the respective
   1582 HTTP requests to include an "Authorization" header with a "Bearer" token
   1583 set to the respective shared secret which must begin with "secret-token:"
   1584 (following RFC 8959).
   1585 
   1586 Note that all of the other endpoints (without ``/private/``)
   1587 are expected to be fully exposed to the Internet, and wallets may have to
   1588 interact with those endpoints directly without client authentication.
   1589 
   1590 
   1591 Status code remapping
   1592 ---------------------
   1593 
   1594 Normal API usage leaks instance existence information.  Distinguishing between
   1595 404 (Not found) and 403 (Forbidden) is useful for diagnostics.
   1596 
   1597 For higher security (by leaking less information), you can add the following
   1598 fragment, which remaps all 404 response codes to 403.
   1599 
   1600 Nginx
   1601 ^^^^^
   1602 
   1603 .. code-block:: nginx
   1604 
   1605       error_page 404 =403 /empty.gif;
   1606 
   1607 Apache
   1608 ^^^^^^
   1609 
   1610 .. code-block:: apacheconf
   1611 
   1612        cond %{STATUS} =404
   1613        set-status 403
   1614 
   1615 
   1616 Customization
   1617 =============
   1618 
   1619 Legal conditions for using the service
   1620 --------------------------------------
   1621 
   1622 .. include:: frags/legal.rst
   1623 
   1624 .. _MerchantTemplateCustomization:
   1625 
   1626 Template Customization
   1627 ----------------------
   1628 
   1629 The installation process will install various HTML templates to be served to
   1630 trigger the wallet interaction. You may change those templates to your own
   1631 design. The templating language used is `C implementation of mustache
   1632 <https://gitlab.com/jobol/mustach>`__, and the templates are in the
   1633 ``share/taler-merchant/templates/`` directory.
   1634 The default templates are maintained directly in the merchant source tree
   1635 under ``src/frontend/``.  They are self-contained HTML documents: their
   1636 styles, scripts and image data are embedded, so serving them does not require
   1637 a separate Web UI build or external runtime resources.
   1638 
   1639 The file names must be of the form ``$NAME.$LANG.must`` where ``$NAME`` is the
   1640 name of the template and ``$LANG`` is the 2-letter language code of the
   1641 template. English templates must exist and will be used as a fallback.  If the
   1642 browser (user-agent) has provided language preferences in the HTTP header and
   1643 the respective language exists, the correct language will be automatically
   1644 served.
   1645 
   1646 The following subsections give details about each of the templates. The
   1647 subsection titles are the ``$NAME`` of the respective template.
   1648 
   1649 request_payment
   1650 ^^^^^^^^^^^^^^^
   1651 
   1652 Page shown to request the user to make a payment.
   1653 
   1654 This template is instantiated using the following information:
   1655 
   1656   * taler_pay_uri: String; the ``taler://pay/`` URI that must be given
   1657     to the wallet to initiate the payment
   1658 
   1659   * taler_pay_qrcode_svg: Image; an SVG image of the QR code with the
   1660     ``taler_pay_uri``.
   1661 
   1662   * order_summary: String; a text summarizing the order
   1663 
   1664   * order_status_url: URL of the merchant backend where the order status
   1665     can be found, useful for long-polling to check if the order has been paid
   1666 
   1667   * order_id: String; the merchant's identifier for the order
   1668 
   1669   * merchant_name: String; the configured display name of the merchant
   1670     instance
   1671 
   1672   * order_amount: Optional amount; the amount to pay when the order has one
   1673     payment option or the wallet has selected an option
   1674 
   1675   * multiple_payment_options: Boolean; true when the wallet must select one
   1676     of multiple payment options and no single amount can be displayed yet
   1677 
   1678   * claimed: Boolean; true after a wallet has claimed the order but before
   1679     payment has completed
   1680 
   1681   * pay_deadline: Timestamp; the deadline for completing the payment
   1682 
   1683 
   1684 offer_refund
   1685 ^^^^^^^^^^^^
   1686 
   1687 Page shown to offer a customer a refund.
   1688 
   1689 This template is instantiated using the following information:
   1690 
   1691   * taler_refund_uri: String; the ``taler://refund/`` URI that must be given
   1692     to the wallet to initiate collection of the refund
   1693 
   1694   * taler_refund_qrcode_svg: Image; an SVG image of the QR code with the
   1695     ``taler_refund_uri``.
   1696 
   1697   * refund_amount: Amount; how much did the merchant refund
   1698 
   1699   * refund_taken: Amount; how much did the customer already take back in refunds
   1700 
   1701   * order_summary: String; a text summarizing the order
   1702 
   1703   * refund_remaining: Amount; how much is still available for the wallet to
   1704     collect
   1705 
   1706   * has_refund_taken: Boolean; true when part of the refund was already
   1707     collected
   1708 
   1709   * order_id: String; the merchant's identifier for the order
   1710 
   1711   * merchant_name: String; the configured display name of the merchant
   1712     instance
   1713 
   1714   * order_status_url: Authenticated URL of the merchant backend where the
   1715     refund status can be long-polled
   1716 
   1717 
   1718 
   1719 show_order_details
   1720 ^^^^^^^^^^^^^^^^^^
   1721 
   1722 Page shown to the user when they go back to the payment page but
   1723 no payment is required and no refund is present.
   1724 
   1725 This template is instantiated using the following information:
   1726 
   1727   * order_summary: String; a text summarizing the order
   1728 
   1729   * contract_terms: Object; the full contract terms (should probably
   1730     not be shown in full!)
   1731 
   1732   * refund_amount: Amount; how much did the merchant refund
   1733 
   1734   * refund_taken: Amount; how much did the customer already take back in refunds
   1735 
   1736   * order_amount: Amount; the amount paid for the selected payment option
   1737 
   1738   * order_id: String; the merchant's identifier for the order
   1739 
   1740   * merchant_name: String; the configured display name of the merchant
   1741     instance
   1742 
   1743   * has_refund: Boolean; true when the merchant granted a refund for this
   1744     order
   1745 
   1746   * fulfillment_message: Optional string; localized message to show after
   1747     successful payment
   1748 
   1749   * fulfillment_url: Optional URL; destination where the customer can
   1750     continue to the purchased product or service
   1751 
   1752 Static files
   1753 ------------
   1754 
   1755 The merchant backend also has the ability to serve small static files
   1756 under the ``/static/{FILENAME}`` endpoint.  This is used by the templating
   1757 logic to load a CSS file, but you can also put other resources such as
   1758 images or JavaScript.
   1759 
   1760 Internationalization
   1761 --------------------
   1762 
   1763 Both templates and static files can be internationalized.  This is done
   1764 by having the language of the resource be a part of the filename.
   1765 For templates the format is ``{BASENAME}.{LANGUAGE}.must``.  The
   1766 language is mandatory for templates, the default language is English (en).
   1767 
   1768 For static files, the format is ``{BASENAME}.{LANGUAGE}.{EXT}`` for
   1769 internationalized files, and ``{BASENAME}.{EXT}`` for resources that do not
   1770 support internationalization.  The HTTP client will always request
   1771 ``/static/{BASENAME}.{EXT}``. If ``{BASENAME}.{EXT}`` exists, that resource is
   1772 returned. Otherwise, an internationalized file based on the language
   1773 preferences indicated by the browser is returned.
   1774 
   1775 Limitations
   1776 -----------
   1777 
   1778 All of the static files must fit into memory and it must be possible for the
   1779 process to hold open file handles for all of these files.  You may want
   1780 to increase the ``ulimit`` of the ``taler-merchant-httpd`` process if you have
   1781 many static files. Note that Mustach templates do not increase the number of
   1782 open files.
   1783 
   1784 The backend determines the MIME type based on the file's extension. The list
   1785 of supported extensions is hard-coded and includes common text and image
   1786 formats.
   1787 
   1788 The current backend only provides a limited set of variables for the Mustach
   1789 template expansion, and does not make use of scopes and other Mustach
   1790 features.
   1791 
   1792 
   1793 
   1794 Upgrade procedure
   1795 =================
   1796 
   1797 This section describes the general upgrade procedure.  Please see the release
   1798 notes for your specific version to check if a particular release has special
   1799 upgrade requirements.
   1800 
   1801 Please note that upgrades are ONLY supported for released version of the
   1802 merchant. Attempting to upgrade from or to a version in Git is not supported
   1803 and may result in subtle data loss.
   1804 
   1805 Debian packages do not migrate the database or restart Merchant on upgrade.
   1806 The operator must coordinate the complete upgrade; leaving old processes running
   1807 is not a way to defer migration, since a crash or runtime limit can start newly
   1808 installed binaries against the old database.
   1809 
   1810 1. Stop all Merchant services **before installing the new packages**. On Debian,
   1811    run ``sudo systemctl stop taler-merchant.target``. This includes workers,
   1812    the HTTP service and socket, and the garbage-collection timer. If multiple
   1813    machines share the database, stop the Merchant services on every machine.
   1814 2. Back up the database using your PostgreSQL backup procedure.
   1815 3. Install the new packages. Services remain stopped.
   1816 4. Apply the migration. For the standard Debian installation using the default
   1817    local PostgreSQL cluster, run:
   1818 
   1819    .. code-block:: console
   1820 
   1821       $ sudo taler-merchant-dbconfig
   1822 
   1823    This preserves the existing database and invokes ``taler-merchant-dbinit``
   1824    as ``taler-merchant-httpd`` with
   1825    ``/etc/taler-merchant/taler-merchant.conf``. Never use ``-r`` for an upgrade:
   1826    it deletes the existing database.
   1827 
   1828    For remote databases or separately administered credentials, run
   1829    ``taler-merchant-dbinit -c /path/to/merchant.conf`` as the database owner
   1830    or designated migration identity instead. Ensure all required migration
   1831    privileges are available; ``dbconfig`` grants permission to set
   1832    ``session_replication_role`` but does not restore other revoked privileges.
   1833 5. After migration succeeds, restore any normal-operation privilege restrictions
   1834    and start the desired Merchant services. Use
   1835    ``sudo systemctl start taler-merchant.target`` to start the complete group.
   1836    If migration fails, keep all Merchant services stopped while resolving it.
   1837 
   1838 ``taler-merchant-dbconfig`` does not stop services, make backups, or restart
   1839 services. Its successful completion only confirms the database setup/migration
   1840 step. Installation without a configured database remains supported.
   1841 
   1842 Scheduled garbage collection (``taler-merchant-dbinit -g``) never initializes
   1843 or upgrades the schema or replaces stored procedures. It requires a current
   1844 schema and fails with a diagnostic if migration is needed.
   1845 
   1846 
   1847 
   1848 
   1849 Advanced topics
   1850 ===============
   1851 
   1852 taler-merchant-config
   1853 ---------------------
   1854 
   1855 .. index:: taler-merchant-config
   1856 
   1857 .. include:: frags/using-taler-config.rst
   1858 
   1859 .. _MerchantDatabaseScheme:
   1860 
   1861 Database Scheme
   1862 ---------------
   1863 
   1864 The merchant database must be initialized using ``taler-merchant-dbinit``.
   1865 This tool creates the tables required by the Taler merchant to operate.
   1866 The tool also allows you to reset the Taler merchant database, which is
   1867 useful for test cases but should never be used in production. Finally,
   1868 ``taler-merchant-dbinit`` has a function to garbage collect a database,
   1869 allowing administrators to purge records that are no longer required.
   1870 
   1871 The database scheme used by the merchant looks as follows:
   1872 
   1873 .. image:: images/merchant-db.png
   1874 
   1875 Schema migrations and stored procedures
   1876 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   1877 
   1878 Merchant schema changes are delivered as contiguously numbered SQL patches
   1879 and tracked through the shared ``versioning.sql`` mechanism.  Once a patch has
   1880 been part of a release or deployed to production or staging, it is immutable;
   1881 a correction requires another patch.  The merchant does not use the
   1882 exchange's partition/shard master-table scheme.
   1883 
   1884 Runtime functions and procedures are collected in the generated
   1885 ``procedures.sql`` and loaded after the numbered schema patches.  Changing a
   1886 runtime procedure therefore does not alter the schema-patch version.  Table,
   1887 index, constraint, or stored-data changes still require a numbered patch.
   1888 Developers must update the procedure sources and regenerate the aggregate
   1889 rather than editing an installed database or generated output directly.
   1890 ``taler-merchant-dbinit`` is the supported installation and upgrade path.
   1891 
   1892 Statistics storage and operation
   1893 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   1894 
   1895 Merchant statistics are maintained transactionally by database triggers, so
   1896 the statistics endpoints return current values rather than a periodically
   1897 computed snapshot.  Amount-valued statistics are separated by currency.
   1898 Operators can query amount and counter slugs as bucketed values, sliding
   1899 intervals, or both through the ``statistics-read`` API endpoints documented in
   1900 the Merchant API.
   1901 
   1902 Bucket metadata defines calendar ranges and the number of generations to
   1903 retain.  Interval metadata defines monotonically increasing windows and their
   1904 precision.  Old event slots may be coarsened to the precision of a larger
   1905 window and are eventually removed by statistics garbage collection.  A
   1906 missing old value can therefore mean that the deployment does not retain that
   1907 granularity; clients must handle the API's ``410 Gone`` response.
   1908 
   1909 Developers adding a built-in statistic use the database procedures
   1910 ``bump_number_bucket_stat``, ``bump_amount_bucket_stat``,
   1911 ``bump_number_interval_stat``, ``bump_amount_interval_stat``,
   1912 ``bump_number_stat``, and ``bump_amount_stat`` rather than updating statistics
   1913 tables directly.  Reads of sliding intervals go through
   1914 ``statistic_interval_number_get`` or ``statistic_interval_amount_get`` so that
   1915 expiry and coarsening are applied consistently.  ``statistic_bucket_gc``
   1916 removes expired buckets and event slots.
   1917 
   1918 Statistics schema changes follow the ordinary merchant database versioning
   1919 and are installed or upgraded by ``taler-merchant-dbinit``.  Operators must not
   1920 modify the core statistics tables or triggers outside that versioning process.
   1921 
   1922 Periodic report scheduling is a separate layer: configured report generators
   1923 consume the stored statistics through the report APIs.  Operators must
   1924 configure the generator section before scheduling it; an unconfigured program
   1925 is reported as ``501 Not Implemented``.
   1926 
   1927 .. _MerchantBenchmarking:
   1928 
   1929 Benchmarking
   1930 ------------
   1931 
   1932 The merchant codebase offers the ``taler-merchant-benchmark`` tool to populate
   1933 the database with fake payments.  The main goal of the benchmarking tool is to
   1934 serve as a starting point (!) for merchants that are interested in developing
   1935 stress tests to see how far their infrastructure can scale. As is, it
   1936 currently is not actually good at stressing the payment system.
   1937 
   1938 The ``taler-unified-setup.sh`` script can be used to launch all required
   1939 services and clients. However, the resulting deployment is simplistic
   1940 (everything on the local machine, one single-threaded process per service
   1941 type) and not optimized for performance at all. However, this can still be
   1942 useful to assess the performance impact of changes
   1943 to the code or configuration.
   1944 
   1945 
   1946 Various configuration files that can be used in the code snippets in this
   1947 section can be found in the ``src/merchant-tools/`` directory of the
   1948 merchant. These are generally intended as starting points.  Note that the
   1949 configuration files ending in ``.edited`` are created by
   1950 ``taler-unified-setup.sh`` and contain some options that are determined at
   1951 runtime by the setup logic provided by ``taler-unified-setup.sh``.
   1952 
   1953 See :ref:`Taler Exchange Manual <Benchmark-choose-bank>` for how to use ``taler-unified-setup.sh`` to setup the system and in particular on how to specify the bank to be used.
   1954 
   1955 
   1956 Running taler-merchant-benchmark
   1957 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   1958 
   1959 You can run the tool as follows:
   1960 
   1961 .. code-block:: console
   1962 
   1963     $ CONF=benchmark-rsa.conf
   1964     $ taler-unified-setup.sh -emwt -c "$CONF" -f -u exchange-account-1
   1965     $ time taler-merchant-benchmark ordinary -c "$CONF".edited -u exchange-account-1 -f -p 20
   1966 
   1967 The current tool has already a few options, but we expect that to deliver
   1968 *relevant* results it will need to be customized to better reflect the
   1969 workload of a particular merchant.  This customization would at this point
   1970 likely involve writing (C) code.  We welcome contributions to make it easier
   1971 to customize the benchmark and/or to cover more realistic workloads from the
   1972 start.
   1973 
   1974 The tool takes all of the values it needs from the command line, with
   1975 some of them being common to all subcommands:
   1976 
   1977 -  ``--exchange-account-section=SECTION`` Specifies which configuration
   1978    section specifies the bank account for the exchange that
   1979    should be used for the benchmark. For the example
   1980    configuration above, the SECTION value provided must be
   1981    ``exchange-account-exchange``.
   1982 -  ``--fakebank`` Specifies that the benchmark should expect to interact
   1983    with a fakebank (instead of libeufin).
   1984 
   1985 The tool comes with two operation modes: *ordinary*, and *corner*.
   1986 The first just executes normal payments, meaning that it uses the
   1987 ``admin`` instance and makes sure that all payments get aggregated. The
   1988 second gives the chance to leave some payments unaggregated, and also to
   1989 use merchant instances other than ``admin`` (which is, actually, the
   1990 one used by default by the tool).
   1991 
   1992 .. note::
   1993 
   1994    The ability to drive the aggregation policy is useful for testing
   1995    the back-office facility.
   1996 
   1997 Any subcommand is also equipped with the canonical ``--help`` option, so
   1998 feel free to issue the following command in order to explore all the
   1999 possibilities. For example:
   2000 
   2001 .. code-block:: console
   2002 
   2003    $ taler-merchant-benchmark corner --help
   2004 
   2005 will show all the options offered by the *corner* mode. Among the most
   2006 interesting, there are:
   2007 
   2008 -  ``--two-coins=TC`` This option instructs the tool to perform *TC*
   2009    many payments that use two coins, because normally only one coin is
   2010    spent per payment.
   2011 
   2012 -  ``--unaggregated-number=UN`` This option instructs the tool to
   2013    perform *UN* (one coin) payments that will be left unaggregated.
   2014 
   2015 As for the ``ordinary`` subcommand, it is worth explaining the following
   2016 option:
   2017 
   2018 -  ``--payments-number=PN`` Instructs the tool to perform *PN* payments.
   2019 
   2020 
   2021 
   2022 Temporarily Abandoned Features
   2023 ==============================
   2024 
   2025 .. [1]
   2026    https://docs.docker.com/
   2027 
   2028 
   2029 Installing Taler using Docker
   2030 -----------------------------
   2031 
   2032 This section provides instructions for the merchant backend installation
   2033 using ‘Docker‘.
   2034 
   2035 For security reasons, we run Docker against a VirtualBox instance, so
   2036 the ``docker`` command should connect to a ``docker-machine`` instance
   2037 that uses the VirtualBox driver.
   2038 
   2039 Therefore, the needed tools are: “docker“, “docker-machine“, and
   2040 “docker-compose“. Please refer to Docker’s official  [1]_ documentation
   2041 in order to get those components installed, as that is not in this
   2042 manual’s scope.
   2043 
   2044 Before starting to build the merchant’s image, make sure a
   2045 “docker-machine“ instance is up and running.
   2046 
   2047 Because all of the Docker source file are kept in our “deployment“
   2048 repository, we start by checking out the ``git://git.taler.net/deployment``
   2049 codebase:
   2050 
   2051 .. code-block:: console
   2052 
   2053    $ git clone git://git.taler.net/deployment
   2054 
   2055 Now we actually build the merchant’s image. From the same directory as
   2056 above:
   2057 
   2058 .. code-block:: console
   2059 
   2060    $ cd deployment/docker/merchant/
   2061    $ docker-compose build
   2062 
   2063 If everything worked as expected, the merchant is ready to be launched.
   2064 From the same directory as the previous step:
   2065 
   2066 .. code-block:: console
   2067 
   2068    # Recall: the docker-machine should be up and running.
   2069    $ docker-compose up
   2070 
   2071 You should see some live logging from all the involved containers. At
   2072 this stage of development, you should also ignore some (harmless) error
   2073 message from postresql about already existing roles and databases.
   2074 
   2075 To test if everything worked as expected, it suffices to issue a simple
   2076 request to the merchant, for example:
   2077 
   2078 .. code-block:: console
   2079 
   2080    $ wget -O - http://$(docker-machine ip)/
   2081    # A greeting message should be returned by the merchant.