taler-exchange-manual.rst (73514B)
1 .. 2 This file is part of GNU TALER. 3 4 Copyright (C) 2014-2024 Taler Systems SA 5 6 TALER is free software; you can redistribute it and/or modify it under the 7 terms of the GNU Affero General Public License as published by the Free Software 8 Foundation; either version 3.0, or (at your option) any later version. 9 10 TALER is distributed in the hope that it will be useful, but WITHOUT ANY 11 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR 12 A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. 13 14 You should have received a copy of the GNU Affero General Public License along with 15 TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> 16 17 @author Christian Grothoff 18 @author Florian Dold 19 20 Exchange Operator Manual 21 ######################## 22 23 Introduction 24 ============ 25 26 About GNU Taler 27 --------------- 28 29 .. include:: frags/about-taler.rst 30 31 32 About this manual 33 ----------------- 34 35 This manual targets system administrators who want to install and 36 operate a GNU Taler exchange. 37 38 39 Organizational prerequisites 40 ---------------------------- 41 42 Operating a GNU Taler exchange means that you are operating a payment service 43 provider, which means that you will most likely need a bank license and/or 44 follow applicable financial regulation. The :ref:`Exchange KYC/AML Operator 45 Manual <KycOperatorManual>` goes into depth for how an exchange can be 46 configured for compliance. Exceptions to compliance rules may apply, 47 especially if you are operating a regional currency or a payment system for an 48 event with a closed user group. If in doubt, please contact a competent 49 lawyer in your country about the legality of operating a GNU Taler exchange. 50 51 GNU Taler payment service providers generally need to ensure high availability 52 and should have *really* good backups (synchronous replication, asynchronous 53 remote replication, off-site backup, 24/7 monitoring, etc.). This manual will 54 not cover these aspects of operating a payment service provider. 55 56 We will assume that you can operate a (sufficiently high-availability, 57 high-assurance) PostgreSQL database. Furthermore, we expect some moderate 58 familiarity with the compilation and installation of free software 59 packages. You need to understand the cryptographic concepts of private and 60 public keys and must be able to protect private keys stored in files on disk. 61 62 .. include:: frags/db-stores-sensitive-data.rst 63 64 65 Architecture overview 66 --------------------- 67 68 GNU Taler is a pure payment system, not a crypto-currency. As such, it 69 operates in a traditional banking context. In particular, this means that 70 payments can be executed in ordinary currencies such as USD or EUR. 71 Furthermore, a typical merchant in Taler has a regular bank account, and would 72 use it to receive funds via Taler. 73 74 Consequently, a typical Taler exchange must interact with a bank. The bank of 75 the exchange holds funds in an account where the balance is basically 76 equivalent to the value of all coins in circulation. (Small mismatches arise 77 whenever customers are about to withdraw coins and have already sent the funds 78 into the bank account, or if merchants just deposited coins and are about to 79 receive wire transfers for deposited coins, or due to fees charged by the 80 exchange and the operator not yet having drained the fees from the account.) 81 82 The exchange uses an intermediary system to talk to its bank. This shifts the 83 technical burden (XML-based communications, additional cryptography, and a 84 vast variety of standards) for this interaction into another bank-specific 85 subsystem. Such intermediary system abstracts the native banking protocol by 86 exposing the *Taler Wire Gateway API*; this way, the exchange can conduct its 87 banking operations in a simplified and JSON-based style. 88 89 When customers wire money to the exchange's bank account, the Taler Wire 90 Gateway API must notify the exchange about the incoming wire transfers. The 91 exchange then creates a *reserve* based on the subject of the wire 92 transfer. The wallet which knows the secret key matching the wire transfer 93 subject can then withdraw coins from the reserve, thereby draining it. The 94 liability of the exchange against the reserve is thereby converted into a 95 liability against digital coins issued by the exchange. When the customer 96 later spends the coins at a merchant, and the merchant *deposits* the coins at 97 the exchange, the exchange first *aggregates* the amount from multiple 98 deposits from the same merchant and then instructs its bank to make a wire 99 transfer to the merchant, thereby fulfilling its obligation and eliminating 100 the liability. The exchange charges *fees* for some or all of its operations 101 to cover costs and possibly make a profit. 102 103 *Auditors* are third parties, for example financial regulators, that verify 104 that the exchange operates correctly. The same software is also used to 105 calculate the exchange’s profits, risk and liabilities by the accountants of 106 the exchange. 107 108 The Taler software stack for an exchange consists of the following 109 components: 110 111 - **HTTP frontend**: 112 The HTTP frontend interacts with Taler wallets and merchant backends. 113 It is used to withdraw coins, deposit coins, refresh coins, issue 114 refunds, map wire transfers to Taler transactions, inquire about the 115 exchange’s bank account details, signing keys and fee structure. The 116 binary is the ``taler-exchange-httpd``. 117 118 - **Crypto-Helpers**: 119 The ``taler-exchange-secmod-rsa``, ``taler-exchange-secmod-cs`` and 120 ``taler-exchange-secmod-eddsa`` 121 are three programs that are responsible for managing the exchange's 122 online signing keys. They must run on the same machine as the 123 ``taler-exchange-httpd`` as the HTTP frontend communicates with the 124 crypto helpers using UNIX Domain Sockets. 125 126 - **Aggregator**: 127 The aggregator combines multiple deposits made by the same merchant 128 and (eventually) triggers wire transfers for the aggregate amount. 129 The merchant can control how quickly wire transfers are made. The 130 exchange may charge a fee per wire transfer to discourage 131 excessively frequent transfers. The binary is the 132 ``taler-exchange-aggregator``. 133 134 - **Closer**: 135 The ``taler-exchange-closer`` tool check that reserves are properly 136 closed. If a customer wires funds to an exchange and then fails 137 to withdraw them, the closer will (eventually) trigger a wire 138 transfer that sends the customer's funds back to the originating 139 wire account. 140 141 - **Transfer**: 142 The ``taler-exchange-transfer`` tool is responsible for actually 143 executing the aggregated wire transfers. It is the only process 144 that needs to have the credentials to execute outgoing wire 145 transfers. The tool uses the Taler Wire Gateway API to execute 146 wire transfers. This API is provided by the Taler Python Bank 147 for stand-alone deployments (like those with ``KUDOS``) and 148 by LibEuFin. LibEuFin is an adapter which maps the Taler Wire 149 REST API to traditional banking protocols like EBICS and FinTS. 150 151 - **Wirewatch**: 152 The ``taler-exchange-wirewatch`` tool is responsible for observing 153 incoming wire transfers to the exchange. It needs to have the 154 credentials to obtain a list of incoming wire transfers. 155 The tool also uses the Taler Wire Gateway API to observe such 156 incoming transfers. It is possible that observing incoming and 157 making outgoing wire transfers is done via different bank accounts 158 and/or credentials. 159 160 - **Wire adapter**: 161 A wire adapter is a component that enables exchange to talk to a bank. 162 Each wire adapter must implement the Taler Wire Gateway API. Three 163 wire adapters are currently provided: 164 165 (1) The **libtalerfakebank** implements a bank with a wire adapter API 166 inside of a testcase. ``taler-fakebank-run`` is a stand-alone 167 process using libtalerfakebank. Note that this adapter is only 168 useful for tests, as all transaction data is kept in memory. 169 The binary is packaged in the ``libtalerexchange-dev`` package 170 on Debian/Ubuntu and possibly other platforms as it is not intended 171 for production. 172 (2) For production, **libeufin**'s ``libeufin-nexus`` component 173 implements a wire adapter towards the traditional SEPA banking 174 system with IBAN accounts using the EBICS protocol. 175 (3) **taler-magnet-bank** component 176 implements a wire adapter towards the Magnet Bank API. 177 (4) **taler-cyclos** component 178 implements a wire adapter towards the Cyclos API. 179 (5) **taler-wise** component 180 implements a wire adapter towards the Wise API. 181 (6) To use GNU Taler with blockchains, the **Depolymerization** 182 component provides a wire gateway API that runs on top of 183 blockchains like Bitcoin and Ethereum. 184 185 The client-side wire adapter API is implemented in **libtalerbank** and 186 is used by ``taler-exchange-transfer`` to execute wire transfers and by 187 ``taler-exchange-wirewatch`` and the Taler auditor to query bank 188 transaction histories. 189 190 - **DBMS**: 191 The exchange requires a DBMS to store the transaction history for 192 the Taler exchange and aggregator, and a (typically separate) DBMS 193 for the Taler auditor. For now, the GNU Taler reference implementation 194 only supports PostgreSQL, but the code could be easily extended to 195 support another DBMS. 196 .. index:: PostgreSQL 197 198 - **Auditor**: 199 The auditor verifies that the transactions performed by the exchange 200 were done properly. It checks the various signatures, totals up the 201 amounts and alerts the operator to any inconsistencies. It also 202 computes the expected bank balance, revenue and risk exposure of the 203 exchange operator. The main binary is the ``taler-auditor``. 204 Aside from the key setup procedures, the most critical setup for 205 deploying an auditor is providing the auditor with an up-to-date 206 copy of the exchange's database. 207 208 209 .. _KeyTypes: 210 211 Key Types 212 --------- 213 214 The exchange works with four types of keys: 215 216 - master key (kept offline, configured manually at merchants and wallets) 217 218 - online message signing keys (signs normal messages from the exchange) 219 220 - denomination keys (signs digital coins) 221 222 - security module keys (signs online message signing keys and denomination keys) 223 224 Additionally, the exchange is sometimes concerned with the auditor's public 225 key (to verify messages signed by auditors approved by the exchange operator) 226 and the merchant's public key (to verify refunds are authorized by the 227 merchant). 228 229 Most of the keys are managed fully automatically or configured as part of the 230 denomination configuration. Some configuration settings must be manually 231 set with regards to the exchange's master key. 232 233 234 Offline keys 235 ------------ 236 237 The exchange (and ideally also its auditor(s)) uses a long-term offline master 238 siging key that identifies the operator and is used to authenticate critical 239 information, such as the exchange's bank account and the actual keys the 240 exchange uses online. 241 242 Interactions with the offline system are performed using the 243 ``taler-exchange-offline`` tool. To use the offline system will require 244 exchange operators to copy JSON files from or to the offline system (say using 245 an USB stick). The offline system does not need any significant amount of 246 computing power, a Raspberry-Pi is perfectly sufficient and the form-factor 247 might be good for safe-keeping! (You should keep a copy of the (encrypted) 248 private offline key on more than one physical medium though.) 249 250 Exchange operators are strongly advised to secure their private master key and 251 any copies on encrypted, always-offline computers. Again, this manual assumes 252 that you are familiar with good best practices in operational security, 253 including securing key material. 254 255 256 Online signing key security 257 --------------------------- 258 259 To provide an additional level of protection for the private *online* signing 260 keys used by the exchange, the actual cryptographic signing operations are 261 performed by three helper processes, ``taler-exchange-secmod-rsa``, 262 ``taler-exchange-secmod-cs`` and ``taler-exchange-secmod-eddsa``. 263 264 The current implementation does not yet support the use of a hardware security 265 module (HSM). If you have such a device with adequate functionality and are 266 interested in Taler supporting it, please contact the developers for HSM 267 integration support. 268 269 270 Functionality 271 ^^^^^^^^^^^^^ 272 273 The UNIX domain sockets of the *secmod* helpers have mode 0620 (u+rw, g+w). 274 The exchange process MUST thus be in the same group as the crypto helper 275 processes to enable access to the keys. No other users should be in that 276 group! 277 278 The three helper processes will create the required private keys, and allow 279 anyone with access to the UNIX domain socket to sign arbitrary messages with 280 the keys or to inform them about a key being revoked. The helper processes 281 are also responsible for deleting the private keys if their validity period 282 expires or if they are informed about a key having been revoked. 283 284 285 Security goals 286 ^^^^^^^^^^^^^^ 287 288 From a security point of view, the helpers are designed to *only* make it 289 harder for an attacker who took control of the HTTP daemon's account to 290 extract the private keys, limiting the attacker's ability to create 291 signatures to the duration of their control of that account. 292 293 .. note:: 294 In the future, the helper processes should additionally provide a mechanism 295 to track the total number of signatures they have made for the various keys. 296 297 Setup 298 ^^^^^ 299 300 The helper processes should be run under a user ID that is separate from that 301 of the user running the main ``taler-exchange-httpd`` service. To get any 302 security benefit from this, it is important that helpers run under a different 303 user ID than the main HTTP frontend. In fact, ideally, each helper should run 304 under its own user ID. The ``taler-exchange-httpd`` service will securely 305 communicate with the helpers using UNIX domain sockets. 306 307 Configuration 308 ^^^^^^^^^^^^^ 309 310 The helpers and the exchange HTTP service need both access to the same 311 configuration information. Having divergent configurations may result in 312 run-time failures. It is recommended that the configuration file (``-c`` 313 option) is simply shared between all of the different processes, even though 314 they run as different system users. The configuration does not contain any 315 sensitive information. 316 317 318 .. _ExchangeInstallation: 319 320 Installation 321 ============ 322 323 Before installing a Taler exchange, please make sure that your system does not 324 have swap space enabled. Swap space is a security risk that Taler does not 325 try to mitigate against. 326 327 We recommend the setup of offline signing keys to be done on a second machine that 328 does not have Internet access. 329 330 In this guide's shell-session fragments, the command prompt shows two pieces 331 of information: 332 333 * Who is performing the command 334 (``$user`` vs ``root``, and ending character ``$`` vs ``#``). 335 * Host where the command is supposed to be executed 336 (``exchange-offline`` vs ``exchange-online``). 337 It is possible to do the entire setup on one machine, 338 but we do not recommend this for security reasons. 339 340 Before you start 341 ---------------- 342 343 To deploy this with a real bank, you need: 344 345 * IBAN of the bank account to use 346 * BIC of the bank 347 * EBICS host, user and partner IDs 348 349 Information to write down during the installation: 350 351 * LibEuFin Nexus superuser password 352 * Taler facade base URL 353 * exchange Nexus username and password 354 355 356 357 Installing from source 358 ---------------------- 359 360 The following instructions will show how to install libgnunetutil and 361 the GNU Taler exchange from source. 362 363 The package sources can be find in our 364 `download directory <http://ftpmirror.gnu.org/taler/>`__. 365 366 .. include:: frags/semver.rst 367 368 First, the following packages need to be installed before we can compile the 369 backend: 370 371 .. include:: frags/list-of-dependencies.rst 372 373 .. include:: frags/installing-gnunet.rst 374 375 .. include:: frags/install-before-check.rst 376 377 There is no need to actually run a GNUnet peer to use the Taler merchant 378 backend -- all the merchant needs from GNUnet is a number of headers and 379 libraries! 380 381 .. include:: frags/installing-taler-exchange.rst 382 383 .. include:: frags/install-before-check.rst 384 385 386 387 Installing the GNU Taler binary packages on Debian 388 -------------------------------------------------- 389 390 .. include:: frags/installing-debian.rst 391 392 To install the Taler exchange, you can now simply run: 393 394 .. code-block:: shell-session 395 396 [root@exchange-online]# apt install taler-exchange 397 398 Note that the package does not perform any configuration work except for 399 setting up the various users and the systemd service scripts. You still must 400 configure at least the database, HTTP reverse proxy (typically with TLS 401 certificates), denomination and fee structure, bank account, auditor(s), 402 offline signing and the terms of service. 403 404 On the offline system, you should run at least: 405 406 .. code-block:: shell-session 407 408 [root@exchange-offline]# apt install taler-exchange-offline 409 410 411 Installing the GNU Taler binary packages on Ubuntu 412 -------------------------------------------------- 413 414 .. include:: frags/installing-ubuntu.rst 415 416 To install the Taler exchange, you can now simply run: 417 418 .. code-block:: shell-session 419 420 [root@exchange-online]# apt install taler-exchange 421 422 Note that the package does not perform any configuration work except for 423 setting up the various users and the systemd service scripts. You still must 424 configure at least the database, HTTP reverse proxy (typically with TLS 425 certificates), denomination and fee structure, bank account, auditor(s), 426 offline signing and the terms of service. 427 428 On the offline system, you should run at least: 429 430 .. code-block:: shell-session 431 432 [root@exchange-offline]# apt install taler-exchange-offline 433 434 435 Services, users, groups and file system hierarchy 436 ------------------------------------------------- 437 438 The *taler-exchange* package will create several system users 439 to compartmentalize different parts of the system: 440 441 * ``taler-exchange-httpd``: runs the HTTP daemon with the core business logic. 442 * ``taler-exchange-secmod-rsa``: manages the RSA private online signing keys. 443 * ``taler-exchange-secmod-cs``: manages the CS private online signing keys. 444 * ``taler-exchange-secmod-eddsa``: manages the EdDSA private online signing keys. 445 * ``taler-exchange-closer``: closes idle reserves by triggering wire transfers that refund the originator. 446 * ``taler-exchange-aggregator``: aggregates deposits into larger wire transfer requests. 447 * ``taler-exchange-transfer``: performs wire transfers with the bank (via LibEuFin/Nexus). 448 * ``taler-exchange-wirewatch``: checks for incoming wire transfers with the bank (via LibEuFin/Nexus). 449 * ``postgres``: runs the PostgreSQL database (from *postgresql* package). 450 * ``www-data``: runs the frontend HTTPS service with the TLS keys (from *nginx* package). 451 452 .. note:: 453 454 The *taler-merchant* package additionally creates a ``taler-merchant-httpd`` user 455 to run the HTTP daemon with the merchant business logic. 456 457 458 The exchange setup uses the following system groups: 459 460 * ``taler-exchange-db``: group for all Taler users with direct database access, specifically taler-exchange-httpd, taler-exchange-wirewatch, taler-exchange-closer and taler-exchange-aggregator. 461 * ``taler-exchange-secmod``: group for processes with access to online signing keys; this group must have four users: taler-exchange-secmod-rsa, taler-exchange-secmod-cs, taler-exchange-secmod-eddsa and taler-exchange-httpd. 462 * ``taler-exchange-offline``: group for the access to the offline private key (only used on the offline host and not used on the online system). 463 464 465 The package will deploy systemd service files in 466 ``/usr/lib/systemd/system/`` for the various components: 467 468 * ``taler-exchange-aggregator.service``: service that schedules wire transfers 469 which combine multiple deposits to the same merchant. 470 * ``taler-exchange-closer.service``: service that watches for reserves that have been abandoned and schedules wire transfers to send the money back to the originator. 471 * ``taler-exchange-httpd.service``: main Taler exchange logic with the public REST API. 472 * ``taler-exchange-httpd.socket``: systemd socket activation for the Taler exchange HTTP daemon. 473 * ``taler-exchange-secmod-eddsa.service``: software security module for making EdDSA signatures. 474 * ``taler-exchange-secmod-rsa.service``: software security module for making RSA signatures. 475 * ``taler-exchange-secmod-cs.service``: software security module for making CS signatures. 476 * ``taler-exchange-transfer.service``: service that triggers outgoing wire transfers (pays merchants). 477 * ``taler-exchange-wirewatch.service``: service that watches for incoming wire transfers (first step of withdraw). 478 * ``taler-exchange.target``: Main target for the Taler exchange to be operational. 479 480 481 The deployment creates the following key locations in the system: 482 483 * ``/etc/taler-exchange/``: configuration files. 484 * ``/run/taler-exchange/``: contains the UNIX domain sockets for inter-process communication (IPC). 485 * ``/var/lib/taler-exchange/``: serves as the $HOME for all Taler exchange users and contains sub-directories 486 with the private keys; which keys are stored here depends on the host: 487 488 * online system: exchange-secmod-eddsa, exchange-secmod-cs and exchange-secmod-rsa keys. 489 * offline system: exchange-offline keys. 490 491 492 Configuration Fundamentals 493 ========================== 494 495 This chapter provides fundamental details about the exchange configuration. 496 497 The configuration for all Taler components uses a single configuration file 498 as entry point: ``/etc/taler-exchange/taler-exchange.conf``. 499 500 System defaults are automatically loaded from files in 501 ``/usr/share/taler-exchange/config.d``. 502 These default files should never be modified. 503 504 The default configuration ``taler-exchange.conf`` configuration file 505 also includes all configuration files in ``/etc/taler-exchange/conf.d``. 506 The settings from files in 507 ``conf.d`` are only relevant to particular components of an exchange, while 508 ``taler-exchange.conf`` contains settings that affect all exchange components. 509 510 511 The directory ``/etc/taler-exchange/secrets`` contains configuration file snippets with 512 values that should only be readable to certain users. They are included with the ``@inline-secret@`` 513 directive and should end with ``.secret.conf``. 514 515 To view the entire configuration annotated with the source of each configuration option, you 516 can use the ``taler-exchange-config`` helper: 517 518 519 .. code-block:: shell-session 520 521 [root@exchange-online]# taler-exchange-config --diagnostics 522 < ... annotated, full configuration ... > 523 524 .. warning:: 525 526 While ``taler-exchange-config`` also supports rewriting configuration files, we strongly 527 recommend to edit configuration files manually, as ``taler-exchange-config`` does not 528 preserve comments and, by default, rewrites ``/etc/taler-exchange/taler-exchange.conf``. 529 530 531 .. include:: frags/configuration-format.rst 532 533 534 Exchange Database Setup 535 ======================= 536 537 The access credentials for the exchange's database are configured in 538 ``/etc/taler-exchange/secrets/exchange-db.secret.conf``. 539 Currently, only PostgreSQL is supported as a database backend. 540 541 The following users must have access to the exchange database: 542 543 * taler-exchange-httpd 544 * taler-exchange-wire 545 * taler-exchange-aggregator 546 * taler-exchange-closer 547 548 These users are all in the taler-exchange-db group, and the 549 ``exchange-db.secret.conf`` should be only readable by users in 550 this group. 551 552 .. note:: 553 554 The **taler-exchange-dbconfig** tool can be used to automate the database 555 setup. When using the Debian/Ubuntu packages, the users should already have 556 been created, so you can just run the tool without any arguments and should 557 have a working database configuration. The rest of this section only 558 explains what the **taler-exchange-dbconfig** shell script fully automates. 559 560 To create a database for the Taler exchange on the local system, run: 561 562 .. code-block:: shell-session 563 564 [root@exchange-online]# su - postgres 565 [postgres@exchange-online]# createuser taler-exchange-httpd 566 [postgres@exchange-online]# createuser taler-exchange-wire 567 [postgres@exchange-online]# createuser taler-exchange-aggregator 568 [postgres@exchange-online]# createuser taler-exchange-closer 569 [postgres@exchange-online]# createdb -O taler-exchange-httpd taler-exchange 570 [postgres@exchange-online]# exit 571 572 This will create a ``taler-exchange`` database owned by the 573 ``taler-exchange-httpd`` user. We will use that user later to perform 574 database maintenance operations. 575 576 577 Assuming the above database setup, the database credentials to configure 578 in the configuration file would simply be: 579 580 .. code-block:: ini 581 :caption: /etc/taler-exchange/secrets/exchange-db.secret.conf 582 583 [exchange] 584 DB = postgres 585 586 [exchangedb-postgres] 587 CONFIG=postgres:///taler-exchange 588 589 590 If the database is run on a different host, please follow the instructions 591 from the PostgreSQL manual for configuring remote access. 592 593 After configuring the database credentials, the exchange database needs 594 to be initialized with the following command: 595 596 .. code-block:: shell-session 597 598 [root@exchange-online]# sudo -u taler-exchange-httpd taler-exchange-dbinit 599 600 .. note:: 601 602 To run this command, the user must have ``CREATE TABLE``, ``CREATE 603 INDEX``, ``ALTER TABLE`` and (in the future possibly even) ``DROP TABLE`` 604 permissions. Those permissions are only required for this step (which may 605 have to be repeated when upgrading a deployment). Afterwards, during 606 normal operation, permissions to ``CREATE`` or ``ALTER`` tables are not 607 required by any of the Taler exchange processes and thus should not be 608 granted. For more information, see 609 :doc:`manpages/taler-exchange-dbinit.1`. 610 611 Finally we need to grant the other accounts limited access: 612 613 .. code-block:: shell-session 614 615 [root@exchange-online]# sudo -u taler-exchange-httpd bash 616 [taler-exchange-httpd@exchange-online]# echo 'GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA exchange TO "taler-exchange-aggregator";' \ 617 | psql taler-exchange 618 [taler-exchange-httpd@exchange-online]# echo 'GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA exchange TO "taler-exchange-closer";' \ 619 | psql taler-exchange 620 [taler-exchange-httpd@exchange-online]# echo 'GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA exchange TO "taler-exchange-wire";' \ 621 | psql taler-exchange 622 [taler-exchange-httpd@exchange-online]# echo 'GRANT USAGE ON ALL SEQUENCES IN SCHEMA exchange TO "taler-exchange-aggregator";' \ 623 | psql taler-exchange 624 [taler-exchange-httpd@exchange-online]# echo 'GRANT USAGE ON ALL SEQUENCES IN SCHEMA exchange TO "taler-exchange-closer";' \ 625 | psql taler-exchange 626 [taler-exchange-httpd@exchange-online]# echo 'GRANT USAGE ON ALL SEQUENCES IN SCHEMA exchange TO "taler-exchange-wire";' \ 627 | psql taler-exchange 628 [taler-exchange-httpd@exchange-online]# exit 629 630 .. note:: 631 632 The above instructions for changing database permissions only work *after* 633 having initialized the database with ``taler-exchange-dbinit``, as 634 the tables need to exist before permissions can be granted on them. The 635 ``taler-exchange-dbinit`` tool cannot setup these permissions, as it 636 does not know which users will be used for which processes. 637 638 .. _Dataprotection: 639 640 Data protection concept 641 ----------------------- 642 643 The ``kyc-attributes`` table contains the most sensitive 644 data, namely the KYC records of the recipients of payments. 645 The data could include passports, business documentation 646 and other sensitive information. At a high-level, it is 647 stored in JSON format (key-value), compressed and then 648 additionally encrypted using a secret key from the 649 exchange configuration (the ``ATTRIBUTE_ENCRYPTION_KEY``). 650 Thus, this data is protected even if the full database 651 is leaked. 652 653 The ``aml-history`` table stores the main information 654 provided by AML officers. This may contain sensitive 655 data about risk assessments on customers, exposed or 656 secret limits imposed on individual accounts, and 657 in general forms supplied by bank staff. AML decisions 658 are also tracked in the ``legitimization-*`` tables, 659 including the latest rules that apply to an account. 660 661 The second most sensitive table is 662 ``wire-targets``, as it contains the IBANs (and possibly 663 other visible information from RFC 8905) of the bank 664 accounts of the customers. Many other tables contain 665 transaction data (incoming, outgoing, p2p transfers, 666 etc) which are somewhat less sensitive. It should be 667 noted that the actual contracts between customers and 668 merchants are not available to the exchange operator; 669 the ``contracts`` table only stores contracts for p2p 670 transfers in an end-to-end encrypted format that the 671 exchange itself can never decrypt. 672 673 The database is periodically cleaned up via the 674 ``taler-exchange-dbinit --gc`` command. It is 675 currently hard-coded (!) to delete KYC/AML data 676 of accounts that have been inactive for 10 years. 677 678 Data on incoming wire transfers (reserves) is 679 deleted after the ``LEGAL_RESERVE_EXPIRATION_TIME`` 680 is reached, as configured in the ``[exchangedb]`` 681 section. The default is set to 7 years. 682 683 Payment data is deleted whenever the legal expiration date for the 684 ``DURATION_LEGAL`` configured in the respective section ``coin`` is reached 685 (typically also after 10 years). 686 687 Because a purse refunds the coins deposited into it when it expires, and 688 the auditor has to be able to account for that refund, the expiration of 689 a purse is bounded by ``PURSE_EXPIRATION_LIMIT`` in the ``[exchange]`` 690 section. It must not exceed the shortest ``DURATION_LEGAL`` of any 691 denomination and is published to clients in ``/keys``; if the option is 692 not given, the exchange uses that shortest ``DURATION_LEGAL`` and logs a 693 warning. Lowering ``DURATION_LEGAL`` by some span X with effect from 694 time T therefore requires lowering ``PURSE_EXPIRATION_LIMIT`` by X at 695 T-X, so that the purses created before the change are short enough for 696 the denominations issued after it; otherwise deposits of the new coins 697 into those older purses are refused. 698 699 700 Basic Setup: Currency, Denominations and Keys 701 ============================================= 702 703 A Taler exchange only supports a single currency. The currency 704 and the smallest currency unit supported by the bank system 705 must be specified in ``/etc/taler-exchange/taler-exchange.conf``. 706 707 .. code-block:: ini 708 :caption: /etc/taler-exchange/taler-exchange.conf 709 710 [exchange] 711 CURRENCY = EUR 712 CURRENCY_ROUND_UNIT = EUR:0.01 713 714 # ... rest of file ... 715 716 .. warning:: 717 718 When editing ``/etc/taler-exchange/taler-exchange.conf``, take care to not accidentally remove 719 the ``@inline-matching@`` directive to include the configuration files in ``conf.d``. 720 721 .. _Coins-denomination-keys: 722 723 Coins (denomination keys) 724 ------------------------- 725 726 Next, the electronic cash denominations that the exchange offers must be 727 specified. 728 729 Sections specifying denomination (coin) information start with ``coin_``. By 730 convention, the name continues with ``$CURRENCY_[$SUBUNIT]_$VALUE_$REVISION``, 731 i.e. ``[coin_eur_ct_10_0]`` for a 10 cent piece. However, only the ``coin_`` 732 prefix is mandatory. Once configured, these configuration values must not 733 change. The ``$REVISION`` part of the section name should be incremented if 734 any of the coin attributes in the section changes. Each ``coin_``-section 735 must then have the following options: 736 737 - ``VALUE``: How much is the coin worth, the format is 738 CURRENCY:VALUE.FRACTION. For example, a 10 cent piece is "EUR:0.10". 739 740 - ``DURATION_WITHDRAW``: How long can a coin of this type be withdrawn? 741 This limits the losses incurred by the exchange when a denomination 742 key is compromised. 743 744 - ``DURATION_SPEND``: How long is a coin of the given type valid? Smaller 745 values result in lower storage costs for the exchange. 746 747 - ``DURATION_LEGAL``: How long is the coin of the given type legal? 748 ``DURATION_WITHDRAW`` plus ``DURATION_SPEND`` plus ``DURATION_LEGAL`` 749 must not exceed ``LEGAL_RESERVE_EXPIRATION_TIME`` of the 750 ``[exchangedb]`` section, as otherwise the garbage collection could 751 drop the denomination before the auditor is done accounting for it. 752 ``DURATION_LEGAL`` must also be at least ``PURSE_EXPIRATION_LIMIT`` 753 of the ``[exchange]`` section. The exchange refuses to start if 754 either is violated. 755 756 - ``FEE_WITHDRAW``: What does it cost to withdraw this coin? Specified 757 using the same format as value. 758 759 - ``FEE_DEPOSIT``: What does it cost to deposit this coin? Specified using 760 the same format as value. 761 762 - ``FEE_REFRESH``: What does it cost to refresh this coin? Specified using 763 the same format as value. 764 765 - ``FEE_REFUND``: What does it cost to refund this coin? 766 Specified using the same format as value. 767 768 - ``CIPHER``: Which cipher to use for this coin? Must be either ``RSA`` or 769 ``CS``. 770 771 - ``RSA_KEYSIZE``: How many bits should the RSA modulus (product of the two 772 primes) have for this type of coin. 773 774 - ``AGE_RESTRICTED``: Set to ``YES`` to make this a denomination with support 775 for age restrictions. See the age-restriction documentation below for 776 details. Age restriction is a protocol feature; it no longer uses the 777 retired generic extension mechanism described by DD06. 778 This option is optional and defaults to ``NO``. 779 780 See :doc:`manpages/taler-exchange.conf.5` for information on *duration* values 781 (i.e. ``DURATION_WITHDRAW`` and ``DURATION_SPEND`` above, 782 and ``OVERLAP_DURATION`` and ``DURATION`` below). 783 Additionally, there are two global configuration options of note: 784 785 - ``[taler-exchange-secmod-rsa/OVERLAP_DURATION]``: What is the overlap of the 786 withdrawal timespan for denomination keys? The value given here must 787 be smaller than any of the ``DURATION_WITHDRAW`` values for any of the coins. 788 789 - ``[taler-exchange-secmod-rsa/LOOKAHEAD_SIGN]``: For how far into the future 790 should denomination keys be pre-generated? This allows the exchange and 791 auditor operators to download, offline-sign, and upload denomination key 792 signatures for denomination keys that will be used in the future by the 793 exchange. 794 795 .. index:: maintenance 796 .. note:: 797 We recommend setting the ``LOOKAHEAD_SIGN`` value to at least one year and 798 then to perform the offline-signing procedure at least once every 6 months 799 to ensure that there is sufficient time for wallets to learn the new keys 800 and to avoid unavailability in case this critical maintenance procedure is 801 delayed. 802 803 .. note:: 804 It is crucial that the configuration provided in these sections is identical (!) 805 for the exchange and the crypto helpers. We recommend pointing both users 806 to the same configuration file! 807 808 The ``taler-harness`` has a helper command that generates a 809 reasonable denomination structure. 810 811 .. code-block:: shell-session 812 813 [root@exchange-online]# taler-harness deployment gen-coin-config \ 814 --min-amount EUR:0.01 \ 815 --max-amount EUR:100 \ 816 > /etc/taler-exchange/conf.d/exchange-coins.conf 817 818 You can manually review and edit the generated configuration file. The main 819 change that is possibly required is updating the various fees. Note that you 820 MUST NOT edit a coin configuration section after the initial setup. If you 821 must ``change`` the values, you must instead create a new section with a 822 different unique name (still with the ``coin_`` prefix) and comment out or 823 remove the existing section. Do take care to not introduce the name of the 824 disabled section again in the future. 825 826 827 .. _Sign-keys: 828 829 Sign keys 830 --------- 831 832 There are three global configuration options of note for sign keys: 833 834 - ``[taler-exchange-secmod-eddsa/DURATION]``: How long are sign keys 835 used to sign messages? After this time interval expires, a fresh 836 sign key will be used (key rotation). We recommend using 837 a ``DURATION`` of a few weeks to a few months for sign keys. 838 839 - ``[taler-exchange-secmod-eddsa/OVERLAP_DURATION]``: What is the overlap of the 840 timespan for sign keys? We recommend a few minutes or hours. Must 841 be smaller than ``DURATION``. 842 843 - ``[taler-exchange-secmod-eddsa/LOOKAHEAD_SIGN]``: For how far into the future 844 should sign keys be pre-generated? This allows the exchange and 845 auditor operators to download, offline-sign, and upload sign key 846 signatures for sign keys that will be used in the future by the exchange. 847 848 .. note:: 849 We recommend setting the ``LOOKAHEAD_SIGN`` value to at least one year and 850 then to perform the offline-signing procedure at least once every 6 months 851 to ensure that there is sufficient time for wallets to learn the new keys 852 and to avoid unavailability in case this critical maintenance procedure is 853 delayed. 854 855 856 .. _OfflineConfiguration: 857 858 Setting up the offline signing key 859 ---------------------------------- 860 861 Before launching an exchange, the offline signing (master) key must be 862 generated and set in the configuration. The offline signing keys of the 863 exchange should be stored on a different machine. The responsibilities of 864 this offline signing machine are: 865 866 * Generation of the exchange's offline master signing key. 867 * Secure storage of the exchange's offline master signing key. 868 * Generation of certificates (signed with the offline master signing key) that will be imported by the exchange. 869 * Revocation of keys when the online system was compromised or is being terminated 870 871 872 Configuration file options related to the master key are: 873 874 - ``[exchange-offline/MASTER_PRIV_FILE]``: Path to the exchange’s master 875 private file. Only needs to be provided on the offline system where the 876 ``taler-exchange-offline`` command is used. The default value is usually 877 fine and does not require adjustment. 878 879 - ``[exchange/MASTER_PUBLIC_KEY]``: Must specify the exchange’s master public 880 key. Needed for the exchange to verify information signed by the offline 881 system. This value must almost always be set explicitly by hand. 882 883 884 885 .. code-block:: shell-session 886 887 [root@exchange-offline]# taler-exchange-offline setup 888 < ... prints the exchange master public key > 889 890 The public key printed as the output of this command must be put into the 891 configuration of the online machine: 892 893 .. code-block:: ini 894 :caption: /etc/taler-exchange/conf.d/exchange-business.conf 895 896 [exchange] 897 MASTER_PUBLIC_KEY = YE6Q6TR1ED... 898 899 # ... rest of file ... 900 901 902 Wire Gateway Setup 903 ================== 904 905 The :ref:`Taler Wire Gateway <taler-wire-gateway-http-api>` is an API that 906 connects the Taler exchange to the underlying core banking system. There are 907 several implementations of wire gateways: 908 909 * `Project deploymerization <https://git.taler.net/taler/depolymerization/files.html>`_ implements a wire gateway on top of Bitcoin or Ethereum 910 * The :ref:`libeufin-bank <libeufin-bank>` provides a wire gateway interface on top of a regional currency bank 911 * The **taler-fakebank-run** command is an in-memory bank simulator with a wire gateway interface for testing 912 913 .. FIXME :ref:`libeufin-nexus <libeufin-nexus>` is an implementation of the Wire Gateway API for the EBICS protocol. Add to list above once nexus implements the TWG directly! 914 915 Before continuing, you need to decide which wire gateway you want to use, 916 and install and configure it on your system. Afterwards, you need to 917 have two key pieces of information from that setup: 918 919 * The username and password to access the exchange's account in the system. 920 * The ``payto://`` URI of that account (see `RFC 8905 <https://www.rfc-editor.org/rfc/rfc8905>`_). 921 922 If you use the libeufin-bank, you need to set a special configuration option 923 because the taler-exchange-wire-gateway-client does not yet support token 924 authentication: 925 926 .. code-block:: ini 927 :caption: /etc/libeufin/libeufin-bank.conf 928 929 [libeufin-bank] 930 PWD_AUTH_COMPAT = YES 931 932 .. FIXME: Remove the text above once #9703 is fixed. 933 934 .. _exchange-bank-account-configuration: 935 936 Exchange Bank Account Configuration 937 ----------------------------------- 938 939 An exchange must be configured with the right settings to access its bank 940 account via a :ref:`Taler wire gateway <taler-wire-gateway-http-api>`. An 941 exchange can be configured to use multiple bank accounts by using multiple 942 wire gateways. Typically only one wire gateway is used. 943 944 To configure a bank account in Taler, we need to furnish two pieces of 945 information: 946 947 - The ``payto://`` URI of the bank account, which uniquely identifies the 948 account. Examples for such URIs include 949 ``payto://iban/CH9300762011623852957`` for a bank account with 950 an IBAN or 951 ``payto://x-taler-bank/localhost:8080/2`` for the 2nd bank account at 952 the Taler bank demonstrator running at ``localhost`` on port 8080. 953 The first part of the URI following ``payto://`` (``iban`` or 954 ``x-taler-bank``) is called the wire method. 955 956 - The ``taler-exchange-wirewatch`` and ``taler-exchange-transfer`` 957 tools needs to be provided resources for authentication 958 to the respective banking service. The format in which the 959 authentication information is currently a username and password 960 for HTTP basic authentication. 961 962 963 Each Taler wire gateway is configured in a configuration section that follows 964 the pattern ``exchange-account-$id``, where ``$id`` is an internal identifier 965 for the bank account accessed by the exchange. The basic information for an 966 account should be put in ``/etc/taler-exchange/conf.d/exchange-business.conf``. The 967 secret credentials to access the Taler Wire Gateway API should be put into a 968 corresponding ``exchange-accountcredentials-$id`` section in 969 ``/etc/taler-exchange/secrets/exchange-accountcredentials.conf``. The latter file 970 should be only readable for the ``taler-exchange-wire`` user. Only the 971 ``taler-exchange-wirewatch`` and ``taler-exchange-transfer`` services should 972 run as the ``taler-exchange-wire`` user. Other exchange processes do not need 973 to have access to the account credentials. 974 975 You can configure multiple accounts for an exchange by creating sections 976 starting with ``exchange-account-`` for the section name. You must specify 977 ``ENABLE_``-settings for each account whether it should be used, and for what 978 (incoming or outgoing wire transfers): 979 980 .. code-block:: ini 981 :caption: /etc/taler-exchange/conf.d/exchange-business.conf 982 983 [exchange-account-1] 984 # Account identifier in the form of an RFC-8905 payto:// URI. 985 # For SEPA, looks like payto://iban/$IBAN?receiver-name=$NAME 986 # Make sure to URL-encode spaces in $NAME! 987 # 988 # With x-taler-bank (for Fakebank) 989 # PAYTO_URI = "payto://x-taler-bank/bank.demo.taler.net/Exchange?receiver-name=exop" 990 # 991 # Example using IBAN (for use with LibEuFin) 992 PAYTO_URI = "payto://iban/CH9300762011623852957?receiver=name=exop" 993 994 # Use for exchange-aggregator (outgoing transfers) 995 ENABLE_DEBIT = YES 996 # Use for exchange-wirewatch (and listed in /wire) 997 ENABLE_CREDIT = YES 998 999 @inline-secret@ exchange-accountcredentials-1 ../secrets/exchange-accountcredentials.secret.conf 1000 1001 1002 .. code-block:: ini 1003 :caption: /etc/taler-exchange/secrets/exchange-accountcredentials.secret.conf 1004 1005 [exchange-accountcredentials-1] 1006 1007 # LibEuFin expects basic auth. 1008 WIRE_GATEWAY_AUTH_METHOD = basic 1009 1010 # Username and password to access the Taler wire gateway. 1011 USERNAME = ... 1012 PASSWORD = ... 1013 1014 # URL for talking to the bank wire the wire API. 1015 WIRE_GATEWAY_URL = https://bank.demo.taler.net/accounts/exchange/taler-wire-gateway/ 1016 1017 1018 Such a wire gateway configuration can be tested with the following commands: 1019 1020 .. code-block:: shell-session 1021 1022 [root@exchange-online]# taler-exchange-wire-gateway-client \ 1023 --section exchange-accountcredentials-1 --debit-history 1024 [root@exchange-online]# taler-exchange-wire-gateway-client \ 1025 --section exchange-accountcredentials-1 --credit-history 1026 1027 On success, you will see some of your account's transaction history (or an 1028 empty history), while on failure you should see an error message. 1029 1030 1031 .. _Deployment: 1032 1033 Deployment 1034 ========== 1035 1036 This chapter describes how to deploy the exchange once the basic installation 1037 and configuration are completed. 1038 1039 .. _Serving: 1040 1041 Serving 1042 ------- 1043 1044 The exchange can serve HTTP over both TCP and UNIX domain socket. 1045 1046 The following options are to be configured in the section ``[exchange]``: 1047 1048 - ``SERVE``: Must be set to ``tcp`` to serve HTTP over TCP, or ``unix`` to serve 1049 HTTP over a UNIX domain socket. 1050 1051 - ``PORT``: Set to the TCP port to listen on if ``SERVE`` is ``tcp``. 1052 1053 - ``UNIXPATH``: Set to the UNIX domain socket path to listen on if ``SERVE`` is 1054 ``unix``. 1055 1056 - ``UNIXPATH_MODE``: Number giving the mode with the access permission mask 1057 for the ``UNIXPATH`` (i.e. 660 = ``rw-rw---``). Make sure to set it in such 1058 a way that your reverse proxy has permissions to access the UNIX domain 1059 socket. The default (660) assumes that the reverse proxy is a member of 1060 the group under which the exchange HTTP server is running. 1061 1062 .. _ReverseProxy: 1063 1064 Reverse Proxy Setup 1065 ------------------- 1066 1067 By default, the ``taler-exchange-httpd`` service listens for HTTP connections 1068 on a UNIX domain socket. To make the service publicly available, a reverse 1069 proxy such as nginx should be used. We strongly recommend to configure nginx 1070 to use TLS. 1071 1072 The public URL that the exchange will be served under should 1073 be put in ``/etc/taler-exchange/conf.d/exchange-business.conf`` configuration file. 1074 1075 .. code-block:: ini 1076 :caption: /etc/taler-exchange/conf.d/exchange-business.conf 1077 1078 [exchange] 1079 BASE_URL = https://example.com/ 1080 1081 # ... rest of file ... 1082 1083 The ``taler-exchange`` package ships with a sample configuration that can be 1084 enabled in nginx: 1085 1086 .. code-block:: shell-session 1087 1088 [root@exchange-online]# vim /etc/nginx/sites-available/taler-exchange 1089 < ... customize configuration ... > 1090 [root@exchange-online]# ln -s /etc/nginx/sites-available/taler-exchange \ 1091 /etc/nginx/sites-enabled/taler-exchange 1092 [root@exchange-online]# systemctl reload nginx 1093 1094 Note that the reverse proxy must set a HTTP ``X-Forwarded-Host`` header to 1095 refer to the hostname used by nginx and a HTTP ``X-Forwarded-Proto`` header to 1096 inform the exchange whether the external protocol was ``http`` or ``https``. 1097 Thus, depending on your setup, you will likely have to edit those parts of the 1098 provided ``taler-exchange`` configuration file. 1099 1100 With this last step, we are finally ready to launch the 1101 main exchange process. 1102 1103 .. _Launch: 1104 1105 Launching an exchange 1106 --------------------- 1107 1108 A running exchange requires starting the following processes: 1109 1110 - ``taler-exchange-secmod-rsa`` (as special user, sharing group with the HTTPD) 1111 - ``taler-exchange-secmod-cs`` (as special user, sharing group with the HTTPD) 1112 - ``taler-exchange-secmod-eddsa`` (as special user, sharing group with the HTTPD) 1113 - ``taler-exchange-httpd`` (needs database access) 1114 - ``taler-exchange-aggregator`` (only needs database access) 1115 - ``taler-exchange-closer`` (only needs database access) 1116 - ``taler-exchange-wirewatch`` (needs bank account read credentials and database access) 1117 - ``taler-exchange-transfer`` (needs credentials to initiate outgoing wire transfers and database access) 1118 1119 The crypto helpers (``secmod``) must be started before the ``taler-exchange-httpd`` and 1120 they should use the same configuration file. 1121 1122 For the most secure deployment, we recommend using separate users for each of 1123 these processes to minimize information disclosures should any of them be 1124 compromised. The helpers do not need access to the PostgreSQL database (and 1125 thus also should not have it). 1126 1127 The processes that require access to the bank account need to have a 1128 configuration file with the respective credentials in it. We recommend using a 1129 separate configuration at least for ``taler-exchange-transfer`` which is the 1130 *only* process that needs to know the credentials to execute outgoing wire 1131 transfers. 1132 1133 All of these processes should also be started via a hypervisor like 1134 ``systemd`` or ``gnunet-arm`` that automatically re-starts them should they 1135 have terminated unexpectedly. If the bank is down (say for maintenance), it is 1136 *possible* to halt the ``taler-exchange-wirewatch`` and/or 1137 ``taler-exchange-transfer`` processes (to avoid them making requests to the 1138 bank API that can only fail) without impacting other operations of the 1139 exchange. Naturally, incoming wire transfers will only be observed once 1140 ``taler-exchange-wirewatch`` is resumed, and merchants may complain if the 1141 disabled ``taler-exchange-transfer`` process causes payment deadlines to be 1142 missed. 1143 1144 .. note:: 1145 The ``taler-exchange-httpd`` does not ship with HTTPS enabled by default. 1146 In production, it should be run behind an HTTPS reverse proxy that performs 1147 TLS termination on the same system. Thus, it would typically be configured 1148 to listen on a UNIX domain socket. The ``/management`` and ``/auditors`` 1149 APIs do technically not have to be exposed on the Internet (only to the 1150 administrators running ``taler-exchange-offline``) and should be blocked 1151 by the reverse proxy for requests originating from outside of the bank. 1152 (However, this is not a strong security assumption: in principle having 1153 these endpoints available should do no harm. However, it increases the 1154 attack surface.) 1155 1156 1157 Given proper packaging, all of the above are realized via a simple systemd 1158 target. This enables the various processes of an exchange service to be 1159 started using a simple command: 1160 1161 .. code-block:: shell-session 1162 1163 [root@exchange-online]# systemctl start taler-exchange.target 1164 1165 .. note:: 1166 1167 At this point, the exchange service is not yet fully operational. 1168 1169 1170 To check whether the exchange is running correctly under the advertised 1171 base URL, run: 1172 1173 .. code-block:: shell-session 1174 1175 [root@exchange-online]# export BASE_URL=$(taler-exchange-config -s exchange -o base_url) 1176 [root@exchange-online]# wget ${BASE_URL}management/keys 1177 1178 The request might take some time to complete on slow machines, because 1179 a lot of key material will be generated. 1180 1181 1182 Offline Signing Setup, Key Maintenance and Tear-Down 1183 ==================================================== 1184 1185 The exchange HTTP service must be running before you can complete the 1186 following offline signing procedure. Note that when an exchange is running 1187 without offline keys its not fully operational. To make the exchange HTTP 1188 service fully operational, the following steps involving the offline signing 1189 machine must be completed: 1190 1191 1. The public keys of various online keys used by the exchange service are exported 1192 via a management HTTP API. 1193 2. The offline signing system validates this request and signs it. 1194 Additionally, the offline signing system signs policy messages 1195 to configure the exchange's bank accounts and associated fees. 1196 3. The messages generated by the offline signing system are uploaded 1197 via the management API of the exchange HTTP service. 1198 1199 A typical minimal setup would look something like this: 1200 1201 .. code-block:: shell-session 1202 1203 [anybody@exchange-online]# taler-exchange-offline \ 1204 download > sig-request.json 1205 1206 [root@exchange-offline]# taler-exchange-offline \ 1207 sign < sig-request.json > sig-response.json 1208 [root@exchange-offline]# taler-exchange-offline \ 1209 enable-account payto://iban/$IBAN?receiver-name=$NAME > acct-response.json 1210 [root@exchange-offline]# taler-exchange-offline \ 1211 wire-fee now iban EUR:0 EUR:0 > fee-response.json 1212 [root@exchange-offline]# taler-exchange-offline \ 1213 global-fee now EUR:0 EUR:0 EUR:0 4weeks 6a 4 > global-response.json 1214 1215 [anybody@exchange-online]# taler-exchange-offline upload < sig-response.json 1216 [anybody@exchange-online]# taler-exchange-offline upload < acct-response.json 1217 [anybody@exchange-online]# taler-exchange-offline upload < fee-response.json 1218 [anybody@exchange-online]# taler-exchange-offline upload < global-response.json 1219 1220 The following sections will discuss these steps in more depth. 1221 1222 .. _Keys-generation: 1223 1224 Signing the online signing keys 1225 ------------------------------- 1226 1227 To sign the online signing keys, first the *future* key material should be downloaded using: 1228 1229 .. code-block:: console 1230 1231 $ taler-exchange-offline download > future-keys.json 1232 1233 Afterwards, *future-keys.json* contains data about denomination and 1234 online signing keys that the exchange operator needs to sign with 1235 the offline tool. The file should be copied to the offline system. 1236 There, the operator should run: 1237 1238 .. code-block:: console 1239 1240 $ taler-exchange-offline show < future-keys.json 1241 1242 and verify that the output contains the fee structure and key lifetimes 1243 they expect to see. They should also note the public keys being shown 1244 and communicate those to the *auditors* over a secure channel. Once 1245 they are convinced the file is acceptable, they should run: 1246 1247 .. code-block:: console 1248 1249 $ taler-exchange-offline sign < future-keys.json > offline-sigs.json 1250 1251 The *offline-sigs.json* file must then be copied to an online system 1252 that is able to again communicate with the exchange. On that system, run: 1253 1254 .. code-block:: console 1255 1256 $ taler-exchange-offline upload < offline-sigs.json 1257 1258 to provision the signatures to the exchange. 1259 1260 The ``download sign upload`` sequence in the commands above has to be done 1261 periodically, as it signs the various online signing keys of the exchange 1262 which periodically expire. 1263 1264 1265 .. _exchange-account-signing: 1266 1267 Account signing 1268 --------------- 1269 1270 The ``enable-account`` step is important and must be used to sign the 1271 ``payto://`` URI in a way suitable to convince wallets that this is the 1272 correct address to wire funds to. Note that for each bank account, additional 1273 options **must** be set in the configuration file to tell the exchange how to 1274 access the bank account. The offline tool *only* configures the externally 1275 visible portions of the setup. The chapter on 1276 :ref:`bank account configuration <exchange-bank-account-configuration>` has 1277 further details. 1278 1279 taler-exchange-offline accepts additional options to configure the use of the 1280 account. For example, additional options can be used to add currency 1281 conversion or to restrict interactions to bank accounts from certain 1282 countries: 1283 1284 .. code-block:: console 1285 1286 $ taler-exchange-offline \ 1287 enable-account payto://iban/CH9300762011623852957 1288 conversion-url https://conversion.example.com/ 1289 1290 For details on optional ``enable-account`` arguments, 1291 see :doc:`manpages/taler-exchange-offline.1`. 1292 1293 .. _Wire-fee-structure: 1294 1295 Wire fee structure 1296 ------------------ 1297 1298 .. index:: wire fee 1299 .. index:: fee 1300 1301 For each wire method (“iban” or “x-taler-bank”) the 1302 exchange must know about applicable wire fees. This is also done 1303 using the ``taler-exchange-offline`` tool: 1304 1305 .. code-block:: console 1306 1307 $ taler-exchange-offline wire-fee 2040 iban EUR:0.05 EUR:0.10 1308 1309 The above sets the wire fees for wire transfers involving ``iban`` accounts 1310 (in Euros) in the year 2040 to 5 cents (wire fee) and 10 cents (closing fee). 1311 The tool only supports setting fees that apply for the entire calendar year. 1312 1313 We recommend provisioning an exchange with wire fees at least for the next two 1314 years. Note that once the fees have been set for a year, they cannot be 1315 changed (basically, by signing the fees the exchange makes a legally binding 1316 offer to the customers). 1317 1318 .. index:: maintenance 1319 .. note:: 1320 Provisioning future wire fees, like provisioning future denomination 1321 and signing keys, are key regular maintenance procedures for every 1322 exchange operator. We recommend setting automated reminders for 1323 this maintenance activity! 1324 1325 1326 .. _Auditor-configuration: 1327 1328 Auditor configuration 1329 --------------------- 1330 1331 At this point, the exchange will be able to use those keys, but wallets and 1332 merchants may not yet trust them! Thus, the next step is for an auditor to 1333 affirm that they are auditing this exchange. Before an auditor can do this, 1334 the exchange service must be informed about any auditor that is expected to 1335 provision it with auditor signatures. 1336 1337 This is also done using the ``taler-exchange-offline`` tool on the offline 1338 system. First, the auditor must be configured and provide the exchange 1339 operator with its public key (using ``taler-auditor-offline setup``) and the 1340 URL of its REST API. The exchange operator also needs a human-readable name 1341 that may be shown to users to identify the auditor. For more information on 1342 how to setup and operate an auditor, see 1343 :doc:`manpages/taler-auditor-offline.1` and :doc:`taler-auditor-manual`. 1344 1345 Given this information, the exchange operator can enable the auditor: 1346 1347 .. code-block:: console 1348 1349 $ taler-exchange-offline enable-auditor $PUB_KEY $REST_URL "$AUDITOR_NAME" > auditor.json 1350 1351 As before, the *auditor.json* file must then be copied from the offline system 1352 to a system connected to the exchange and there ``uploaded`` to the exchange using ``taler-exchange-offline upload``. 1353 1354 1355 .. _Revocations: 1356 1357 Revocations 1358 ----------- 1359 1360 When an exchange goes out of business or detects that the private key of 1361 a denomination key pair has been compromised, it may revoke some or all 1362 of its denomination keys. At this point, the hashes of the revoked keys 1363 must be returned as part of the ``/keys`` response under “recoup”. 1364 Wallets detect this, and then return unspent coins of the respective 1365 denomination key using the ``/recoup`` API. 1366 1367 To revoke a denomination key, you need to know the hash of the denomination 1368 public key, ``$HDP``. The ``$HDP`` value is usually included in the security 1369 report that is generated when a compromise is detected). Given this 1370 value, the key revocation can be approved on the offline system: 1371 1372 .. code-block:: console 1373 1374 $ taler-exchange-offline revoke-denominatin $HDP > revocation.json 1375 1376 The resulting *revocation.json* must be copied to a system connected to the 1377 exchange and uploaded to the exchange using the ``upload`` subcommand 1378 of ``taler-exchange-offline``. 1379 1380 .. note:: 1381 Denomination key revocations should only happen 1382 under highly unusual (“emergency”) conditions and not in normal 1383 operation. 1384 1385 1386 Setup Linting 1387 ============= 1388 1389 The ``taler-harness`` package comes with a tool that runs various 1390 checks on the current GNU Taler exchange deployment: 1391 1392 .. code-block:: shell-session 1393 1394 [root@exchange-online]# apt install taler-harness 1395 [root@exchange-online]# taler-harness deployment lint-exchange 1396 1397 You can optionally pass the ``--debug`` option to get more verbose output, and 1398 ``--continue`` to continue with further checks even though a previous one has 1399 failed. 1400 1401 1402 Testing and Troubleshooting 1403 =========================== 1404 1405 We recommend testing whether an exchange deployment is functional by using the 1406 Taler wallet command line interface. The tool can be used to withdraw and 1407 deposit electronic cash via the exchange without having to deploy and operate a 1408 separate merchant backend and storefront. 1409 1410 The following shell session illustrates how the wallet can be used to withdraw 1411 electronic cash from the exchange and subsequently spend it. For these steps, 1412 a merchant backend is not required, as the wallet acts as a merchant. 1413 1414 .. code-block:: shell-session 1415 1416 # This will now output a payto URI that money needs to be sent to in order to allow withdrawal 1417 # of taler coins. 1418 $ taler-wallet-cli advanced withdraw-manually --exchange $EXCHANGE_URL --amount EUR:10.50 1419 1420 1421 Show the status of the manual withdrawal operation. 1422 1423 .. code-block:: shell-session 1424 1425 $ taler-wallet-cli transactions 1426 1427 At this point, a bank transfer to the exchange's bank account 1428 needs to be made with the correct subject / remittance information 1429 as instructed by the wallet after the first step. With the 1430 above configuration, it should take about 5 minutes after the 1431 wire transfer for the incoming transfer to be observed by the 1432 Nexus. 1433 1434 Run the following command to check whether the exchange received 1435 an incoming bank transfer: 1436 1437 .. code-block:: shell-session 1438 1439 [root@exchange-online]# taler-exchange-wire-gateway-client \ 1440 --section exchange-accountcredentials-1 --credit-history 1441 1442 Once the transfer has been made, try completing the withdrawal 1443 using: 1444 1445 .. code-block:: shell-session 1446 1447 $ taler-wallet-cli run-pending 1448 1449 Afterwards, check the status of transactions and show the 1450 current wallet balance: 1451 1452 .. code-block:: shell-session 1453 1454 $ taler-wallet-cli transactions 1455 $ taler-wallet-cli balance 1456 1457 1458 Now, we can directly deposit coins via the exchange into a target 1459 account. (Usually, a payment is made via a merchant. The wallet 1460 provides this functionality for testing.) 1461 1462 .. code-block:: shell-session 1463 1464 $ taler-wallet-cli deposit create EUR:5 \ 1465 payto://iban/$IBAN?receiver-name=Name 1466 $ taler-wallet-cli run-pending 1467 1468 1469 Check if this transaction was successful (from the perspective 1470 of the wallet): 1471 1472 .. code-block:: shell-session 1473 1474 $ taler-wallet-cli transactions 1475 1476 If the transaction failed, fix any open issue(s) with the exchange and 1477 run the "run-pending" command. 1478 1479 The wallet can also track if the exchange wired the money to the merchant 1480 account. The "deposit group id" can be found in the output of the 1481 transactions list. 1482 1483 .. code-block:: shell-session 1484 1485 $ taler-wallet-cli deposit track $DEPOSIT_GROUP_ID 1486 1487 You can also check using the exchange-tools whether the exchange sent 1488 an outgoing transfer: 1489 1490 .. code-block:: shell-session 1491 1492 [root@exchange-online]# taler-exchange-wire-gateway-client \ 1493 --section exchange-accountcredentials-1 --debit-history 1494 1495 After enough time has passed, the money should arrive at the specified IBAN. 1496 1497 For more information on the taler-wallet-cli tool, see 1498 :doc:`manpages/taler-wallet-cli.1`. 1499 1500 taler-exchange-config 1501 --------------------- 1502 1503 .. _Using_taler_exchange_config: 1504 1505 .. index:: taler-exchange-config 1506 1507 .. include:: frags/using-taler-config.rst 1508 1509 1510 Private key storage 1511 ------------------- 1512 1513 Keeping the private keys the helpers create secret is paramount. If the 1514 private keys are lost, it is easy to provision fresh keys (with the help of 1515 the auditor). Thus, we recommend that the private keys of the crypto helpers 1516 are *not* backed up: in the rare event of a disk failure, they can be 1517 regenerated. However, we do recommend using RAID (1+1 or 1+1+1) for all 1518 disks of the system. 1519 1520 1521 .. _Internal-audit: 1522 1523 Internal audits 1524 --------------- 1525 1526 While an exchange should use an external auditor to attest to regulators that 1527 it is operating correctly, an exchange operator can also use the auditor's 1528 logic to perform internal checks. For this, an exchange operator can generally 1529 follow the auditor guide. However, instead of using ``taler-auditor-sync``, 1530 an internal audit can and likely should be performed either directly against 1531 the production exchange database or against a synchronous copy created using 1532 standard database replication techniques. After all, the exchange operator 1533 runs this for diagnostics and can generally trust its own database to maintain 1534 the database invariants. 1535 1536 Running the auditor against the original production database (without 1537 using ``taler-auditor-sync``) enables the auditing logic to perform a few 1538 additional checks that can detect inconsistencies. These checks are enabled 1539 by passing the **-i** option to the ``taler-auditor`` command. As always, 1540 the resulting report should be read carefully to see if there are any problems 1541 with the setup. 1542 1543 Reports are generally created incrementally, with ``taler-auditor`` reporting 1544 only incidents and balance changes that were not covered in previous reports. 1545 While it is possible to reset the auditor database and to restart the audit 1546 from the very beginning, this is generally not recommended as this may be too 1547 expensive. 1548 1549 1550 .. _Database-Scheme: 1551 1552 Database Scheme 1553 --------------- 1554 1555 The exchange database must be initialized using ``taler-exchange-dbinit``. 1556 This tool creates the tables required by the Taler exchange to operate. 1557 The tool also allows you to reset the Taler exchange database, which is 1558 useful for test cases but should never be used in production. Finally, 1559 ``taler-exchange-dbinit`` has a function to garbage collect a database, 1560 allowing administrators to purge records that are no longer required. 1561 Garbage collection (``taler-exchange-dbinit -g``) requires a current schema; 1562 it never applies schema upgrades or reloads stored procedures. Stop the 1563 Exchange services and apply any pending migration explicitly before running 1564 GC. This also applies to the scheduled Debian GC job. An explicit ``--reset`` 1565 is destructive and retains its schema-initialization behavior even with ``-g``. 1566 1567 The database scheme used by the exchange looks as follows: 1568 1569 .. image:: images/exchange-db.png 1570 1571 The ``jmeasures`` JSON in the ``legitimization_measures`` 1572 table is of type `LegitimizationMeasures`: 1573 1574 .. ts:def:: LegitimizationMeasures 1575 1576 interface LegitimizationMeasures { 1577 1578 // Array of legitimization measures that 1579 // are to be applied. 1580 measures: MeasureInformation[]; 1581 1582 // True if the client is expected to eventually satisfy all requirements. 1583 // Default (if missing) is false. 1584 is_and_combinator?: boolean; 1585 1586 // True if the requested operation is categorically forbidden. 1587 // The measures array will be empty in this case. 1588 verboten: boolean; 1589 } 1590 1591 1592 .. _Database-upgrades: 1593 1594 Database upgrades 1595 ----------------- 1596 1597 Before installing a new exchange version, you should probably make a backup of 1598 the existing database and study the release notes on migration. In general, 1599 the way to migrate is to stop all existing Taler exchange processes and run: 1600 1601 .. code-block:: console 1602 1603 $ taler-exchange-dbinit 1604 1605 This will migrate the existing schema to the new schema. You also may need 1606 to grant Taler exchange processes the rights to the new tables (see last 1607 step of database setup). 1608 1609 .. note:: 1610 1611 The **taler-exchange-dbconfig** tool can be used to automate the database 1612 migration. In general, simply invoking it again should trigger the 1613 migration including **taler-exchange-dbinit** and setting the permissions. 1614 1615 1616 If you do not want to keep any data from the previous installation, the 1617 exchange database can be fully re-initialized using: 1618 1619 .. code-block:: console 1620 1621 $ taler-exchange-dbinit --reset 1622 1623 However, running this command will result in all data in the database 1624 being lost, which may result in significant financial liabilities as the 1625 exchange can then not detect double-spending. Hence this operation must 1626 not be performed in a production system. You still also need to then 1627 grant the permissions to the other exchange processes again. 1628 1629 .. _ExchangeBenchmarking: 1630 1631 Benchmarking 1632 ============ 1633 1634 This chapter describes how to run various benchmarks against a Taler exchange. 1635 These benchmark can be used to measure the performance of the exchange by 1636 running a (possibly large) number of simulated clients against one Taler 1637 deployment with a bank, exchange and (optionally) auditor. 1638 1639 Real benchmarks that are intended to demonstrate the scalability of GNU Taler 1640 should not use the tools presented in this section: they may be suitable for 1641 microbenchmarking and tuning, but the setup is inherently not optimized for 1642 performance or realism, both for the load generation and the server side. 1643 Thus, we do not recommend using these performance numbers to assess the 1644 scalability of GNU Taler. That said, the tools can be useful to help identify 1645 performance issues. 1646 1647 The ``taler-unified-setup.sh`` script can be used to launch all required 1648 services and clients. However, the resulting deployment is simplistic 1649 (everything on the local machine, one single-threaded process per service 1650 type) and not optimized for performance at all. However, this can still be 1651 useful to assess the performance impact of changes 1652 to the code or configuration. 1653 1654 The various configuration files used in the code snippets in this section can 1655 be found in the ``src/benchmark/`` directory of the exchange. These are 1656 generally intended as starting points. Note that the configuration files 1657 ending in ``.edited`` are created by ``taler-unified-setup.sh`` and contain 1658 some options that are determined at runtime by the setup logic provided by 1659 ``taler-unified-setup.sh``. 1660 1661 1662 .. _Benchmark-choose-bank: 1663 1664 Choosing a bank 1665 --------------- 1666 1667 For the bank, both a fakebank (``-f``) and libeufin-based (``-ns``) 1668 bank deployment are currently supported by all benchmark tools and 1669 configuration templates. 1670 1671 Fakebank is an ultra-fast in-memory implementation of the Taler bank API. It 1672 is suitable when the goal is to benchmark the core GNU Taler payment system 1673 and to ignore the real-time gross settlement (RTGS) system typically provided 1674 by an existing bank. When using the fakebank, ``taler-unified-setup.sh`` must 1675 be started with the ``-f`` option and be told to use the right exchange bank 1676 account from the configuration files via ``-u exchange-account-1``. 1677 1678 .. code-block:: console 1679 1680 $ dropdb talercheck; createdb talercheck 1681 $ taler-unified-setup.sh -emwt -c $CONF -f -u exchange-account-1 1682 1683 1684 libeufin is GNU Taler's adapter to the core banking system using the EBICS 1685 banking protocol standard. It uses a Postgres database to persist data and is 1686 thus much slower than fakebank. If your GNU Taler deployment uses libeufin in 1687 production, it likely makes sense to benchmark with libeufin. When using the 1688 fakebank, ``taler-unified-setup.sh`` must be started with the ``-ns`` options 1689 (starting libeufin-nexus and libeufin-bank) and be told to use the right 1690 exchange bank account from the configuration files via ``-u 1691 exchange-account-2``. Note that ``taler-unified-setup.sh`` currently cannot 1692 reset a libeufin database, and also will not run if the database is already 1693 initialized. Thus, you must re-create the database every time before 1694 running the command: 1695 1696 .. code-block:: console 1697 1698 $ dropdb talercheck; createdb talercheck 1699 $ taler-unified-setup.sh -emwt -c $CONF -ns -u exchange-account-2 1700 1701 1702 taler-bank-benchmark 1703 -------------------- 1704 1705 This is the simplest benchmarking tool, simulating only the bank 1706 interaction. 1707 1708 .. code-block:: console 1709 1710 $ CONF="benchmark-cs.conf" 1711 $ # or with libeufin 1712 $ dropdb talercheck; createdb talercheck 1713 $ taler-unified-setup.sh -emwt -c "$CONF" -f -u exchange-account-1 1714 $ # Once <<READY>>, in another shell (remember to set $CONF): 1715 $ time taler-bank-benchmark -c "$CONF" -r 40 -p 4 -P4 -u exchange-account-1 -f 1716 $ # or with libeufin 1717 $ dropdb talercheck; createdb talercheck 1718 $ taler-unified-setup.sh -emwt -c "$CONF" -ns -u exchange-account-2 1719 $ # Once <<READY>>, in another shell (remember to set $CONF): 1720 $ time taler-bank-benchmark -c "$CONF" -r 40 -p 1 -P1 -u exchange-account-2 1721 1722 For each *parallel* (``-p``) client, a number of *reserves* (``-r``) is first 1723 established by **transferring** money from a "user" account (42) to the 1724 Exchange's account with the respective reserve public key as wire subject. 1725 Processing is then handled by *parallel* (``-P``) service workers. 1726 1727 1728 taler-exchange-benchmark 1729 ------------------------ 1730 1731 This benchmarking tool simulates a number of clients withdrawing, 1732 depositing and refreshing coins. Operations that are not covered by the 1733 ``taler-exchange-benchmark`` tool today include closing reserves, refunds, 1734 recoups and P2P payments. 1735 1736 .. code-block:: console 1737 1738 $ CONF="benchmark-cs.conf" # -rsa also makes sense 1739 $ # With fakebank 1740 $ dropdb talercheck; createdb talercheck 1741 $ taler-unified-setup.sh -aemwt -c "$CONF" -f -u exchange-account-1 1742 $ # Once <<READY>>, in another shell (remember to set $CONF): 1743 $ taler-exchange-benchmark -c "$CONF".edited -u exchange-account-1 -n 1 -p1 -r 5 -f 1744 $ # 1745 $ # With libeufin 1746 $ dropdb talercheck; createdb talercheck 1747 $ taler-unified-setup.sh -aemwt -c "$CONF" -ns -u exchange-account-2 1748 $ # Once <<READY>>, in another shell (remember to set $CONF): 1749 $ taler-exchange-benchmark -c "$CONF".edited -u exchange-account-2 -L WARNING -n 1 -p1 -r 5 1750 1751 For each *parallel* (``-p``) client, a number of *reserves* (``-r``) is first 1752 established by **transferring** money from a "user" account (42) to the 1753 Exchange's account with the respective reserve public key as wire subject. 1754 Next, the client will **withdraw** a *number of coins* (``-n``) from the 1755 reserve and **deposit** them. Additionally, a *fraction* (``-R``) of the dirty 1756 coins will then be subject to **refreshing**. For some deposits, the auditor 1757 will receive **deposit confirmations**. 1758 1759 The output of ``taler-exchange-benchmark`` will include for each parallel 1760 client the total time spent in each of the major operations, possible 1761 repetitions (i.e. if the operation failed the first time), total execution 1762 time (operating system and user space) and other details. 1763 1764 1765 taler-aggregator-benchmark 1766 -------------------------- 1767 1768 This is another simple benchmark tool that merely prepares an exchange 1769 database to run a stand-alone benchmark of the ``taler-exchange-aggregator`` 1770 tool. After preparing a database and running the tool, you can then 1771 run one or more ``taler-exchange-aggregator`` processes and measure how 1772 quickly they perform the aggregation work. 1773 1774 .. code-block:: console 1775 1776 $ CONF=benchmark-rsa.conf 1777 $ taler-exchange-dbinit -c "$CONF" --reset 1778 $ ./taler-aggregator-benchmark -c "$CONF" -m 500 -r 10 -d 100 1779 $ time taler-exchange-aggregator -c "$CONF" --test 1780 1781 This above commands will first create 100 deposits with 10 refunds into each 1782 of 500 merchant accounts using randomized time stamps. Afterwards, it will 1783 time a single aggregator process in ``--test`` mode (asking it to terminate 1784 as soon as there is no more pending work). 1785 1786 1787 FIXMEs 1788 ====== 1789 1790 * We should have some summary with the inventory of services that should be 1791 running. Systemd by default doesn't show this nicely. Maybe suggest running 1792 "systemd list-dependencies taler-exchange.target"? 1793 * What happens when the TWG doesn't like one particular outgoing transaction? 1794 How to recover from that as a sysadmin when it happens in practice?