taler-developer-manual.rst (65522B)
1 .. 2 This file is part of GNU TALER. 3 4 Copyright (C) 2014-2025 Taler Systems SA 5 6 TALER is free software; you can redistribute it and/or modify it under the 7 terms of the GNU Affero General Public License as published by the Free Software 8 Foundation; either version 2.1, or (at your option) any later version. 9 10 TALER is distributed in the hope that it will be useful, but WITHOUT ANY 11 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR 12 A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. 13 14 You should have received a copy of the GNU Affero General Public License along with 15 TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> 16 17 @author Christian Grothoff 18 19 General Developer Manual 20 ######################## 21 22 .. note:: 23 24 This manual contains information for developers working on GNU Taler 25 and related components. It is not intended for a general audience. 26 27 28 Project Overview 29 ================ 30 31 GNU Taler consists of a large (and growing) number of components 32 in various Git repositories. The following list gives a first 33 overview: 34 35 * exchange: core payment processing logic with a REST API, plus various 36 helper processes for interaction with banks and cryptographic 37 computations. Also includes the logic for the auditor and an 38 in-memory "bank" API implementation for testing. 39 40 * libeufin: implementation of the "bank" API using the EBICS protocol 41 used by banks in the EU. Allows an exchange to interact with 42 European banks. 43 44 * taler-magnet-bank: implementation of the "bank" API using the Magnet Bank 45 API.Allows an exchange to interact with Magnet Bank. 46 47 * taler-cyclos: implementation of the "bank" API using the Cyclos API. Allows an exchange to interact with a Cyclos network. 48 49 * depolymerization: implementation of the "bank" API on top of 50 blockchains, specifically Bitcoin and Ethereum. Allows an exchange 51 to interact with crypto-currencies. 52 53 * merchant: payment processing backend to be run by merchants, 54 offering a REST API. 55 56 * wallet-core: platform-independent implementation of a wallet to be run by 57 normal users. Includes also the WebExtension for various browsers. 58 Furthermore, includes various single-page apps used by other 59 components (especially as libeufin and merchant). Also includes 60 command-line wallet and tools for testing. 61 62 * taler-android: Android Apps including the Android wallet, the 63 Android point-of-sale App and the Android casher app. 64 65 * taler-ios: iOS wallet App. 66 67 * sync: backup service, provides a simple REST API to allow users to 68 make encrypted backups of their wallet state. 69 70 * anastasis: key escrow service, provides a simple REST API to allow 71 users to distribute encryption keys across multiple providers and 72 define authorization policies for key recovery. 73 74 * taler-mdb: integration of Taler with the multi-drop-bus (MDB) API 75 used by vending machines. Allows Taler payments to be integrated 76 with vending machines. 77 78 * gnu-taler-payment-for-woocommerce: payment plugin for the 79 woocommerce (wordpress) E-commerce solution. 80 81 * twister: man-in-the-middle proxy for tests that require fuzzing a 82 REST/JSON protocol. Used for some of our testing. 83 84 * challenger: implementation of an OAuth 2.0 provider that can be used 85 to verify that a user can receive SMS or E-mail at particular addresses. 86 Used as part of KYC processes of the exchange. 87 88 * taler-mailbox: messaging service used to store and forward payment 89 messages to Taler wallets. 90 91 * taldir: directory service used to lookup Taler wallet addresses for 92 sending invoices or payments to other wallets. 93 94 * taler-merchant-demos: various demonstration services operated at 95 'demo.taler.net', including a simple shop and a donation page. 96 97 There are other important repositories without code, including: 98 99 * gana: Hosted on git.gnunet.org, this repository defines various 100 constants used in the GNU Taler project. 101 102 * docs: documentation, including this very document. 103 104 * marketing: various presentations, papers and other resources for 105 outreach. 106 107 * large-media: very large data objects, such as videos. 108 109 * www: the taler.net website. 110 111 Fundamentals 112 ============ 113 114 Versioning 115 ---------- 116 117 A central rule is to never break anything for any dependency. To accomplish 118 this, we use versioning, of the APIs, database schema and the protocol. The 119 database versioning approach is described in the :ref:`Database schema 120 versioning <DatabaseVersioning>` section. Here, we will focus on API and 121 protocol versioning. 122 123 The key issue we need to solve with protocols and APIs (and that does not 124 apply to database versioning) is being able to introduce and remove features 125 without requiring a flag day where all components must update at the same 126 time. For this, we use GNU libtool style versioning with MAJOR:REVISION:AGE 127 and *not* semantic versioning (SEMVER). With GNU libtool style versioning, 128 first the REVISION should be increased on every change to the respective code. 129 Then, each time a feature is introduced or deprecated, the MAJOR and AGE 130 numbers are increased. Whenever an API is actually removed the AGE number is 131 reduced to match the distance since the removed API was deprecated. Thus, if 132 some client implements version X of the protocol (including not using any APIs 133 that have been deprecated), it is compatible for any implementation where 134 MAJOR is larger or equal to X, and MAJOR minus AGE is smaller or equal to X. 135 REVISION is not used for expected compatibility issues and merely serves to 136 uniquely identify each version (in combination with MAJOR). 137 138 To evolve any implementation, it is thus critical to first of all never 139 just break an existing API or endpoint. The only acceptable modifications 140 are to return additional information (being aware of binary compatibility!) 141 or to accept additional optional arguments (again, in a way that does not 142 break existing users). Thus, the most common way to introduce changes will 143 be the addition of new endpoints. Breaking existing endpoints is only ever 144 at best acceptable while in the process of introducing it and if you are 145 absolutely sure that there are zero users in other components. 146 147 When removing endpoints (or fields being returned), you must first deprecate 148 the existing API (incrementing MAJOR and AGE) and then wait for all clients, 149 including all clients in operation (e.g. Android and iOS Apps, e-commerce 150 integrations, etc.) to upgrade to a protocol implementation above the 151 deprecated MAJOR revision. Only then you should remove the endpoint and reduce 152 AGE. 153 154 To document these changes, please try to use ``@since`` annotations in the API 155 specifications to explain the MAJOR revision when a feature became available, 156 but most importantly use ``@deprecated X`` annotations to indicate that an API 157 was deprecated and will be removed once MAJOR minus AGE is above X. When using 158 an API, use the ``/config`` endpoints to check for compatibility and show a 159 warning if the version(s) you support and the version(s) offered by the server 160 are incompatible. 161 162 163 Tagging and Package Versioning 164 ------------------------------ 165 166 Release tags are of the form ``v${major}.${minor}.${patch}``. Release tags *should* be 167 annotated git tags. 168 169 We usually consider Debian packaging files (in ``debian/``) to be part of a release. 170 When only the Debian packaging files need to be changed, there are two options: 171 172 * Make a new patch release (``v${major}.${minor}.${patch+1}``) 173 * Make a Debian release: 174 175 * Debian version now includes a revision: ``${major}.${minor}.${patch}-${debrevision}`` 176 * The tag is Debian-specific: ``debian-${major}.${minor}.${patch}-${debrevision}`` 177 178 All source repos *should* include a ``contrib/bump`` script that automates bumping the\ 179 version in all relevant source and packaging files. 180 In the future, we might add an option to the script to only release a packaging bump. 181 Right now, that process is manual. 182 183 We support tagged and published pre-release versions via tags of the form ``v${major}.${minor}.${patch}-dev.${n}``. 184 The corresponding Debian version must be ``${major}.${minor}.${patch}~dev${n}``. 185 186 Nightly Debian packages should follow the `Debian conventions <https://wiki.debian.org/Versioning>`__ of ``{upcoming_version}~git{date}.{hash}-{revision}``. 187 188 Testing Tools 189 ------------- 190 191 For full ``make check`` support, install these programs: 192 193 - `jq <https://github.com/stedolan/jq>`__ 194 - `curl <http://curl.haxx.se>`__ 195 - `faketime <https://github.com/wolfcw/libfaketime>`__ 196 197 The ``make check`` should be able to function without them, but 198 their presence permits some tests to run that would otherwise be skipped. 199 200 Manual Testing Database Reset 201 ----------------------------- 202 203 Sometimes ``make check`` will fail with some kind of database (SQL) 204 error, perhaps with a message like ``OBJECT does not exist`` in the 205 ``test-suite.log`` file, where ``OBJECT`` is the name of a table or function. 206 In that case, it may be necessary to reset the ``talercheck`` database 207 with the commands: 208 209 .. code-block:: console 210 211 $ dropdb talercheck 212 $ createdb talercheck 213 214 This is because, at the moment, there is no support for 215 doing these steps automatically in the ``make check`` flow. 216 217 (If ``make check`` still fails after the reset, file a bug report as usual.) 218 219 Bug Tracking 220 ------------ 221 222 Bug tracking is done with Mantis (https://www.mantisbt.org/). The bug tracker 223 is available at `<https://bugs.taler.net>`_. A registration on the Web site is 224 needed in order to use the bug tracker, only read access is granted without a 225 login. 226 227 We use the following conventions for the bug states: 228 229 * NEW: Incoming bugs are in 'new' so that management (or developers) 230 can easily identify those that need to be checked (report correct? 231 something we want to fix?), prioritized and targeted for releases. 232 "NEW" bugs are never assigned to a developer. 233 234 * FEEDBACK: When blocked on feedback from reporter or other developer. 235 Assigned to other developer (but cannot be assigned to reporter, 236 in this case MAY remain associated with the developer who expects 237 the feedback). If a bug is on feedback, it automatically should be 238 considered to be high-priority to give the feedback (as it is 239 blocking someone else!). 240 241 * ACKNOWLEDGED: The bug has been reviewed, but no decision about 242 what action to take has been made yet. Should not be worked on 243 until management (or a developer) comes up with a plan. 244 "ACKNOWLEDGED" bugs should NOT be assigned to a developer. 245 246 * CONFIRMED: This is a real issue that should be worked on, but 247 is not yet actively worked on. If working on this bug requires 248 other bugs to be fixed first, they should be added as 249 child-bugs (via relationships). Developers are always welcome 250 to self-assign bugs that are "CONFIRMED" if they start to work 251 on a bug. "CONFIRMED" bugs should NOT be assigned to a developer. 252 253 * ASSIGNED: The specific developer the bug is assigned to is 254 **actively** working on the issue. Developers should strive to 255 not have more than 5-10 bugs assigned to them at any time. 256 Only having one assigned to you is totally OK! 257 Developers should aggressively un-assign bugs that they are 258 blocked on, cannot make progress on, or are no longer actively 259 working on (but of course, better *resolve* them before 260 moving on if possible). If the bug remains open, it probably 261 should go back to "CONFIRMED" or "ACKNOWLEDGED". 262 263 * RESOLVED: The bug has been fixed in Git. 264 265 * CLOSED: An official release was made with the fix in it. 266 267 When developers want to keep an eye on certain bugs, they should 268 **monitor** them. Multiple developers can be monitoring a bug, but 269 it can only be assigned to one. Developers should also keep an 270 eye on the roadmap (by release), bug categories they care about, 271 and of course priorities / severities. 272 273 We use **tags** to categorize bugs. Common tags that also imply 274 some urgency include (in alphabetical order): 275 276 * accounting: issues required for accounting (such as taxes by merchants) 277 * compliance: issues related to regulatory compliance 278 * $CUSTOMER: issues requested by a particular customer 279 * performance: performance problems or ideas for improvement 280 * security: security issues (including planned improvements to security) 281 * UX: user experience issues 282 283 These tags **should** be attached to "NEW" bugs if they apply. 284 285 286 Code Repositories 287 ----------------- 288 289 Taler code is versioned with Git. For those users without write access, all the 290 codebases are found at the following URL: 291 292 .. code-block:: none 293 294 git://git.taler.net/<repository> 295 296 A complete list of all the existing repositories is currently found at 297 `<https://git.taler.net/>`_. 298 299 300 Committing code 301 --------------- 302 303 Before you can obtain Git write access, you must sign the copyright 304 agreement. As we collaborate closely with GNUnet, we use their 305 copyright agreement -- with the understanding that your contributions 306 to GNU Taler are included in the assignment. You can find the 307 agreement on the `GNUnet site <https://gnunet.org/en/copyright.html>`_. 308 Please sign and mail it to Christian Grothoff as he currently collects 309 all the documents for GNUnet e.V. 310 311 To obtain Git access, you need to send us your SSH public key. Most core 312 team members have administrative Git access, so simply contact whoever 313 is your primary point of contact so far. You can 314 find instructions on how to generate an SSH key 315 in the `Git book <https://git-scm.com/book/en/v2/Git-on-the-Server-Generating-Your-SSH-Public-Key>`_. 316 If you have been granted write access, you first of all must change the URL of 317 the respective repository to: 318 319 .. code-block:: none 320 321 ssh://git@git.taler.net/<repository> 322 323 For an existing checkout, this can be done by editing the ``.git/config`` file. 324 325 The server is configured to reject all commits that have not been signed with 326 GnuPG. If you do not yet have a GnuPG key, you must create one, as explained 327 in the `GNU Privacy Handbook <https://www.gnupg.org/gph/en/manual/c14.html>`_. 328 You do not need to share the respective public key with us to make commits. 329 However, we recommend that you upload it to key servers, put it on your 330 business card and personally meet with other GNU hackers to have it signed 331 such that others can verify your commits later. 332 333 To sign all commits, you should run 334 335 .. code-block:: console 336 337 $ git config --global commit.gpgsign true 338 339 You can also sign individual commits only by adding the ``-S`` option to the 340 ``git commit`` command. If you accidentally already made commits but forgot 341 to sign them, you can retroactively add signatures using: 342 343 .. code-block:: console 344 345 $ git rebase -S 346 347 348 Whether you commit to a personal branch (recommended: ``dev/$USER/...``), 349 a feature branch or to ``master`` should 350 depend on your level of comfort and the nature of the change. As a general 351 rule, the code in ``master`` must always build and tests should always pass, at 352 least on your own system. However, we all make mistakes and you should expect 353 to receive friendly reminders if your change did not live up to this simple 354 standard. We plan to move to a system where the CI guarantees this invariant 355 in the future. 356 357 In order to keep a linear and clean commits history, we advise to avoid 358 merge commits and instead always rebase your changes before pushing to 359 the ``master`` branch. If you commit and later find out that new commits were 360 pushed, the following command will pull the new commits and rebase yours 361 on top of them. 362 363 .. code-block:: console 364 365 # -S instructs Git to (re)sign your commits 366 $ git pull --rebase -S 367 368 369 370 Observing changes 371 ----------------- 372 373 Every commit to the ``master`` branch of any of our public repositories 374 (and almost all are public) is automatically sent to the 375 gnunet-svn@gnu.org mailinglist. That list is for Git commits only, 376 and must not be used for discussions. It also carries commits from 377 our main dependencies, namely GNUnet and GNU libmicrohttpd. While 378 it can be high volume, the lists is a good way to follow overall 379 development. 380 381 382 Code generator usage policy 383 --------------------------- 384 385 We do neither encourage nor discourage the use of tools for code generation. 386 It is up to the individual developer to decide if a tool is acceptable for 387 a particular task. But of course, we do encourage you to use FLOSS tools 388 and we MUST NOT become dependend on non-free software! That said, if you 389 use tools, you must document their use and in particular satisfy the 390 `NLnet policy on the use of "AI" <https://nlnet.nl/news/2025/20250829-policy-on-use-of-AI.html>`__. 391 392 Specifically, we ask developers to always put generated code into a *separate* 393 Git commit and to include the full prompt in the commit message. Naturally, 394 you may clean up the code generator's output, but then you should do so in 395 separate Git commits (and of course only merge into master/stable after the 396 clean up is complete). But do preserve (not squash!) the commit with the 397 generated code so that it remains documented what the prompts were and which 398 code is generated. This will go a long way to keep code auditors sane! 399 400 401 Communication 402 ------------- 403 404 For public discussions we use the taler@gnu.org mailinglist. All developers 405 should subscribe to the low-volume Taler mailinglist. There are separate 406 low-volume mailinglists for gnunet-developers (@gnu.org) and for libmicrohttpd 407 (@gnu.org). For internal discussions we use https://mattermost.taler.net/ 408 (invitation only, but also achieved). 409 410 411 What to put in bootstrap 412 ------------------------ 413 414 Each repository has a ``bootstrap`` script, which contains commands for the 415 developer to run after a repository checkout (i.e., after ``git clone`` or 416 ``git pull``). 417 Typically, this updates and initializes submodules, prepares the tool chain, 418 and runs ``autoreconf``. 419 The last step generates the ``configure`` script, whether for immediate use or 420 for inclusion in the distribution tarball. 421 422 One common submodule is ``contrib/gana``, which pulls from the 423 `GNUnet GANA repository <https://git.gnunet.org/gana.git/>`__. 424 For example, in the 425 `Taler exchange repository <https://git.taler.net/exchange.git>`__, 426 the bootstrap script eventually runs the ``git submodule update --init`` command 427 early on, and later runs script ``./contrib/gana-generate.sh``, which 428 generates files such as ``src/include/taler_signatures.h``. 429 430 Thus, to update that file, you need to: 431 432 - (in GANA repo) Find a suitable (unused) name and number for the Signature 433 Purposes database. 434 435 - Add it to GANA, in ``gnunet-signatures/registry.rec``. 436 (You can check for uniqueness with the ``recfix`` utility.) 437 438 - Commit the change, and push it to the GANA Git repo. 439 440 - (in Taler Repo) Run the ``contrib/gana-latest.sh`` script. 441 442 - Bootstrap, configure, do ``make install``, ``make check``, etc. 443 (Basically, make sure the change does not break anything.) 444 445 - Commit the submodule change, and push it to the Taler exchange Git repo. 446 447 A similar procedure is required for other databases in GANA. 448 See file ``README`` in the various directories for specific instructions. 449 450 451 Debian and Ubuntu Repositories 452 ============================== 453 454 We package our software for Debian and Ubuntu. 455 456 Nightly Repositories 457 -------------------- 458 459 To try the latest, unstable and untested versions of packages, 460 you can add the nightly package sources. 461 462 .. code-block:: shell-session 463 464 # For Debian (trixie) 465 $ curl -sS https://deb.taler.net/apt-nightly/taler-trixie-ci.sources \ 466 | tee /etc/apt/sources.list.d/taler-trixie-nightly.sources 467 468 469 Taler Deployment on gv.taler.net 470 ================================ 471 472 This section describes the GNU Taler deployment on ``gv.taler.net``. ``gv`` 473 is our server at BFH. It hosts the Git repositories, Web sites, CI and other 474 services. Developers can receive an SSH account and e-mail alias for the 475 system, you should contact Javier, Christian or Florian. As with Git, ask 476 your primary team contact for shell access if you think you need it. 477 478 479 DNS 480 --- 481 482 DNS records for taler.net are controlled by the GNU Taler maintainers, 483 specifically Christian and Florian, and our system administrator, Javier. If 484 you need a sub-domain to be added, please contact one of them. 485 486 487 User Acccounts 488 -------------- 489 490 On ``gv.taler.net``, there are three system users that are set up to 491 serve Taler on the Internet: 492 493 - ``head``: serves ``*.head.taler.net`` and gets automatically 494 built by Buildbot every 2 hours from the ``sandcastle-ng.git``. 495 Master key may be reset occasionally 496 497 - ``taler-test``: serves ``*.test.taler.net`` and does *NOT* get 498 automatically built, and runs more recent tags and/or unreleased 499 versions of Taler components. Master key may be reset 500 occasionally. 501 502 - ``demo``: serves ``*.demo.taler.net``. Never automatically built. 503 Master key is retained. 504 505 Demo Upgrade Procedure 506 ====================== 507 508 #. Login as the ``demo`` user on ``gv.taler.net``. 509 #. Pull the latest ``sandcastle-ng.git`` code in checkout at ``$HOME/sandcastle-ng``. 510 #. Run ``systemctl --user restart container-taler-sandcastle-demo.service`` 511 #. Refer to the sandcastle-ng README (https://git.taler.net/sandcastle-ng.git/about/) 512 for more info. 513 514 515 Upgrading the ``demo`` environment should be done with care, and ideally be 516 coordinated on the mailing list before. It is our goal for ``demo`` to always 517 run a "working version" that is compatible with various published wallets. 518 Please use the :doc:`demo upgrade checklist <../checklists/checklist-demo-upgrade>` to make 519 sure everything is working. 520 Nginx is already configured to reach the services as exported by the user unit. 521 522 523 Tagging components 524 ------------------ 525 526 All Taler components must be tagged with git before they are deployed on the 527 ``demo`` environment, using a tag of the following form: 528 529 .. code-block:: none 530 531 demo-YYYY-MM-DD-SS 532 YYYY = year 533 MM = month 534 DD = day 535 SS = serial 536 537 Environments and Builders on taler.net 538 ====================================== 539 540 Buildbot implementation 541 ----------------------- 542 543 GNU Taler uses a buildbot implementation (front end at https://buildbot.taler.net) to manage continuous integration. Buildbot documentation is at https://docs.buildbot.net/. 544 545 Here are some highlights: 546 547 - The WORKER is the config that that lives on a shell account on a localhost (taler.net), where this host has buildbot-worker installed. The WORKER executes the commands that perform all end-functions of buildbot. 548 549 - The WORKER running buildbot-worker receives these commands by authenticating and communicating with the buildbot server using parameters that were specified when the worker was created in that shell account with the ``buildbot-worker`` command. 550 551 - The buildbot server's master.cfg file contains FACTORY declarations which specify the commands that the WORKER will run on localhost. 552 553 - The FACTORY is tied to the WORKER in master.cfg by a BUILDER. 554 555 - The master.cfg also allows for SCHEDULER that defines how and when the BUILDER is executed. 556 557 - Our master.cfg file is checked into git, and then periodically updated on a particular account on taler.net (ask Christian for access if needed). Do not edit this file directly/locally on taler.net, but check changes into Git. 558 559 560 Best Practices: 561 562 - When creating a new WORKER in the ``master.cfg`` file, leave a comment specifying the server and user account that this WORKER is called from. (At this time, taler.net is the only server used by this implementation, but it's still good practice.) 563 564 - Create a worker from a shell account with this command: ``buildbot-worker create-worker <workername> localhost <username> <password>`` 565 566 Then make sure there is a WORKER defined in master.cfg like: ``worker.Worker("<username>", "<password>")`` 567 568 Test builder 569 ------------ 570 571 This builder (``test-builder``) compiles and starts every Taler component. 572 The associated worker is run by the ``taler-test`` Gv user, via the SystemD 573 unit ``buildbot-worker-taler``. The following commands start/stop/restart 574 the worker: 575 576 .. code-block:: 577 578 systemctl --user start buildbot-worker-taler 579 systemctl --user stop buildbot-worker-taler 580 systemctl --user restart buildbot-worker-taler 581 582 .. note:: 583 the mentioned unit file can be found at ``deployment.git/systemd-services/`` 584 585 Wallet builder 586 -------------- 587 588 This builder (``wallet-builder``) compiles every Taler component 589 and runs the wallet integration tests. The associated worker is 590 run by the ``walletbuilder`` Gv user, via the SystemD unit ``buildbot-worker-wallet``. 591 The following commands start/stop/restart the worker: 592 593 .. code-block:: 594 595 systemctl --user start buildbot-worker-wallet 596 systemctl --user stop buildbot-worker-wallet 597 systemctl --user restart buildbot-worker-wallet 598 599 .. note:: 600 the mentioned unit file can be found at ``deployment.git/systemd-services/`` 601 602 Documentation Builder 603 --------------------- 604 605 All the Taler documentation is built by the user ``docbuilder`` that 606 runs a Buildbot worker. The following commands set the ``docbuilder`` up, 607 starting with an empty home directory. 608 609 .. code-block:: console 610 611 # Log-in as the 'docbuilder' user. 612 613 $ cd $HOME 614 $ git clone git://git.taler.net/deployment 615 $ ./deployment/bootstrap-docbuilder 616 617 # If the previous step worked, the setup is 618 # complete and the Buildbot worker can be started. 619 620 $ buildbot-worker start worker/ 621 622 623 Website Builder 624 --------------- 625 626 627 Taler Websites, ``www.taler.net`` and ``stage.taler.net``, are built by the 628 user ``taler-websites`` by the means of a Buildbot worker. The following 629 commands set the ``taler-websites`` up, starting with an empty home directory. 630 631 .. code-block:: console 632 633 # Log-in as the 'taler-websites' user. 634 635 $ cd $HOME 636 $ git clone git://git.taler.net/deployment 637 $ ./deployment/bootstrap-sitesbuilder 638 639 # If the previous step worked, the setup is 640 # complete and the Buildbot worker can be started. 641 642 $ buildbot-worker start worker/ 643 644 645 Code coverage 646 ------------- 647 648 Code coverage tests are run by the ``lcovworker`` user, and are also driven 649 by Buildbot. 650 651 .. code-block:: console 652 653 # Log-in as the 'lcovworker' user. 654 655 $ cd $HOME 656 $ git clone git://git.taler.net/deployment 657 $ ./deployment/bootstrap-taler lcov 658 659 # If the previous step worked, the setup is 660 # complete and the Buildbot worker can be started. 661 662 $ buildbot-worker start worker/ 663 664 The results are then published at ``https://lcov.taler.net/``. 665 666 Producing auditor reports 667 ------------------------- 668 669 Both 'test' and 'demo' setups get their auditor reports compiled 670 by a Buildbot worker. The following steps get the reports compiler 671 prepared. 672 673 .. code-block:: console 674 675 # Log-in as <env>-auditor, with <env> being either 'test' or 'demo' 676 677 $ git clone git://git.taler.net/deployment 678 $ ./deployment/buildbot/bootstrap-scripts/prepare-auditorreporter <env> 679 680 # If the previous steps worked, then it should suffice to start 681 # the worker, with: 682 683 $ buildbot-worker start worker/ 684 685 686 .. _DatabaseVersioning: 687 688 Database schema versioning 689 -------------------------- 690 691 The PostgreSQL databases of the exchange and the auditor are versioned. 692 See the ``versioning.sql`` file in the respective directory for documentation. 693 694 Every set of changes to the database schema must be stored in a new 695 versioned SQL script. The scripts must have contiguous numbers. After 696 any release (or version being deployed to a production or staging 697 environment), existing scripts MUST be immutable. 698 699 Developers and operators MUST NOT make changes to database schema 700 outside of this versioning. All tables of a GNU Taler component should live in their own schema. 701 702 703 QA Plans 704 ======== 705 706 .. include:: ../checklists/qa-1.0.rst 707 708 709 Releases 710 ======== 711 712 .. include:: ../checklists/checklist-release.rst 713 714 Release Process 715 --------------- 716 717 This document describes the process for releasing a new version of the 718 various Taler components to the official GNU mirrors. 719 720 The following components are published on the GNU mirrors 721 722 - taler-exchange (exchange.git) 723 - taler-merchant (merchant.git) 724 - sync (sync.git) 725 - taler-mdb (taler-mdb.git) 726 - libeufin (libeufin.git) 727 - challenger (challenger.git) 728 - wallet-core (wallet-core.git) 729 730 Tagging 731 ------- 732 733 Tag releases with an **annotated** commit, like 734 735 .. code-block:: console 736 737 $ git tag -a v0.1.0 -m "Official release v0.1.0" 738 $ git push origin v0.1.0 739 740 741 Database for tests 742 ------------------ 743 744 For tests in the exchange and merchant to run, make sure that a database 745 *talercheck* is accessible by *$USER*. Otherwise tests involving the 746 database logic are skipped. 747 748 .. include:: ../frags/db-stores-sensitive-data.rst 749 750 Exchange, merchant 751 ------------------ 752 753 Set the version in ``configure.ac``. The commit being tagged should be 754 the change of the version. 755 756 Tag the current GANA version that works with the exchange and merchant and 757 checkout that tag of gana.git (instead of master). Otherwise, if there are 758 incompatible changes in GANA (like removed symbols), old builds could break. 759 760 Update the Texinfo documentation using the files from docs.git: 761 762 .. code-block:: console 763 764 # Get the latest documentation repository 765 $ cd $GIT/docs 766 $ git pull 767 $ make texinfo 768 # The *.texi files are now in _build/texinfo 769 # 770 # This checks out the prebuilt branch in the prebuilt directory 771 $ git worktree add prebuilt prebuilt 772 $ cd prebuilt 773 # Copy the pre-built documentation into the prebuilt directory 774 $ cp -r ../_build/texinfo . 775 # Push and commit to branch 776 $ git commit -a -S -m "updating texinfo" 777 $ git status 778 # Verify that all files that should be tracked are tracked, 779 # new files will have to be added to the Makefile.am in 780 # exchange.git as well! 781 $ git push 782 # Remember $REVISION of commit 783 # 784 # Go to exchange 785 $ cd $GIT/exchange/doc/prebuilt 786 # Update submodule to point to latest commit 787 $ git checkout $REVISION 788 789 Finally, the Automake ``Makefile.am`` files may have to be adjusted to 790 include new ``*.texi`` files or images. 791 792 For bootstrap, you will need to install 793 `GNU Recutils <https://www.gnu.org/software/recutils/>`_. 794 795 For the exchange test cases to pass, ``make install`` must be run first. 796 Without it, test cases will fail because plugins can't be located. 797 798 .. code-block:: console 799 800 $ ./bootstrap 801 $ ./configure # add required options for your system 802 $ make dist 803 $ tar -xf taler-$COMPONENT-$VERSION.tar.gz 804 $ cd taler-$COMPONENT-$VERSION 805 $ make install check 806 807 Wallet WebExtension 808 ------------------- 809 810 The version of the wallet is in *manifest.json*. The ``version_name`` 811 should be adjusted, and *version* should be increased independently on 812 every upload to the WebStore. 813 814 .. code-block:: console 815 816 $ ./configure 817 $ make dist 818 819 Upload to GNU mirrors 820 --------------------- 821 822 See https://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads 823 824 Directive file: 825 826 .. code-block:: none 827 828 version: 1.2 829 directory: taler 830 filename: taler-exchange-0.1.0.tar.gz 831 symlink: taler-exchange-0.1.0.tar.gz taler-exchange-latest.tar.gz 832 833 Upload the files in **binary mode** to the ftp servers. 834 835 836 Creating Debian packages 837 ------------------------ 838 839 Our general setup is based on 840 https://wiki.debian.org/DebianRepository/SetupWithReprepro 841 842 First, update at least the version of the Debian package in 843 debian/changelog, and then run: 844 845 .. code-block:: bash 846 847 $ dpkg-buildpackage -rfakeroot -b -uc -us 848 849 in the respective source directory (GNUnet, exchange, merchant) to create the 850 ``.deb`` files. Note that they will be created in the parent directory. This 851 can be done on gv.taler.net, or on another (secure) machine. 852 Actual release builds should be done via the Docker images 853 that can be found in ``deployment.git`` under packaging. 854 855 On ``gv``, we use the ``aptbuilder`` user to manage the reprepro repository. 856 857 Next, the ``*.deb`` files should be copied to gv.taler.net, say to 858 ``/home/aptbuilder/incoming``. Then, run 859 860 .. code-block:: bash 861 862 # cd /home/aptbuilder/apt 863 # reprepro includedeb bullseye ~/incoming/*.deb 864 865 to import all Debian files from ``~/incoming/`` into the ``bullseye`` 866 distribution. If Debian packages were build against other distributions, 867 reprepro may need to be first configured for those and the import command 868 updated accordingly. 869 870 Finally, make sure to clean up ``~/incoming/`` (by deleting the 871 now imported ``*.deb`` files). 872 873 874 875 Continuous integration 876 ====================== 877 878 CI is done with Buildbot (https://buildbot.net/), and builds are 879 triggered by the means of Git hooks. The results are published at 880 https://buildbot.taler.net/ . 881 882 In order to avoid downtimes, CI uses a "blue/green" deployment 883 technique. In detail, there are two users building code on the system, 884 the "green" and the "blue" user; and at any given time, one is running 885 Taler services and the other one is either building the code or waiting 886 for that. 887 888 There is also the possibility to trigger builds manually, but this is 889 only reserved to "admin" users. 890 891 892 Internationalisation 893 ==================== 894 895 Internationalisation (a.k.a "translation") is handled with using text-based 896 localization files named PO (Portable Object) holding pairs of original and 897 translated strings. 898 899 iOS Apps 900 ======== 901 902 .. _Build-iOS-from-source: 903 904 Building Taler Wallet for iOS from source 905 ----------------------------------------- 906 907 The GNU Taler Wallet iOS app is in 908 `the official Git repository <https://git.taler.net/taler-ios.git>`__. 909 910 Compatibility 911 ^^^^^^^^^^^^^ 912 913 The minimum version of iOS supported is 15.0. 914 This app runs on all iPhone models at least as new as the iPhone 6S. 915 916 917 Building 918 ^^^^^^^^ 919 920 Before building the iOS wallet, you must first checkout the 921 `quickjs-tart repo <https://git.taler.net/quickjs-tart.git>`__ 922 and the 923 `wallet-core repo <https://git.taler.net/wallet-core.git>`__. 924 925 Have all 3 local repos (wallet-core, quickjs-tart, and this one) adjacent at 926 the same level (e.g. in a "GNU_Taler" folder) 927 Taler.xcworkspace expects the QuickJS framework sub-project to be at 928 ``../quickjs-tart/QuickJS-rt.xcodeproj``. 929 930 Build wallet-core first: 931 932 .. code-block:: shell-session 933 934 $ cd wallet-core 935 $ make embedded 936 $ open packages/taler-wallet-embedded/dist 937 938 then drag or move its product "taler-wallet-core-qjs.mjs" 939 into your quickjs-tart folder right at the top level. 940 941 Open Taler.xcworkspace, and set scheme / target to Taler_Wallet. Build&run... 942 943 Don't open QuickJS-rt.xcodeproj or TalerWallet.xcodeproj and build anything 944 there - all needed libraries and frameworks will be built automatically from 945 Taler.xcworkspace. 946 947 948 Android Apps 949 ============ 950 951 Android App Nightly Builds 952 -------------------------- 953 954 There are currently three Android apps in 955 `the official Git repository <https://git.taler.net/taler-android.git>`__: 956 957 * Wallet 958 [`CI <https://git.taler.net/taler-android.git/tree/wallet/.gitlab-ci.yml>`__] 959 * Merchant PoS Terminal 960 [`CI <https://git.taler.net/taler-android.git/tree/merchant-terminal/.gitlab-ci.yml>`__] 961 * Cashier 962 [`CI <https://git.taler.net/taler-android.git/tree/cashier/.gitlab-ci.yml>`__] 963 964 Their git repositories are `mirrored at Gitlab <https://gitlab.com/gnu-taler/taler-android>`__ 965 to utilize their CI 966 and `F-Droid <https://f-droid.org>`_'s Gitlab integration 967 to `publish automatic nightly builds <https://f-droid.org/docs/Publishing_Nightly_Builds/>`_ 968 for each change on the ``master`` branch. 969 970 All three apps publish their builds to the same F-Droid nightly repository 971 (which is stored as a git repository): 972 https://gitlab.com/gnu-taler/fdroid-repo-nightly 973 974 You can download the APK files directly from that repository 975 or add it to the F-Droid app for automatic updates 976 by clicking the following link (on the phone that has F-Droid installed). 977 978 `GNU Taler Nightly F-Droid Repository <fdroidrepos://gnu-taler.gitlab.io/fdroid-repo-nightly/fdroid/repo?fingerprint=55F8A24F97FAB7B0960016AF393B7E57E7A0B13C2D2D36BAC50E1205923A7843>`_ 979 980 .. note:: 981 Nightly apps can be installed alongside official releases 982 and thus are meant **only for testing purposes**. 983 Use at your own risk! 984 985 .. _Build-apps-from-source: 986 987 Building apps from source 988 ------------------------- 989 990 Note that this guide is different from other guides for building Android apps, 991 because it does not require you to run non-free software. 992 It uses the Merchant PoS Terminal as an example, but works as well for the other apps 993 if you replace ``merchant-terminal`` with ``wallet`` or ``cashier``. 994 995 First, ensure that you have the required dependencies installed: 996 997 * Java Development Kit 8 or higher (default-jdk-headless) 998 * git 999 * unzip 1000 1001 Then you can get the app's source code using git: 1002 1003 .. code-block:: console 1004 1005 # Start by cloning the Android git repository 1006 $ git clone https://git.taler.net/taler-android.git 1007 1008 # Change into the directory of the cloned repository 1009 $ cd taler-android 1010 1011 # Find out which Android SDK version you will need 1012 $ grep -i compileSdkVersion merchant-terminal/build.gradle 1013 1014 The last command will return something like ``compileSdkVersion 29``. 1015 So visit the `Android Rebuilds <http://android-rebuilds.beuc.net/>`_ project 1016 and look for that version of the Android SDK there. 1017 If the SDK version is not yet available as a free rebuild, 1018 you can try to lower the ``compileSdkVersion`` in the app's ``merchant-terminal/build.gradle`` file. 1019 Note that this might break things 1020 or require you to also lower other versions such as ``targetSdkVersion``. 1021 1022 In our example, the version is ``29`` which is available, 1023 so download the "SDK Platform" package of "Android 10.0.0 (API 29)" 1024 and unpack it: 1025 1026 .. code-block:: console 1027 1028 # Change into the directory that contains your downloaded SDK 1029 $ cd $HOME 1030 1031 # Unpack/extract the Android SDK 1032 $ unzip android-sdk_eng.10.0.0_r14_linux-x86.zip 1033 1034 # Tell the build system where to find the SDK 1035 $ export ANDROID_SDK_ROOT="$HOME/android-sdk_eng.10.0.0_r14_linux-x86" 1036 1037 # Change into the directory of the cloned repository 1038 $ cd taler-android 1039 1040 # Build the merchant-terminal app 1041 $ ./gradlew :merchant-terminal:assembleRelease 1042 1043 If you get an error message complaining about build-tools 1044 1045 > Failed to install the following Android SDK packages as some licences have not been accepted. 1046 build-tools;29.0.3 Android SDK Build-Tools 29.0.3 1047 1048 you can try changing the ``buildToolsVersion`` in the app's ``merchant-terminal/build.gradle`` file 1049 to the latest "Android SDK build tools" version supported by the Android Rebuilds project. 1050 1051 After the build finished successfully, 1052 you will find your APK in ``merchant-terminal/build/outputs/apk/release/``. 1053 1054 Update translations 1055 ------------------- 1056 1057 Translations are managed with Taler's weblate instance: 1058 https://weblate.taler.net/projects/gnu-taler/ 1059 1060 To update translations, enter the taler-android git repository 1061 and ensure that the weblate remote exists: 1062 1063 .. code-block:: console 1064 1065 $ git config -l | grep weblate 1066 1067 If it does not yet exist (empty output), you can add it like this: 1068 1069 .. code-block:: console 1070 1071 $ git remote add weblate https://weblate.taler.net/git/gnu-taler/wallet-android/ 1072 1073 Then you can merge in translations commit from the weblate remote: 1074 1075 .. code-block:: console 1076 1077 # ensure you have latest version 1078 $ git fetch weblate 1079 1080 # merge in translation commits 1081 $ git merge weblate/master 1082 1083 Afterwards, build the entire project from source and test the UI 1084 to ensure that no erroneous translations (missing placeholders) are breaking things. 1085 1086 Release process 1087 --------------- 1088 1089 After extensive testing, the code making up a new release should get a signed git tag. 1090 The current tag format is: 1091 1092 * cashier-$VERSION 1093 * pos-$VERSION 1094 * wallet-$VERSION (where $VERSION has a v prefix) 1095 1096 .. code-block:: console 1097 1098 $ git tag -s $APP-$VERSION 1099 1100 F-Droid 1101 ^^^^^^^ 1102 Nightly builds get published automatically (see above) after pushing code to the official repo. 1103 Actual releases get picked up by F-Droid's official repository via git tags. 1104 So ensure that all releases get tagged properly. 1105 1106 Some information for F-Droid official repository debugging: 1107 1108 * Wallet: [`metadata <https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.wallet.fdroid.yml>`__] [`build log <https://f-droid.org/wiki/page/net.taler.wallet.fdroid/lastbuild>`__] 1109 * Cashier: [`metadata <https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.cashier.yml>`__] [`build log <https://f-droid.org/wiki/page/net.taler.cashier/lastbuild>`__] 1110 * PoS: [`metadata <https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.merchantpos.yml>`__] [`build log <https://f-droid.org/wiki/page/net.taler.merchantpos/lastbuild>`__] 1111 1112 Google Play 1113 ^^^^^^^^^^^ 1114 Google Play uploads are managed via `Fastlane <https://docs.fastlane.tools/getting-started/android/setup/>`__. 1115 Before proceeding, ensure that this is properly set up 1116 and that you have access to the Google Play API. 1117 1118 It is important to have access to the signing keys and Google Play access keys 1119 (JSON) and to ensure that the following environment variables are set 1120 correctly and made available to Fastlane: 1121 1122 .. code-block:: bash 1123 1124 TALER_KEYSTORE_PATH= 1125 TALER_KEYSTORE_PASS= 1126 TALER_KEYSTORE_WALLET_ALIAS= 1127 TALER_KEYSTORE_WALLET_PASS= 1128 TALER_KEYSTORE_POS_ALIAS= 1129 TALER_KEYSTORE_POS_PASS= 1130 TALER_KEYSTORE_CASHIER_ALIAS= 1131 TALER_KEYSTORE_CASHIER_PASS= 1132 TALER_JSON_KEY_FILE= 1133 1134 To release an app, enter into its respective folder and run fastlane: 1135 1136 .. code-block:: console 1137 1138 $ bundle exec fastlane 1139 1140 Then select the deploy option. 1141 1142 All uploads are going to the beta track by default. These can be promoted to 1143 production later or immediately after upload if you feel daring. It is also 1144 important to bump the version and build code with every release. 1145 1146 .. _Code-coverage: 1147 1148 Code Coverage 1149 ============= 1150 1151 Code coverage is done with the Gcov / Lcov 1152 (http://ltp.sourceforge.net/coverage/lcov.php) combo, and it is run 1153 nightly (once a day) by a Buildbot worker. The coverage results are 1154 then published at https://lcov.taler.net/ . 1155 1156 1157 Coding Conventions 1158 ================== 1159 1160 GNU Taler is developed primarily in C, Kotlin, Python, Swift and TypeScript. 1161 1162 Components written in C 1163 ----------------------- 1164 1165 These are the general coding style rules for Taler. 1166 1167 * Baseline rules are to follow GNU guidelines, modified or extended 1168 by the GNUnet style: https://docs.gnunet.org/handbook/gnunet.html#Coding-style 1169 1170 Naming conventions 1171 ^^^^^^^^^^^^^^^^^^ 1172 1173 * include files (very similar to GNUnet): 1174 1175 * if installed, must start with "``taler_``" (exception: platform.h), 1176 and MUST live in src/include/ 1177 * if NOT installed, must NOT start with "``taler_``" and 1178 MUST NOT live in src/include/ and 1179 SHOULD NOT be included from outside of their own directory 1180 * end in "_lib" for "simple" libraries 1181 * end in "_plugin" for plugins 1182 * end in "_service" for libraries accessing a service, i.e. the exchange 1183 1184 * binaries: 1185 1186 * taler-exchange-xxx: exchange programs 1187 * taler-merchant-xxx: merchant programs (demos) 1188 * taler-wallet-xxx: wallet programs 1189 * plugins should be libtaler_plugin_xxx_yyy.so: plugin yyy for API xxx 1190 * libtalerxxx: library for API xxx 1191 1192 * logging 1193 1194 * tools use their full name in GNUNET_log_setup 1195 (i.e. 'taler-exchange-offline') and log using plain 'GNUNET_log'. 1196 * pure libraries (without associated service) use 'GNUNET_log_from' 1197 with the component set to their library name (without lib or '.so'), 1198 which should also be their directory name (i.e. 'util') 1199 * plugin libraries (without associated service) use 'GNUNET_log_from' 1200 with the component set to their type and plugin name (without lib or '.so'), 1201 which should also be their directory name (i.e. 'exchangedb-postgres') 1202 * libraries with associated service) use 'GNUNET_log_from' 1203 with the name of the service, which should also be their 1204 directory name (i.e. 'exchange') 1205 * for tools with ``-l LOGFILE``, its absence means write logs to stderr 1206 1207 * configuration 1208 1209 * same rules as for GNUnet 1210 1211 * exported symbols 1212 1213 * must start with TALER_[SUBSYSTEMNAME]_ where SUBSYSTEMNAME 1214 MUST match the subdirectory of src/ in which the symbol is defined 1215 * from libtalerutil start just with ``TALER_``, without subsystemname 1216 * if scope is ONE binary and symbols are not in a shared library, 1217 use binary-specific prefix (such as TMH = taler-exchange-httpd) for 1218 globals, possibly followed by the subsystem (TMH_DB_xxx). 1219 1220 * structs: 1221 1222 * structs that are 'packed' and do not contain pointers and are 1223 thus suitable for hashing or similar operations are distinguished 1224 by adding a "P" at the end of the name. (NEW) Note that this 1225 convention does not hold for the GNUnet-structs (yet). 1226 * structs that are used with a purpose for signatures, additionally 1227 get an "S" at the end of the name. 1228 1229 * private (library-internal) symbols (including structs and macros) 1230 1231 * must not start with ``TALER_`` or any other prefix 1232 1233 * testcases 1234 1235 * must be called "test_module-under-test_case-description.c" 1236 1237 * performance tests 1238 1239 * must be called "perf_module-under-test_case-description.c" 1240 1241 Shell Scripts 1242 ------------- 1243 1244 Shell scripts should be avoided if at all possible. The only permissible uses of shell scripts 1245 in GNU Taler are: 1246 1247 * Trivial invocation of other commands. 1248 * Scripts for compatibility (e.g. ``./configure``) that must run on 1249 as many systems as possible. 1250 1251 When shell scripts are used, they ``MUST`` begin with the following ``set`` command: 1252 1253 .. code-block:: console 1254 1255 # Make the shell fail on undefined variables and 1256 # commands with non-zero exit status. 1257 $ set -eu 1258 1259 Kotlin 1260 ------ 1261 1262 We so far have no specific guidelines, please follow best practices 1263 for the language. 1264 1265 1266 Python 1267 ------ 1268 1269 Supported Python Versions 1270 ^^^^^^^^^^^^^^^^^^^^^^^^^ 1271 1272 Python code should be written and build against version 3.7 of Python. 1273 1274 Style 1275 ^^^^^ 1276 1277 We use `yapf <https://github.com/google/yapf>`_ to reformat the 1278 code to conform to our style instructions. 1279 A reusable yapf style file can be found in ``build-common``, 1280 which is intended to be used as a git submodule. 1281 1282 Python for Scripting 1283 ^^^^^^^^^^^^^^^^^^^^ 1284 1285 When using Python for writing small utilities, the following libraries 1286 are useful: 1287 1288 * ``click`` for argument parsing (should be preferred over argparse) 1289 * ``pathlib`` for path manipulation (part of the standard library) 1290 * ``subprocess`` for "shelling out" to other programs. Prefer ``subprocess.run`` 1291 over the older APIs. 1292 1293 1294 Swift 1295 ----- 1296 1297 Please follow best practices for the language. 1298 1299 1300 TypeScript 1301 ---------- 1302 1303 Please follow best practices for the language. 1304 1305 1306 Testing library 1307 =============== 1308 1309 This chapter is a VERY ABSTRACT description of how testing is 1310 implemented in Taler, and in NO WAY wants to substitute the reading of 1311 the actual source code by the user. 1312 1313 In Taler, a test case is an array of ``struct TALER_TESTING_Command``, 1314 informally referred to as ``CMD``, that is iteratively executed by the 1315 testing interpreter. This latter is transparently initiated by the 1316 testing library. 1317 1318 However, the developer does not have to defined CMDs manually, but 1319 rather call the proper constructor provided by the library. For example, 1320 if a CMD is supposed to test feature ``x``, then the library would 1321 provide the ``TALER_TESTING_cmd_x ()`` constructor for it. Obviously, 1322 each constructor has its own particular arguments that make sense to 1323 test ``x``, and all constructor are thoroughly commented within the 1324 source code. 1325 1326 Internally, each CMD has two methods: ``run ()`` and ``cleanup ()``. The 1327 former contains the main logic to test feature ``x``, whereas the latter 1328 cleans the memory up after execution. 1329 1330 In a test life, each CMD needs some internal state, made by values it 1331 keeps in memory. Often, the test has to *share* those values with other 1332 CMDs: for example, CMD1 may create some key material and CMD2 needs this 1333 key material to encrypt data. 1334 1335 The offering of internal values from CMD1 to CMD2 is made by *traits*. A 1336 trait is a ``struct TALER_TESTING_Trait``, and each CMD contains an array 1337 of traits, that it offers via the public trait interface to other 1338 commands. The definition and filling of such array happens transparently 1339 to the test developer. 1340 1341 For example, the following example shows how CMD2 takes an amount object 1342 offered by CMD1 via the trait interface. 1343 1344 Note: the main interpreter and the most part of CMDs and traits are 1345 hosted inside the exchange codebase, but nothing prevents the developer 1346 from implementing new CMDs and traits within other codebases. 1347 1348 .. code-block:: c 1349 1350 /* Without loss of generality, let's consider the 1351 * following logic to exist inside the run() method of CMD1 */ 1352 ... 1353 1354 struct TALER_Amount *a; 1355 /** 1356 * the second argument (0) points to the first amount object offered, 1357 * in case multiple are available. 1358 */ 1359 if (GNUNET_OK != TALER_TESTING_get_trait_amount_obj (cmd2, 0, &a)) 1360 return GNUNET_SYSERR; 1361 ... 1362 1363 use(a); /* 'a' points straight into the internal state of CMD2 */ 1364 1365 In the Taler realm, there is also the possibility to alter the behaviour 1366 of supposedly well-behaved components. This is needed when, for example, 1367 we want the exchange to return some corrupted signature in order to 1368 check if the merchant backend detects it. 1369 1370 This alteration is accomplished by another service called *twister*. The 1371 twister acts as a proxy between service A and B, and can be programmed 1372 to tamper with the data exchanged by A and B. 1373 1374 Please refer to the Twister codebase (under the ``test`` directory) in 1375 order to see how to configure it. 1376 1377 1378 User-Facing Terminology 1379 ======================= 1380 1381 This section contains terminology that should be used and that should not be 1382 used in the user interface and help materials. 1383 1384 Terms to Avoid 1385 -------------- 1386 1387 Refreshing 1388 Refreshing is the internal technical terminology for the protocol to 1389 give change for partially spent coins 1390 1391 **Use instead**: "Obtaining change" 1392 1393 Charge 1394 Charge has two opposite meanings (charge to a credit card vs. charge a battery). 1395 This can confuse users. 1396 1397 **Use instead**: "Obtain", "Credit", "Debit", "Withdraw", "Top up" 1398 1399 Coin 1400 Coins are an internal construct, the user should never be aware that their balance 1401 is represented by coins of different denominations. 1402 1403 **Use instead**: "(Digital) Cash" or "(Wallet) Balance" 1404 1405 Consumer 1406 Has bad connotation of consumption. 1407 1408 **Use instead**: Customer or user. 1409 1410 Proposal 1411 The term used to describe the process of the merchant facilitating the download 1412 of the signed contract terms for an order. 1413 1414 **Avoid**. Generally events that relate to proposal downloads 1415 should not be shown to normal users, only developers. Instead, use 1416 "communication with mechant failed" if a proposed order can't be downloaded. 1417 1418 Anonymous E-Cash 1419 Should be generally avoided, since Taler is only anonymous for 1420 the customer. Also some people are scared of anonymity (which as 1421 a term is also way too absolute, as anonymity is hardly ever perfect). 1422 1423 **Use instead**: "Privacy-preserving", "Privacy-friendly" 1424 1425 Payment Replay 1426 The process of proving to the merchant that the customer is entitled 1427 to view a digital product again, as they already paid for it. 1428 1429 **Use instead**: In the event history, "re-activated digital content purchase" 1430 could be used. (FIXME: this is still not nice.) 1431 1432 Session ID 1433 See Payment Replay. 1434 1435 Order 1436 Too ambiguous in the wallet. 1437 1438 **Use instead**: Purchase 1439 1440 Fulfillment URL 1441 URL that the serves the digital content that the user purchased 1442 with their payment. Can also be something like a donation receipt. 1443 1444 Donau 1445 Developer-internal name for the tax authority component. 1446 1447 **Use instead**: Tax authority 1448 1449 Terms to Use 1450 ------------ 1451 1452 Auditor 1453 Regulatory entity that certifies exchanges and oversees their operation. 1454 1455 Exchange Operator 1456 The entity/service that gives out digital cash in exchange for some 1457 other means of payment. 1458 1459 In some contexts, using "Issuer" could also be appropriate. 1460 When showing a balance breakdown, 1461 we can say "100 Eur (issued by exchange.euro.taler.net)". 1462 Sometimes we may also use the more generic term "Payment Service Provider" 1463 when the concept of an "Exchange" is still unclear to the reader. 1464 1465 Refund 1466 A refund is given by a merchant to the customer (rather the customer's wallet) 1467 and "undoes" a previous payment operation. 1468 1469 Payment 1470 The act of sending digital cash to a merchant to pay for an order. 1471 1472 Purchase 1473 Used to refer to the "result" of a payment, as in "view purchase". 1474 Use sparsingly, as the word doesn't fit for all payments, such as donations. 1475 1476 Contract Terms 1477 Partially machine-readable representation of the merchant's obligation after the 1478 customer makes a payment. 1479 1480 Merchant 1481 Party that receives a payment. 1482 1483 Wallet 1484 Also "Taler Wallet". Software component that manages the user's digital cash 1485 and payments. 1486 1487 1488 Developer Glossary 1489 ================== 1490 1491 This glossary is meant for developers. It contains some terms that we usually do not 1492 use when talking to end users or even system administrators. 1493 1494 .. glossary:: 1495 :sorted: 1496 1497 absolute time 1498 method of keeping time in :term:`GNUnet` where the time is represented 1499 as the number of microseconds since 1.1.1970 (UNIX epoch). Called 1500 absolute time in contrast to :term:`relative time`. 1501 1502 aggregate 1503 the :term:`exchange` combines multiple payments received by the 1504 same :term:`merchant` into one larger :term:`wire transfer` to 1505 the respective merchant's :term:`bank` account 1506 1507 auditor 1508 trusted third party that verifies that the :term:`exchange` is operating correctly 1509 1510 bank 1511 traditional financial service provider who offers 1512 :term:`wire transfers <wire transfer>` between accounts 1513 1514 buyer 1515 individual in control of a Taler :term:`wallet`, usually using it to 1516 :term:`spend` the :term:`coins <coin>` on :term:`contracts <contract>` (see also :term:`customer`). 1517 1518 close 1519 operation an :term:`exchange` performs on a :term:`reserve` that has not been 1520 :term:`emptied <empty>` by :term:`withdraw` operations. When closing a reserve, the 1521 exchange wires the remaining funds back to the customer, minus a :term:`fee` 1522 for closing 1523 1524 customer 1525 individual that directs the buyer (perhaps the same individual) to make a purchase 1526 1527 coin 1528 coins are individual token representing a certain amount of value, also known as the :term:`denomination` of the coin 1529 1530 refresh commitment 1531 data that the wallet commits to during the :term:`melt` stage of the 1532 :term:`refresh` protocol where it 1533 has to prove to the :term:`exchange` that it is deriving the :term:`fresh` 1534 coins as specified by the Taler protocol. The commitment is verified 1535 probabilistically (see: :term:`kappa`) during the :term:`reveal` stage. 1536 1537 contract 1538 formal agreement between :term:`merchant` and :term:`customer` specifying the 1539 :term:`contract terms` and signed by the merchant and the :term:`coins <coin>` of the 1540 customer 1541 1542 contract terms 1543 the individual clauses specifying what the buyer is purchasing from the 1544 :term:`merchant` 1545 1546 denomination 1547 unit of currency, specifies both the currency and the face value of a :term:`coin`, 1548 as well as associated fees and validity periods 1549 1550 denomination key 1551 (RSA) key used by the exchange to certify that a given :term:`coin` is valid and of a 1552 particular :term:`denomination` 1553 1554 deposit 1555 operation by which a merchant passes coins to an exchange, expecting the 1556 exchange to credit his bank account in the future using an 1557 :term:`aggregate` :term:`wire transfer` 1558 1559 drain 1560 process by which an exchange operator takes the profits 1561 (from :term:`fees <fee>`) out of the escrow account and moves them into 1562 their regular business account 1563 1564 dirty 1565 a :term:`coin` is dirty if its public key may be known to an entity other than 1566 the customer, thereby creating the danger of some entity being able to 1567 link multiple transactions of coin's owner if the coin is not refreshed 1568 1569 empty 1570 a :term:`reserve` is being emptied when a :term:`wallet` is using the 1571 reserve's private key to :term:`withdraw` coins from it. This reduces 1572 the balance of the reserve. Once the balance reaches zero, we say that 1573 the reserve has been (fully) emptied. Reserves that are not emptied 1574 (which is the normal process) are :term:`closed <close>` by the exchange. 1575 1576 exchange 1577 Taler's payment service operator. Issues electronic coins during 1578 withdrawal and redeems them when they are deposited by merchants 1579 1580 expired 1581 Various operations come with time limits. In particular, denomination keys 1582 come with strict time limits for the various operations involving the 1583 coin issued under the denomination. The most important limit is the 1584 deposit expiration, which specifies until when wallets are allowed to 1585 use the coin in deposit or refreshing operations. There is also a "legal" 1586 expiration, which specifies how long the exchange keeps records beyond the 1587 deposit expiration time. This latter expiration matters for legal disputes 1588 in courts and also creates an upper limit for refreshing operations on 1589 special zombie coin 1590 1591 GNUnet 1592 Codebase of various libraries for a better Internet, some of which 1593 GNU Taler depends upon. 1594 1595 fakebank 1596 implementation of the :term:`bank` API in memory to be used only for test 1597 cases. 1598 1599 fee 1600 an :term:`exchange` charges various fees for its service. The different 1601 fees are specified in the protocol. There are fees per coin for 1602 :term:`withdrawing <withdraw>`, :term:`depositing <deposit>`, :term:`melting <melt>`, and 1603 :term:`refunding <refund>`. Furthermore, there are fees per wire transfer 1604 when a :term:`reserve` is :term:`closed <close>` 1605 and for :term:`aggregate` :term:`wire transfers <wire transfer>` 1606 to the :term:`merchant`. 1607 1608 fresh 1609 a :term:`coin` is fresh if its public key is only known to the customer 1610 1611 JSON 1612 JavaScript Object Notation (JSON) is a 1613 serialization format derived from the JavaScript language which is 1614 commonly used in the Taler protocol as the payload of HTTP requests 1615 and responses. 1616 1617 kappa 1618 security parameter used in the :term:`refresh` protocol. Defined to be 3. 1619 The probability of successfully evading the income transparency with the 1620 refresh protocol is 1:kappa. 1621 1622 libeufin 1623 Kotlin component that implements a regional currency bank and an 1624 adapter to communicate via EBICS with European core banking systems. 1625 1626 link 1627 specific step in the :term:`refresh` protocol that an exchange must offer 1628 to prevent abuse of the :term:`refresh` mechanism. The link step is 1629 not needed in normal operation, it just must be offered. 1630 1631 master key 1632 offline key used by the exchange to certify denomination keys and 1633 message signing keys 1634 1635 melt 1636 step of the :term:`refresh` protocol where a :term:`dirty` :term:`coin` 1637 is invalidated to be reborn :term:`fresh` in a subsequent 1638 :term:`reveal` step. 1639 1640 merchant 1641 party receiving payments (usually in return for goods or services) 1642 1643 message signing key 1644 key used by the exchange to sign online messages, other than coins 1645 1646 order 1647 offer made by the merchant to a wallet; pre-cursor to 1648 a contract where the wallet is not yet fixed. Turns 1649 into a :term:`contract` when a wallet claims the order. 1650 1651 owner 1652 a coin is owned by the entity that knows the private key of the coin 1653 1654 relative time 1655 method of keeping time in :term:`GNUnet` where the time is represented 1656 as a relative number of microseconds. Thus, a relative time specifies 1657 an offset or a duration, but not a date. Called relative time in 1658 contrast to :term:`absolute time`. 1659 1660 recoup 1661 Operation by which an exchange returns the value of coins affected 1662 by a :term:`revocation <revoke>` to their :term:`owner`, either by allowing the owner to 1663 withdraw new coins or wiring funds back to the bank account of the :term:`owner`. 1664 1665 planchet 1666 precursor data for a :term:`coin`. A planchet includes the coin's internal 1667 secrets (coin private key, blinding factor), but lacks the RSA signature 1668 of the :term:`exchange`. When :term:`withdrawing <withdraw>`, a :term:`wallet` 1669 creates and persists a planchet before asking the exchange to sign it to 1670 get the coin. 1671 1672 purchase 1673 Refers to the overall process of negotiating a :term:`contract` and then 1674 making a payment with :term:`coins <coin>` to a :term:`merchant`. 1675 1676 privacy policy 1677 Statement of an operator how they will protect the privacy of users. 1678 1679 proof 1680 Message that cryptographically demonstrates that a particular claim is correct. 1681 1682 proposal 1683 a list of :term:`contract terms` that has been completed and signed by the 1684 merchant backend. 1685 1686 refresh 1687 operation by which a :term:`dirty` :term:`coin` is converted into one or more 1688 :term:`fresh` coins. Involves :term:`melting <melt>` the :term:`dirty` coins and 1689 then :term:`revealing <reveal>` so-called :term:`transfer keys <transfer key>`. 1690 1691 refund 1692 operation by which a merchant steps back from the right to funds that he 1693 obtained from a :term:`deposit` operation, giving the right to the funds back 1694 to the customer 1695 1696 refund transaction id 1697 unique number by which a merchant identifies a :term:`refund`. Needed 1698 as refunds can be partial and thus there could be multiple refunds for 1699 the same :term:`purchase`. 1700 1701 reserve 1702 accounting mechanism used by the exchange to track customer funds 1703 from incoming :term:`wire transfers <wire transfer>`. A reserve is created whenever 1704 a customer wires money to the exchange using a well-formed public key 1705 in the subject. The exchange then allows the customer's :term:`wallet` 1706 to :term:`withdraw` up to the amount received in :term:`fresh` 1707 :term:`coins <coin>` from the reserve, thereby emptying the reserve. If a 1708 reserve is not emptied, the exchange will eventually :term:`close` it. 1709 1710 Other definition: Funds set aside for future use; either the balance of a customer at the 1711 exchange ready for withdrawal, or the funds kept in the exchange;s bank 1712 account to cover obligations from coins in circulation. 1713 1714 reveal 1715 step in the :term:`refresh` protocol where some of the transfer private 1716 keys are revealed to prove honest behavior on the part of the wallet. 1717 In the reveal step, the exchange returns the signed :term:`fresh` coins. 1718 1719 revoke 1720 exceptional operation by which an exchange withdraws a denomination from 1721 circulation, either because the signing key was compromised or because 1722 the exchange is going out of operation; unspent coins of a revoked 1723 denomination are subjected to recoup. 1724 1725 sharing 1726 users can share ownership of a :term:`coin` by sharing access to the coin's 1727 private key, thereby allowing all co-owners to spend the coin at any 1728 time. 1729 1730 spend 1731 operation by which a customer gives a merchant the right to deposit 1732 coins in return for merchandise 1733 1734 transfer key 1735 special cryptographic key used in the :term:`refresh` protocol, some of which 1736 are revealed during the :term:`reveal` step. Note that transfer keys have, 1737 despite the name, no relationship to :term:`wire transfers <wire transfer>`. They merely 1738 help to transfer the value from a :term:`dirty` coin to a :term:`fresh` coin 1739 1740 terms 1741 the general terms of service of an operator, possibly including 1742 the :term:`privacy policy`. Not to be confused with the 1743 :term:`contract terms` which are about the specific purchase. 1744 1745 transaction 1746 method by which ownership is exclusively transferred from one entity 1747 1748 user 1749 any individual using the Taler payment system 1750 (see :term:`customer`, :term:`buyer`, :term:`merchant`). 1751 1752 version 1753 Taler uses various forms of versioning. There is a database 1754 schema version (stored itself in the database, see \*-0000.sql) describing 1755 the state of the table structure in the database of an :term:`exchange`, 1756 :term:`auditor` or :term:`merchant`. There is a protocol 1757 version (CURRENT:REVISION:AGE, see GNU libtool) which specifies 1758 the network protocol spoken by an :term:`exchange` or :term:`merchant` 1759 including backwards-compatibility. And finally there is the software 1760 release version (MAJOR.MINOR.PATCH, see https://semver.org/) of 1761 the respective code base. 1762 1763 wallet 1764 software running on a customer's computer; withdraws, stores and 1765 spends coins 1766 1767 WebExtension 1768 Cross-browser API used to implement the GNU Taler wallet browser extension. 1769 1770 wire gateway 1771 API used by the exchange to talk with some real-time gross settlement system 1772 (core banking system, blockchain) to notice inbound credits wire transfers 1773 (during withdraw) and to trigger outbound debit wire transfers (primarily 1774 for deposits). 1775 1776 wire transfer 1777 a wire transfer is a method of sending funds between :term:`bank` accounts 1778 1779 wire transfer identifier 1780 Subject of a wire transfer from the exchange to a merchant; 1781 set by the aggregator to a random nonce which uniquely 1782 identifies the transfer. 1783 1784 withdraw 1785 operation by which a :term:`wallet` can convert funds from a :term:`reserve` to 1786 fresh coins 1787 1788 zombie 1789 :term:`coin` where the respective :term:`denomination key` is past its 1790 :term:`deposit` :term:`expiration <expired>` time, but which is still (again) valid 1791 for an operation because it was :term:`melted <melt>` while it was still 1792 valid, and then later again credited during a :term:`recoup` process 1793 1794 1795 1796 Developer Tools 1797 =============== 1798 1799 This section describes various internal programs to make life easier for the 1800 developer. 1801 1802 1803 taler-harness 1804 ------------- 1805 1806 **taler-harness deployment gen-coin-config** is a tool to simplify Taler configuration generation. 1807 1808 1809 **taler-harness deployment gen-coin-config** 1810 [**-min-amount**=**\ \ *VALUE*] 1811 [**-max-amount**=**\ \ *VALUE*]