taler-developer-manual.rst (65543B)
1 .. 2 This file is part of GNU TALER. 3 4 Copyright (C) 2014-2025 Taler Systems SA 5 6 TALER is free software; you can redistribute it and/or modify it under the 7 terms of the GNU Affero General Public License as published by the Free Software 8 Foundation; either version 3.0, or (at your option) any later version. 9 10 TALER is distributed in the hope that it will be useful, but WITHOUT ANY 11 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR 12 A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. 13 14 You should have received a copy of the GNU Affero General Public License along with 15 TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> 16 17 @author Christian Grothoff 18 19 General Developer Manual 20 ######################## 21 22 .. note:: 23 24 This manual contains information for developers working on GNU Taler 25 and related components. It is not intended for a general audience. 26 27 28 Project Overview 29 ================ 30 31 GNU Taler consists of a large (and growing) number of components 32 in various Git repositories. The following list gives a first 33 overview: 34 35 * exchange: core payment processing logic with a REST API, plus various 36 helper processes for interaction with banks and cryptographic 37 computations. Also includes the logic for the auditor and an 38 in-memory "bank" API implementation for testing. 39 40 * libeufin: implementation of the "bank" API using the EBICS protocol 41 used by banks in the EU. Allows an exchange to interact with 42 European banks. 43 44 * taler-magnet-bank: implementation of the "bank" API using the Magnet Bank 45 API. Allows an exchange to interact with Magnet Bank. 46 47 * taler-cyclos: implementation of the "bank" API using the Cyclos API. Allows an exchange to interact with a Cyclos network. 48 49 * depolymerization: implementation of the "bank" API on top of 50 blockchains, specifically Bitcoin and Ethereum. Allows an exchange 51 to interact with crypto-currencies. 52 53 * merchant: payment processing backend to be run by merchants, 54 offering a REST API. 55 56 * wallet-core: platform-independent implementation of a wallet to be run by 57 normal users. Includes also the WebExtension for various browsers. 58 Furthermore, includes various single-page apps used by other 59 components (especially as libeufin and merchant). Also includes 60 command-line wallet and tools for testing. 61 62 * taler-android: Android Apps including the Android wallet, the 63 Android point-of-sale App and the Android casher app. 64 65 * taler-ios: iOS wallet App. 66 67 * sync: backup service, provides a simple REST API to allow users to 68 make encrypted backups of their wallet state. 69 70 * anastasis: key escrow service, provides a simple REST API to allow 71 users to distribute encryption keys across multiple providers and 72 define authorization policies for key recovery. 73 74 * taler-mdb: integration of Taler with the multi-drop-bus (MDB) API 75 used by vending machines. Allows Taler payments to be integrated 76 with vending machines. 77 78 * gnu-taler-payment-for-woocommerce: payment plugin for the 79 woocommerce (wordpress) E-commerce solution. 80 81 * twister: man-in-the-middle proxy for tests that require fuzzing a 82 REST/JSON protocol. Used for some of our testing. 83 84 * challenger: implementation of an OAuth 2.0 provider that can be used 85 to verify that a user can receive SMS or E-mail at particular addresses. 86 Used as part of KYC processes of the exchange. 87 88 * taler-mailbox: messaging service used to store and forward payment 89 messages to Taler wallets. 90 91 * taldir: directory service used to lookup Taler wallet addresses for 92 sending invoices or payments to other wallets. 93 94 * taler-merchant-demos: various demonstration services operated at 95 'demo.taler.net', including a simple shop and a donation page. 96 97 There are other important repositories without code, including: 98 99 * gana: Hosted on git.gnunet.org, this repository defines various 100 constants used in the GNU Taler project. 101 102 * docs: documentation, including this very document. 103 104 * marketing: various presentations, papers and other resources for 105 outreach. 106 107 * large-media: very large data objects, such as videos. 108 109 * www: the taler.net website. 110 111 Fundamentals 112 ============ 113 114 Versioning 115 ---------- 116 117 A central rule is to never break anything for any dependency. To accomplish 118 this, we use versioning, of the APIs, database schema and the protocol. The 119 database versioning approach is described in the :ref:`Database schema 120 versioning <DatabaseVersioning>` section. Here, we will focus on API and 121 protocol versioning. 122 123 The key issue we need to solve with protocols and APIs (and that does not 124 apply to database versioning) is being able to introduce and remove features 125 without requiring a flag day where all components must update at the same 126 time. For this, we use GNU libtool style versioning with MAJOR:REVISION:AGE 127 and *not* semantic versioning (SEMVER). With GNU libtool style versioning, 128 first the REVISION should be increased on every change to the respective code. 129 Then, each time a feature is introduced or deprecated, the MAJOR and AGE 130 numbers are increased. Whenever an API is actually removed the AGE number is 131 reduced to match the distance since the removed API was deprecated. Thus, if 132 some client implements version X of the protocol (including not using any APIs 133 that have been deprecated), it is compatible for any implementation where 134 MAJOR is larger or equal to X, and MAJOR minus AGE is smaller or equal to X. 135 REVISION is not used for expected compatibility issues and merely serves to 136 uniquely identify each version (in combination with MAJOR). 137 138 To evolve any implementation, it is thus critical to first of all never 139 just break an existing API or endpoint. The only acceptable modifications 140 are to return additional information (being aware of binary compatibility!) 141 or to accept additional optional arguments (again, in a way that does not 142 break existing users). Thus, the most common way to introduce changes will 143 be the addition of new endpoints. Breaking existing endpoints is only ever 144 at best acceptable while in the process of introducing it and if you are 145 absolutely sure that there are zero users in other components. 146 147 When removing endpoints (or fields being returned), you must first deprecate 148 the existing API (incrementing MAJOR and AGE) and then wait for all clients, 149 including all clients in operation (e.g. Android and iOS Apps, e-commerce 150 integrations, etc.) to upgrade to a protocol implementation above the 151 deprecated MAJOR revision. Only then you should remove the endpoint and reduce 152 AGE. 153 154 To document these changes, please try to use ``@since`` annotations in the API 155 specifications to explain the MAJOR revision when a feature became available, 156 but most importantly use ``@deprecated X`` annotations to indicate that an API 157 was deprecated and will be removed once MAJOR minus AGE is above X. When using 158 an API, use the ``/config`` endpoints to check for compatibility and show a 159 warning if the version(s) you support and the version(s) offered by the server 160 are incompatible. 161 162 163 Tagging and Package Versioning 164 ------------------------------ 165 166 Release tags are of the form ``v${major}.${minor}.${patch}``. Release tags *should* be 167 annotated git tags. 168 169 We usually consider Debian packaging files (in ``debian/``) to be part of a release. 170 When only the Debian packaging files need to be changed, there are two options: 171 172 * Make a new patch release (``v${major}.${minor}.${patch+1}``) 173 * Make a Debian release: 174 175 * Debian version now includes a revision: ``${major}.${minor}.${patch}-${debrevision}`` 176 * The tag is Debian-specific: ``debian-${major}.${minor}.${patch}-${debrevision}`` 177 178 All source repos *should* include a ``contrib/bump`` script that automates bumping the\ 179 version in all relevant source and packaging files. 180 In the future, we might add an option to the script to only release a packaging bump. 181 Right now, that process is manual. 182 183 We support tagged and published pre-release versions via tags of the form ``v${major}.${minor}.${patch}-dev.${n}``. 184 The corresponding Debian version must be ``${major}.${minor}.${patch}~dev${n}``. 185 186 Nightly Debian packages should follow the `Debian conventions <https://wiki.debian.org/Versioning>`__ of ``{upcoming_version}~git{date}.{hash}-{revision}``. 187 188 Testing Tools 189 ------------- 190 191 For full ``make check`` support, install these programs: 192 193 - `jq <https://github.com/stedolan/jq>`__ 194 - `curl <http://curl.haxx.se>`__ 195 - `faketime <https://github.com/wolfcw/libfaketime>`__ 196 197 The ``make check`` should be able to function without them, but 198 their presence permits some tests to run that would otherwise be skipped. 199 200 Manual Testing Database Reset 201 ----------------------------- 202 203 Sometimes ``make check`` will fail with some kind of database (SQL) 204 error, perhaps with a message like ``OBJECT does not exist`` in the 205 ``test-suite.log`` file, where ``OBJECT`` is the name of a table or function. 206 In that case, it may be necessary to reset the ``talercheck`` database 207 with the commands: 208 209 .. code-block:: console 210 211 $ dropdb talercheck 212 $ createdb talercheck 213 214 This is because, at the moment, there is no support for 215 doing these steps automatically in the ``make check`` flow. 216 217 (If ``make check`` still fails after the reset, file a bug report as usual.) 218 219 Bug Tracking 220 ------------ 221 222 Bug tracking is done with Mantis (https://www.mantisbt.org/). The bug tracker 223 is available at `<https://bugs.taler.net>`_. A registration on the Web site is 224 needed in order to use the bug tracker, only read access is granted without a 225 login. 226 227 We use the following conventions for the bug states: 228 229 * NEW: Incoming bugs are in 'new' so that management (or developers) 230 can easily identify those that need to be checked (report correct? 231 something we want to fix?), prioritized and targeted for releases. 232 "NEW" bugs are never assigned to a developer. 233 234 * FEEDBACK: When blocked on feedback from reporter or other developer. 235 Assigned to other developer (but cannot be assigned to reporter, 236 in this case MAY remain associated with the developer who expects 237 the feedback). If a bug is on feedback, it automatically should be 238 considered to be high-priority to give the feedback (as it is 239 blocking someone else!). 240 241 * ACKNOWLEDGED: The bug has been reviewed, but no decision about 242 what action to take has been made yet. Should not be worked on 243 until management (or a developer) comes up with a plan. 244 "ACKNOWLEDGED" bugs should NOT be assigned to a developer. 245 246 * CONFIRMED: This is a real issue that should be worked on, but 247 is not yet actively worked on. If working on this bug requires 248 other bugs to be fixed first, they should be added as 249 child-bugs (via relationships). Developers are always welcome 250 to self-assign bugs that are "CONFIRMED" if they start to work 251 on a bug. "CONFIRMED" bugs should NOT be assigned to a developer. 252 253 * ASSIGNED: The specific developer the bug is assigned to is 254 **actively** working on the issue. Developers should strive to 255 not have more than 5-10 bugs assigned to them at any time. 256 Only having one assigned to you is totally OK! 257 Developers should aggressively un-assign bugs that they are 258 blocked on, cannot make progress on, or are no longer actively 259 working on (but of course, better *resolve* them before 260 moving on if possible). If the bug remains open, it probably 261 should go back to "CONFIRMED" or "ACKNOWLEDGED". 262 263 * RESOLVED: The bug has been fixed in Git. 264 265 * CLOSED: An official release was made with the fix in it. 266 267 When developers want to keep an eye on certain bugs, they should 268 **monitor** them. Multiple developers can be monitoring a bug, but 269 it can only be assigned to one. Developers should also keep an 270 eye on the roadmap (by release), bug categories they care about, 271 and of course priorities / severities. 272 273 We use **tags** to categorize bugs. Common tags that also imply 274 some urgency include (in alphabetical order): 275 276 * accounting: issues required for accounting (such as taxes by merchants) 277 * compliance: issues related to regulatory compliance 278 * $CUSTOMER: issues requested by a particular customer 279 * performance: performance problems or ideas for improvement 280 * security: security issues (including planned improvements to security) 281 * UX: user experience issues 282 283 These tags **should** be attached to "NEW" bugs if they apply. 284 285 286 Code Repositories 287 ----------------- 288 289 Taler code is versioned with Git. For those users without write access, all the 290 codebases are found at the following URL: 291 292 .. code-block:: none 293 294 git://git.taler.net/<repository> 295 296 A complete list of all the existing repositories is currently found at 297 `<https://git.taler.net/>`_. 298 299 300 Committing code 301 --------------- 302 303 Before you can obtain Git write access, you must sign the copyright 304 agreement. As we collaborate closely with GNUnet, we use their 305 copyright agreement -- with the understanding that your contributions 306 to GNU Taler are included in the assignment. You can find the 307 agreement on the `GNUnet site <https://gnunet.org/en/copyright.html>`_. 308 Please sign and mail it to Christian Grothoff as he currently collects 309 all the documents for GNUnet e.V. 310 311 To obtain Git access, you need to send us your SSH public key. Most core 312 team members have administrative Git access, so simply contact whoever 313 is your primary point of contact so far. You can 314 find instructions on how to generate an SSH key 315 in the `Git book <https://git-scm.com/book/en/v2/Git-on-the-Server-Generating-Your-SSH-Public-Key>`_. 316 If you have been granted write access, you first of all must change the URL of 317 the respective repository to: 318 319 .. code-block:: none 320 321 ssh://git@git.taler.net/<repository> 322 323 For an existing checkout, this can be done by editing the ``.git/config`` file. 324 325 The server is configured to reject all commits that have not been signed with 326 GnuPG. If you do not yet have a GnuPG key, you must create one, as explained 327 in the `GNU Privacy Handbook <https://www.gnupg.org/gph/en/manual/c14.html>`_. 328 You do not need to share the respective public key with us to make commits. 329 However, we recommend that you upload it to key servers, put it on your 330 business card and personally meet with other GNU hackers to have it signed 331 such that others can verify your commits later. 332 333 To sign all commits, you should run 334 335 .. code-block:: console 336 337 $ git config --global commit.gpgsign true 338 339 You can also sign individual commits only by adding the ``-S`` option to the 340 ``git commit`` command. If you accidentally already made commits but forgot 341 to sign them, you can retroactively add signatures using: 342 343 .. code-block:: console 344 345 $ git rebase -S 346 347 348 Whether you commit to a personal branch (recommended: ``dev/$USER/...``), 349 a feature branch or to ``master`` should 350 depend on your level of comfort and the nature of the change. As a general 351 rule, the code in ``master`` must always build and tests should always pass, at 352 least on your own system. However, we all make mistakes and you should expect 353 to receive friendly reminders if your change did not live up to this simple 354 standard. We plan to move to a system where the CI guarantees this invariant 355 in the future. 356 357 In order to keep a linear and clean commits history, we advise to avoid 358 merge commits and instead always rebase your changes before pushing to 359 the ``master`` branch. If you commit and later find out that new commits were 360 pushed, the following command will pull the new commits and rebase yours 361 on top of them. 362 363 .. code-block:: console 364 365 # -S instructs Git to (re)sign your commits 366 $ git pull --rebase -S 367 368 369 370 Observing changes 371 ----------------- 372 373 Every commit to the ``master`` branch of any of our public repositories 374 (and almost all are public) is automatically sent to the 375 gnunet-svn@gnu.org mailinglist. That list is for Git commits only, 376 and must not be used for discussions. It also carries commits from 377 our main dependencies, namely GNUnet and GNU libmicrohttpd. While 378 it can be high volume, the lists is a good way to follow overall 379 development. 380 381 382 Code generator usage policy 383 --------------------------- 384 385 We do neither encourage nor discourage the use of tools for code generation. 386 It is up to the individual developer to decide if a tool is acceptable for 387 a particular task. But of course, we do encourage you to use FLOSS tools 388 and we MUST NOT become dependent on non-free software! That said, if you 389 use tools, you must document their use and in particular satisfy the 390 `NLnet policy on the use of "AI" <https://nlnet.nl/news/2025/20250829-policy-on-use-of-AI.html>`__. 391 392 Specifically, we ask developers to always put generated code into a *separate* 393 Git commit and to include the full prompt in the commit message. Naturally, 394 you may clean up the code generator's output, but then you should do so in 395 separate Git commits (and of course only merge into master/stable after the 396 clean up is complete). But do preserve (not squash!) the commit with the 397 generated code so that it remains documented what the prompts were and which 398 code is generated. This will go a long way to keep code auditors sane! 399 400 401 Communication 402 ------------- 403 404 For public discussions we use the taler@gnu.org mailinglist. All developers 405 should subscribe to the low-volume Taler mailinglist. There are separate 406 low-volume mailinglists for gnunet-developers (@gnu.org) and for libmicrohttpd 407 (@gnu.org). For internal discussions we use https://mattermost.taler.net/ 408 (invitation only, but also archived). 409 410 411 What to put in bootstrap 412 ------------------------ 413 414 Each repository has a ``bootstrap`` script, which contains commands for the 415 developer to run after a repository checkout (i.e., after ``git clone`` or 416 ``git pull``). 417 Typically, this updates and initializes submodules, prepares the tool chain, 418 and runs ``autoreconf``. 419 The last step generates the ``configure`` script, whether for immediate use or 420 for inclusion in the distribution tarball. 421 422 One common submodule is ``contrib/gana``, which pulls from the 423 `GNUnet GANA repository <https://git.gnunet.org/gana.git/>`__. 424 For example, in the 425 `Taler exchange repository <https://git.taler.net/exchange.git>`__, 426 the bootstrap script eventually runs the ``git submodule update --init`` command 427 early on, and later runs script ``./contrib/gana-generate.sh``, which 428 generates files such as ``src/include/taler_signatures.h``. 429 430 Thus, to update that file, you need to: 431 432 - (in GANA repo) Find a suitable (unused) name and number for the Signature 433 Purposes database. 434 435 - Add it to GANA, in ``gnunet-signatures/registry.rec``. 436 (You can check for uniqueness with the ``recfix`` utility.) 437 438 - Commit the change, and push it to the GANA Git repo. 439 440 - (in Taler Repo) Run the ``contrib/gana-latest.sh`` script. 441 442 - Bootstrap, configure, do ``make install``, ``make check``, etc. 443 (Basically, make sure the change does not break anything.) 444 445 - Commit the submodule change, and push it to the Taler exchange Git repo. 446 447 A similar procedure is required for other databases in GANA. 448 See file ``README`` in the various directories for specific instructions. 449 450 451 Debian and Ubuntu Repositories 452 ============================== 453 454 We package our software for Debian and Ubuntu. 455 456 Nightly Repositories 457 -------------------- 458 459 To try the latest, unstable and untested versions of packages, 460 you can add the nightly package sources. 461 462 .. code-block:: shell-session 463 464 # For Debian (trixie) 465 $ curl -sS https://deb.taler.net/apt-nightly/taler-trixie-ci.sources \ 466 | tee /etc/apt/sources.list.d/taler-trixie-nightly.sources 467 468 469 Taler Deployment on gv.taler.net 470 ================================ 471 472 This section describes the GNU Taler deployment on ``gv.taler.net``. ``gv`` 473 is our server at BFH. It hosts the Git repositories, Web sites, CI and other 474 services. Developers can receive an SSH account and e-mail alias for the 475 system, you should contact Javier, Christian or Florian. As with Git, ask 476 your primary team contact for shell access if you think you need it. 477 478 479 DNS 480 --- 481 482 DNS records for taler.net are controlled by the GNU Taler maintainers, 483 specifically Christian and Florian, and our system administrator, Javier. If 484 you need a sub-domain to be added, please contact one of them. 485 486 487 User Acccounts 488 -------------- 489 490 On ``gv.taler.net``, there are three system users that are set up to 491 serve Taler on the Internet: 492 493 - ``head``: serves ``*.head.taler.net`` and gets automatically 494 built by Buildbot every 2 hours from the ``sandcastle-ng.git``. 495 Master key may be reset occasionally 496 497 - ``taler-test``: serves ``*.test.taler.net`` and does *NOT* get 498 automatically built, and runs more recent tags and/or unreleased 499 versions of Taler components. Master key may be reset 500 occasionally. 501 502 - ``demo``: serves ``*.demo.taler.net``. Never automatically built. 503 Master key is retained. 504 505 Demo Upgrade Procedure 506 ====================== 507 508 #. Login as the ``demo`` user on ``gv.taler.net``. 509 #. Pull the latest ``sandcastle-ng.git`` code in checkout at ``$HOME/sandcastle-ng``. 510 #. Run ``systemctl --user restart container-taler-sandcastle-demo.service`` 511 #. Refer to the sandcastle-ng README (https://git.taler.net/sandcastle-ng.git/about/) 512 for more info. 513 514 515 Upgrading the ``demo`` environment should be done with care, and ideally be 516 coordinated on the mailing list before. It is our goal for ``demo`` to always 517 run a "working version" that is compatible with various published wallets. 518 Please use the :doc:`demo upgrade checklist <../checklists/checklist-demo-upgrade>` to make 519 sure everything is working. 520 Nginx is already configured to reach the services as exported by the user unit. 521 522 523 Tagging components 524 ------------------ 525 526 All Taler components must be tagged with git before they are deployed on the 527 ``demo`` environment, using a tag of the following form: 528 529 .. code-block:: none 530 531 demo-YYYY-MM-DD-SS 532 YYYY = year 533 MM = month 534 DD = day 535 SS = serial 536 537 Environments and Builders on taler.net 538 ====================================== 539 540 Buildbot implementation 541 ----------------------- 542 543 GNU Taler uses a buildbot implementation (front end at https://buildbot.taler.net) to manage continuous integration. Buildbot documentation is at https://docs.buildbot.net/. 544 545 Here are some highlights: 546 547 - The WORKER is the config that that lives on a shell account on a localhost (taler.net), where this host has buildbot-worker installed. The WORKER executes the commands that perform all end-functions of buildbot. 548 549 - The WORKER running buildbot-worker receives these commands by authenticating and communicating with the buildbot server using parameters that were specified when the worker was created in that shell account with the ``buildbot-worker`` command. 550 551 - The buildbot server's master.cfg file contains FACTORY declarations which specify the commands that the WORKER will run on localhost. 552 553 - The FACTORY is tied to the WORKER in master.cfg by a BUILDER. 554 555 - The master.cfg also allows for SCHEDULER that defines how and when the BUILDER is executed. 556 557 - Our master.cfg file is checked into git, and then periodically updated on a particular account on taler.net (ask Christian for access if needed). Do not edit this file directly/locally on taler.net, but check changes into Git. 558 559 560 Best Practices: 561 562 - When creating a new WORKER in the ``master.cfg`` file, leave a comment specifying the server and user account that this WORKER is called from. (At this time, taler.net is the only server used by this implementation, but it's still good practice.) 563 564 - Create a worker from a shell account with this command: ``buildbot-worker create-worker <workername> localhost <username> <password>`` 565 566 Then make sure there is a WORKER defined in master.cfg like: ``worker.Worker("<username>", "<password>")`` 567 568 Test builder 569 ------------ 570 571 This builder (``test-builder``) compiles and starts every Taler component. 572 The associated worker is run by the ``taler-test`` Gv user, via the SystemD 573 unit ``buildbot-worker-taler``. The following commands start/stop/restart 574 the worker: 575 576 .. code-block:: 577 578 systemctl --user start buildbot-worker-taler 579 systemctl --user stop buildbot-worker-taler 580 systemctl --user restart buildbot-worker-taler 581 582 .. note:: 583 the mentioned unit file can be found at ``deployment.git/systemd-services/`` 584 585 Wallet builder 586 -------------- 587 588 This builder (``wallet-builder``) compiles every Taler component 589 and runs the wallet integration tests. The associated worker is 590 run by the ``walletbuilder`` Gv user, via the SystemD unit ``buildbot-worker-wallet``. 591 The following commands start/stop/restart the worker: 592 593 .. code-block:: 594 595 systemctl --user start buildbot-worker-wallet 596 systemctl --user stop buildbot-worker-wallet 597 systemctl --user restart buildbot-worker-wallet 598 599 .. note:: 600 the mentioned unit file can be found at ``deployment.git/systemd-services/`` 601 602 Documentation Builder 603 --------------------- 604 605 All the Taler documentation is built by the user ``docbuilder`` that 606 runs a Buildbot worker. The following commands set the ``docbuilder`` up, 607 starting with an empty home directory. 608 609 .. code-block:: console 610 611 # Log-in as the 'docbuilder' user. 612 613 $ cd $HOME 614 $ git clone git://git.taler.net/deployment 615 $ ./deployment/bootstrap-docbuilder 616 617 # If the previous step worked, the setup is 618 # complete and the Buildbot worker can be started. 619 620 $ buildbot-worker start worker/ 621 622 623 Website Builder 624 --------------- 625 626 627 Taler Websites, ``www.taler.net`` and ``stage.taler.net``, are built by the 628 user ``taler-websites`` by the means of a Buildbot worker. The following 629 commands set the ``taler-websites`` up, starting with an empty home directory. 630 631 .. code-block:: console 632 633 # Log-in as the 'taler-websites' user. 634 635 $ cd $HOME 636 $ git clone git://git.taler.net/deployment 637 $ ./deployment/bootstrap-sitesbuilder 638 639 # If the previous step worked, the setup is 640 # complete and the Buildbot worker can be started. 641 642 $ buildbot-worker start worker/ 643 644 645 Code coverage 646 ------------- 647 648 Code coverage tests are run by the ``lcovworker`` user, and are also driven 649 by Buildbot. 650 651 .. code-block:: console 652 653 # Log-in as the 'lcovworker' user. 654 655 $ cd $HOME 656 $ git clone git://git.taler.net/deployment 657 $ ./deployment/bootstrap-taler lcov 658 659 # If the previous step worked, the setup is 660 # complete and the Buildbot worker can be started. 661 662 $ buildbot-worker start worker/ 663 664 The results are then published at ``https://lcov.taler.net/``. 665 666 Producing auditor reports 667 ------------------------- 668 669 Both 'test' and 'demo' setups get their auditor reports compiled 670 by a Buildbot worker. The following steps get the reports compiler 671 prepared. 672 673 .. code-block:: console 674 675 # Log-in as <env>-auditor, with <env> being either 'test' or 'demo' 676 677 $ git clone git://git.taler.net/deployment 678 $ ./deployment/buildbot/bootstrap-scripts/prepare-auditorreporter <env> 679 680 # If the previous steps worked, then it should suffice to start 681 # the worker, with: 682 683 $ buildbot-worker start worker/ 684 685 686 .. _DatabaseVersioning: 687 688 Database schema versioning 689 -------------------------- 690 691 The PostgreSQL databases of the exchange and the auditor are versioned. 692 See the ``versioning.sql`` file in the respective directory for documentation. 693 694 Every set of changes to the database schema must be stored in a new 695 versioned SQL script. The scripts must have contiguous numbers. After 696 any release (or version being deployed to a production or staging 697 environment), existing scripts MUST be immutable. 698 699 Developers and operators MUST NOT make changes to database schema 700 outside of this versioning. All tables of a GNU Taler component should live in their own schema. 701 702 703 QA Plans 704 ======== 705 706 .. include:: ../checklists/qa-1.0.rst 707 708 709 Releases 710 ======== 711 712 .. include:: ../checklists/checklist-release.rst 713 714 Release Process 715 --------------- 716 717 This document describes the process for releasing a new version of the 718 various Taler components to the official GNU mirrors. 719 720 The following components are published on the GNU mirrors 721 722 - taler-exchange (exchange.git) 723 - taler-merchant (merchant.git) 724 - sync (sync.git) 725 - taler-mdb (taler-mdb.git) 726 - libeufin (libeufin.git) 727 - challenger (challenger.git) 728 - wallet-core (wallet-core.git) 729 730 Tagging 731 ------- 732 733 Tag releases with an **annotated** commit, like 734 735 .. code-block:: console 736 737 $ git tag -a v0.1.0 -m "Official release v0.1.0" 738 $ git push origin v0.1.0 739 740 741 Database for tests 742 ------------------ 743 744 For tests in the exchange and merchant to run, make sure that a database 745 *talercheck* is accessible by *$USER*. Otherwise tests involving the 746 database logic are skipped. 747 748 .. include:: ../frags/db-stores-sensitive-data.rst 749 750 Exchange, merchant 751 ------------------ 752 753 Set the version in ``configure.ac``. The commit being tagged should be 754 the change of the version. 755 756 Tag the current GANA version that works with the exchange and merchant and 757 checkout that tag of gana.git (instead of master). Otherwise, if there are 758 incompatible changes in GANA (like removed symbols), old builds could break. 759 760 Update the Texinfo documentation using the files from docs.git: 761 762 .. code-block:: console 763 764 # Get the latest documentation repository 765 $ cd $GIT/docs 766 $ git pull 767 $ make texinfo 768 # The *.texi files are now in _build/texinfo 769 # 770 # This checks out the prebuilt branch in the prebuilt directory 771 $ git worktree add prebuilt prebuilt 772 $ cd prebuilt 773 # Copy the pre-built documentation into the prebuilt directory 774 $ cp -r ../_build/texinfo . 775 # Push and commit to branch 776 $ git commit -a -S -m "updating texinfo" 777 $ git status 778 # Verify that all files that should be tracked are tracked, 779 # new files will have to be added to the Makefile.am in 780 # exchange.git as well! 781 $ git push 782 # Remember $REVISION of commit 783 # 784 # Go to exchange 785 $ cd $GIT/exchange/doc/prebuilt 786 # Update submodule to point to latest commit 787 $ git checkout $REVISION 788 789 Finally, the Automake ``Makefile.am`` files may have to be adjusted to 790 include new ``*.texi`` files or images. 791 792 For bootstrap, you will need to install 793 `GNU Recutils <https://www.gnu.org/software/recutils/>`_. 794 795 For the exchange test cases to pass, ``make install`` must be run first. 796 Without it, test cases will fail because plugins can't be located. 797 798 .. code-block:: console 799 800 $ ./bootstrap 801 $ ./configure # add required options for your system 802 $ make dist 803 $ tar -xf taler-$COMPONENT-$VERSION.tar.gz 804 $ cd taler-$COMPONENT-$VERSION 805 $ make install check 806 807 Wallet WebExtension 808 ------------------- 809 810 The version of the wallet is in *manifest.json*. The ``version_name`` 811 should be adjusted, and *version* should be increased independently on 812 every upload to the WebStore. 813 814 .. code-block:: console 815 816 $ ./configure 817 $ make dist 818 819 Upload to GNU mirrors 820 --------------------- 821 822 See https://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads 823 824 Directive file: 825 826 .. code-block:: none 827 828 version: 1.2 829 directory: taler 830 filename: taler-exchange-0.1.0.tar.gz 831 symlink: taler-exchange-0.1.0.tar.gz taler-exchange-latest.tar.gz 832 833 Upload the files in **binary mode** to the ftp servers. 834 835 836 Creating Debian packages 837 ------------------------ 838 839 Our general setup is based on 840 https://wiki.debian.org/DebianRepository/SetupWithReprepro 841 842 First, update at least the version of the Debian package in 843 debian/changelog, and then run: 844 845 .. code-block:: bash 846 847 $ dpkg-buildpackage -rfakeroot -b -uc -us 848 849 in the respective source directory (GNUnet, exchange, merchant) to create the 850 ``.deb`` files. Note that they will be created in the parent directory. This 851 can be done on gv.taler.net, or on another (secure) machine. 852 Actual release builds should be done via the Docker images 853 that can be found in ``deployment.git`` under packaging. 854 855 On ``gv``, we use the ``aptbuilder`` user to manage the reprepro repository. 856 857 Next, the ``*.deb`` files should be copied to gv.taler.net, say to 858 ``/home/aptbuilder/incoming``. Then, run 859 860 .. code-block:: bash 861 862 # cd /home/aptbuilder/apt 863 # reprepro includedeb bullseye ~/incoming/*.deb 864 865 to import all Debian files from ``~/incoming/`` into the ``bullseye`` 866 distribution. If Debian packages were build against other distributions, 867 reprepro may need to be first configured for those and the import command 868 updated accordingly. 869 870 Finally, make sure to clean up ``~/incoming/`` (by deleting the 871 now imported ``*.deb`` files). 872 873 874 875 Continuous integration 876 ====================== 877 878 CI is done with Buildbot (https://buildbot.net/), and builds are 879 triggered by the means of Git hooks. The results are published at 880 https://buildbot.taler.net/ . 881 882 In order to avoid downtimes, CI uses a "blue/green" deployment 883 technique. In detail, there are two users building code on the system, 884 the "green" and the "blue" user; and at any given time, one is running 885 Taler services and the other one is either building the code or waiting 886 for that. 887 888 There is also the possibility to trigger builds manually, but this is 889 only reserved to "admin" users. 890 891 892 Internationalisation 893 ==================== 894 895 Internationalisation (a.k.a "translation") is handled using text-based 896 localization files named PO (Portable Object) holding pairs of original and 897 translated strings. 898 899 .. include:: dictionary.rst 900 901 902 iOS Apps 903 ======== 904 905 .. _Build-iOS-from-source: 906 907 Building Taler Wallet for iOS from source 908 ----------------------------------------- 909 910 The GNU Taler Wallet iOS app is in 911 `the official Git repository <https://git.taler.net/taler-ios.git>`__. 912 913 Compatibility 914 ^^^^^^^^^^^^^ 915 916 The minimum version of iOS supported is 15.0. 917 This app runs on all iPhone models at least as new as the iPhone 6S. 918 919 920 Building 921 ^^^^^^^^ 922 923 Before building the iOS wallet, you must first checkout the 924 `quickjs-tart repo <https://git.taler.net/quickjs-tart.git>`__ 925 and the 926 `wallet-core repo <https://git.taler.net/wallet-core.git>`__. 927 928 Have all 3 local repos (wallet-core, quickjs-tart, and this one) adjacent at 929 the same level (e.g. in a "GNU_Taler" folder) 930 Taler.xcworkspace expects the QuickJS framework sub-project to be at 931 ``../quickjs-tart/QuickJS-rt.xcodeproj``. 932 933 Build wallet-core first: 934 935 .. code-block:: shell-session 936 937 $ cd wallet-core 938 $ make embedded 939 $ open packages/taler-wallet-embedded/dist 940 941 then drag or move its product "taler-wallet-core-qjs.mjs" 942 into your quickjs-tart folder right at the top level. 943 944 Open Taler.xcworkspace, and set scheme / target to Taler_Wallet. Build&run... 945 946 Don't open QuickJS-rt.xcodeproj or TalerWallet.xcodeproj and build anything 947 there - all needed libraries and frameworks will be built automatically from 948 Taler.xcworkspace. 949 950 951 Android Apps 952 ============ 953 954 Android App Nightly Builds 955 -------------------------- 956 957 There are currently three Android apps in 958 `the official Git repository <https://git.taler.net/taler-android.git>`__: 959 960 * Wallet 961 [`CI <https://git.taler.net/taler-android.git/tree/wallet/.gitlab-ci.yml>`__] 962 * Merchant PoS Terminal 963 [`CI <https://git.taler.net/taler-android.git/tree/merchant-terminal/.gitlab-ci.yml>`__] 964 * Cashier 965 [`CI <https://git.taler.net/taler-android.git/tree/cashier/.gitlab-ci.yml>`__] 966 967 Their git repositories are `mirrored at Gitlab <https://gitlab.com/gnu-taler/taler-android>`__ 968 to utilize their CI 969 and `F-Droid <https://f-droid.org>`_'s Gitlab integration 970 to `publish automatic nightly builds <https://f-droid.org/docs/Publishing_Nightly_Builds/>`_ 971 for each change on the ``master`` branch. 972 973 All three apps publish their builds to the same F-Droid nightly repository 974 (which is stored as a git repository): 975 https://gitlab.com/gnu-taler/fdroid-repo-nightly 976 977 You can download the APK files directly from that repository 978 or add it to the F-Droid app for automatic updates 979 by clicking the following link (on the phone that has F-Droid installed). 980 981 `GNU Taler Nightly F-Droid Repository <fdroidrepos://gnu-taler.gitlab.io/fdroid-repo-nightly/fdroid/repo?fingerprint=55F8A24F97FAB7B0960016AF393B7E57E7A0B13C2D2D36BAC50E1205923A7843>`_ 982 983 .. note:: 984 Nightly apps can be installed alongside official releases 985 and thus are meant **only for testing purposes**. 986 Use at your own risk! 987 988 .. _Build-apps-from-source: 989 990 Building apps from source 991 ------------------------- 992 993 Note that this guide is different from other guides for building Android apps, 994 because it does not require you to run non-free software. 995 It uses the Merchant PoS Terminal as an example, but works as well for the other apps 996 if you replace ``merchant-terminal`` with ``wallet`` or ``cashier``. 997 998 First, ensure that you have the required dependencies installed: 999 1000 * Java Development Kit 8 or higher (default-jdk-headless) 1001 * git 1002 * unzip 1003 1004 Then you can get the app's source code using git: 1005 1006 .. code-block:: console 1007 1008 # Start by cloning the Android git repository 1009 $ git clone https://git.taler.net/taler-android.git 1010 1011 # Change into the directory of the cloned repository 1012 $ cd taler-android 1013 1014 # Find out which Android SDK version you will need 1015 $ grep -i compileSdkVersion merchant-terminal/build.gradle 1016 1017 The last command will return something like ``compileSdkVersion 29``. 1018 So visit the `Android Rebuilds <http://android-rebuilds.beuc.net/>`_ project 1019 and look for that version of the Android SDK there. 1020 If the SDK version is not yet available as a free rebuild, 1021 you can try to lower the ``compileSdkVersion`` in the app's ``merchant-terminal/build.gradle`` file. 1022 Note that this might break things 1023 or require you to also lower other versions such as ``targetSdkVersion``. 1024 1025 In our example, the version is ``29`` which is available, 1026 so download the "SDK Platform" package of "Android 10.0.0 (API 29)" 1027 and unpack it: 1028 1029 .. code-block:: console 1030 1031 # Change into the directory that contains your downloaded SDK 1032 $ cd $HOME 1033 1034 # Unpack/extract the Android SDK 1035 $ unzip android-sdk_eng.10.0.0_r14_linux-x86.zip 1036 1037 # Tell the build system where to find the SDK 1038 $ export ANDROID_SDK_ROOT="$HOME/android-sdk_eng.10.0.0_r14_linux-x86" 1039 1040 # Change into the directory of the cloned repository 1041 $ cd taler-android 1042 1043 # Build the merchant-terminal app 1044 $ ./gradlew :merchant-terminal:assembleRelease 1045 1046 If you get an error message complaining about build-tools 1047 1048 > Failed to install the following Android SDK packages as some licences have not been accepted. 1049 build-tools;29.0.3 Android SDK Build-Tools 29.0.3 1050 1051 you can try changing the ``buildToolsVersion`` in the app's ``merchant-terminal/build.gradle`` file 1052 to the latest "Android SDK build tools" version supported by the Android Rebuilds project. 1053 1054 After the build finished successfully, 1055 you will find your APK in ``merchant-terminal/build/outputs/apk/release/``. 1056 1057 Update translations 1058 ------------------- 1059 1060 Translations are managed with Taler's weblate instance: 1061 https://weblate.taler.net/projects/gnu-taler/ 1062 1063 To update translations, enter the taler-android git repository 1064 and ensure that the weblate remote exists: 1065 1066 .. code-block:: console 1067 1068 $ git config -l | grep weblate 1069 1070 If it does not yet exist (empty output), you can add it like this: 1071 1072 .. code-block:: console 1073 1074 $ git remote add weblate https://weblate.taler.net/git/gnu-taler/wallet-android/ 1075 1076 Then you can merge in translations commit from the weblate remote: 1077 1078 .. code-block:: console 1079 1080 # ensure you have latest version 1081 $ git fetch weblate 1082 1083 # merge in translation commits 1084 $ git merge weblate/master 1085 1086 Afterwards, build the entire project from source and test the UI 1087 to ensure that no erroneous translations (missing placeholders) are breaking things. 1088 1089 Release process 1090 --------------- 1091 1092 After extensive testing, the code making up a new release should get a signed git tag. 1093 The current tag format is: 1094 1095 * cashier-$VERSION 1096 * pos-$VERSION 1097 * wallet-$VERSION (where $VERSION has a v prefix) 1098 1099 .. code-block:: console 1100 1101 $ git tag -s $APP-$VERSION 1102 1103 F-Droid 1104 ^^^^^^^ 1105 Nightly builds get published automatically (see above) after pushing code to the official repo. 1106 Actual releases get picked up by F-Droid's official repository via git tags. 1107 So ensure that all releases get tagged properly. 1108 1109 Some information for F-Droid official repository debugging: 1110 1111 * Wallet: [`metadata <https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.wallet.fdroid.yml>`__] [`build log <https://f-droid.org/wiki/page/net.taler.wallet.fdroid/lastbuild>`__] 1112 * Cashier: [`metadata <https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.cashier.yml>`__] [`build log <https://f-droid.org/wiki/page/net.taler.cashier/lastbuild>`__] 1113 * PoS: [`metadata <https://gitlab.com/fdroid/fdroiddata/-/blob/master/metadata/net.taler.merchantpos.yml>`__] [`build log <https://f-droid.org/wiki/page/net.taler.merchantpos/lastbuild>`__] 1114 1115 Google Play 1116 ^^^^^^^^^^^ 1117 Google Play uploads are managed via `Fastlane <https://docs.fastlane.tools/getting-started/android/setup/>`__. 1118 Before proceeding, ensure that this is properly set up 1119 and that you have access to the Google Play API. 1120 1121 It is important to have access to the signing keys and Google Play access keys 1122 (JSON) and to ensure that the following environment variables are set 1123 correctly and made available to Fastlane: 1124 1125 .. code-block:: bash 1126 1127 TALER_KEYSTORE_PATH= 1128 TALER_KEYSTORE_PASS= 1129 TALER_KEYSTORE_WALLET_ALIAS= 1130 TALER_KEYSTORE_WALLET_PASS= 1131 TALER_KEYSTORE_POS_ALIAS= 1132 TALER_KEYSTORE_POS_PASS= 1133 TALER_KEYSTORE_CASHIER_ALIAS= 1134 TALER_KEYSTORE_CASHIER_PASS= 1135 TALER_JSON_KEY_FILE= 1136 1137 To release an app, enter into its respective folder and run fastlane: 1138 1139 .. code-block:: console 1140 1141 $ bundle exec fastlane 1142 1143 Then select the deploy option. 1144 1145 All uploads are going to the beta track by default. These can be promoted to 1146 production later or immediately after upload if you feel daring. It is also 1147 important to bump the version and build code with every release. 1148 1149 .. _Code-coverage: 1150 1151 Code Coverage 1152 ============= 1153 1154 Code coverage is done with the Gcov / Lcov 1155 (http://ltp.sourceforge.net/coverage/lcov.php) combo, and it is run 1156 nightly (once a day) by a Buildbot worker. The coverage results are 1157 then published at https://lcov.taler.net/ . 1158 1159 1160 Coding Conventions 1161 ================== 1162 1163 GNU Taler is developed primarily in C, Kotlin, Python, Swift and TypeScript. 1164 1165 Components written in C 1166 ----------------------- 1167 1168 These are the general coding style rules for Taler. 1169 1170 * Baseline rules are to follow GNU guidelines, modified or extended 1171 by the GNUnet style: https://docs.gnunet.org/handbook/gnunet.html#Coding-style 1172 1173 Naming conventions 1174 ^^^^^^^^^^^^^^^^^^ 1175 1176 * include files (very similar to GNUnet): 1177 1178 * if installed, must start with "``taler_``" (exception: platform.h), 1179 and MUST live in src/include/ 1180 * if NOT installed, must NOT start with "``taler_``" and 1181 MUST NOT live in src/include/ and 1182 SHOULD NOT be included from outside of their own directory 1183 * end in "_lib" for "simple" libraries 1184 * end in "_plugin" for plugins 1185 * end in "_service" for libraries accessing a service, i.e. the exchange 1186 1187 * binaries: 1188 1189 * taler-exchange-xxx: exchange programs 1190 * taler-merchant-xxx: merchant programs (demos) 1191 * taler-wallet-xxx: wallet programs 1192 * plugins should be libtaler_plugin_xxx_yyy.so: plugin yyy for API xxx 1193 * libtalerxxx: library for API xxx 1194 1195 * logging 1196 1197 * tools use their full name in GNUNET_log_setup 1198 (i.e. 'taler-exchange-offline') and log using plain 'GNUNET_log'. 1199 * pure libraries (without associated service) use 'GNUNET_log_from' 1200 with the component set to their library name (without lib or '.so'), 1201 which should also be their directory name (i.e. 'util') 1202 * plugin libraries (without associated service) use 'GNUNET_log_from' 1203 with the component set to their type and plugin name (without lib or '.so'), 1204 which should also be their directory name (i.e. 'exchangedb-postgres') 1205 * libraries with associated service) use 'GNUNET_log_from' 1206 with the name of the service, which should also be their 1207 directory name (i.e. 'exchange') 1208 * for tools with ``-l LOGFILE``, its absence means write logs to stderr 1209 1210 * configuration 1211 1212 * same rules as for GNUnet 1213 1214 * exported symbols 1215 1216 * must start with TALER_[SUBSYSTEMNAME]_ where SUBSYSTEMNAME 1217 MUST match the subdirectory of src/ in which the symbol is defined 1218 * from libtalerutil start just with ``TALER_``, without subsystemname 1219 * if scope is ONE binary and symbols are not in a shared library, 1220 use binary-specific prefix (such as TMH = taler-exchange-httpd) for 1221 globals, possibly followed by the subsystem (TMH_DB_xxx). 1222 1223 * structs: 1224 1225 * structs that are 'packed' and do not contain pointers and are 1226 thus suitable for hashing or similar operations are distinguished 1227 by adding a "P" at the end of the name. (NEW) Note that this 1228 convention does not hold for the GNUnet-structs (yet). 1229 * structs that are used with a purpose for signatures, additionally 1230 get an "S" at the end of the name. 1231 1232 * private (library-internal) symbols (including structs and macros) 1233 1234 * must not start with ``TALER_`` or any other prefix 1235 1236 * testcases 1237 1238 * must be called "test_module-under-test_case-description.c" 1239 1240 * performance tests 1241 1242 * must be called "perf_module-under-test_case-description.c" 1243 1244 Shell Scripts 1245 ------------- 1246 1247 Shell scripts should be avoided if at all possible. The only permissible uses of shell scripts 1248 in GNU Taler are: 1249 1250 * Trivial invocation of other commands. 1251 * Scripts for compatibility (e.g. ``./configure``) that must run on 1252 as many systems as possible. 1253 1254 When shell scripts are used, they ``MUST`` begin with the following ``set`` command: 1255 1256 .. code-block:: console 1257 1258 # Make the shell fail on undefined variables and 1259 # commands with non-zero exit status. 1260 $ set -eu 1261 1262 Kotlin 1263 ------ 1264 1265 We so far have no specific guidelines, please follow best practices 1266 for the language. 1267 1268 1269 Python 1270 ------ 1271 1272 Supported Python Versions 1273 ^^^^^^^^^^^^^^^^^^^^^^^^^ 1274 1275 Python code should be written and built against version 3.7 of Python. 1276 1277 Style 1278 ^^^^^ 1279 1280 We use `yapf <https://github.com/google/yapf>`_ to reformat the 1281 code to conform to our style instructions. 1282 A reusable yapf style file can be found in ``build-common``, 1283 which is intended to be used as a git submodule. 1284 1285 Python for Scripting 1286 ^^^^^^^^^^^^^^^^^^^^ 1287 1288 When using Python for writing small utilities, the following libraries 1289 are useful: 1290 1291 * ``click`` for argument parsing (should be preferred over argparse) 1292 * ``pathlib`` for path manipulation (part of the standard library) 1293 * ``subprocess`` for "shelling out" to other programs. Prefer ``subprocess.run`` 1294 over the older APIs. 1295 1296 1297 Swift 1298 ----- 1299 1300 Please follow best practices for the language. 1301 1302 1303 TypeScript 1304 ---------- 1305 1306 Please follow best practices for the language. 1307 1308 1309 Testing library 1310 =============== 1311 1312 This chapter is a VERY ABSTRACT description of how testing is 1313 implemented in Taler, and in NO WAY wants to substitute the reading of 1314 the actual source code by the user. 1315 1316 In Taler, a test case is an array of ``struct TALER_TESTING_Command``, 1317 informally referred to as ``CMD``, that is iteratively executed by the 1318 testing interpreter. This latter is transparently initiated by the 1319 testing library. 1320 1321 However, the developer does not have to define CMDs manually, but 1322 rather call the proper constructor provided by the library. For example, 1323 if a CMD is supposed to test feature ``x``, then the library would 1324 provide the ``TALER_TESTING_cmd_x ()`` constructor for it. Obviously, 1325 each constructor has its own particular arguments that make sense to 1326 test ``x``, and all constructors are thoroughly commented within the 1327 source code. 1328 1329 Internally, each CMD has two methods: ``run ()`` and ``cleanup ()``. The 1330 former contains the main logic to test feature ``x``, whereas the latter 1331 cleans the memory up after execution. 1332 1333 In a test life, each CMD needs some internal state, made by values it 1334 keeps in memory. Often, the test has to *share* those values with other 1335 CMDs: for example, CMD1 may create some key material and CMD2 needs this 1336 key material to encrypt data. 1337 1338 The offering of internal values from CMD1 to CMD2 is made by *traits*. A 1339 trait is a ``struct TALER_TESTING_Trait``, and each CMD contains an array 1340 of traits, that it offers via the public trait interface to other 1341 commands. The definition and filling of such array happens transparently 1342 to the test developer. 1343 1344 For example, the following example shows how CMD2 takes an amount object 1345 offered by CMD1 via the trait interface. 1346 1347 Note: the main interpreter and the most part of CMDs and traits are 1348 hosted inside the exchange codebase, but nothing prevents the developer 1349 from implementing new CMDs and traits within other codebases. 1350 1351 .. code-block:: c 1352 1353 /* Without loss of generality, let's consider the 1354 * following logic to exist inside the run() method of CMD1 */ 1355 ... 1356 1357 struct TALER_Amount *a; 1358 /** 1359 * the second argument (0) points to the first amount object offered, 1360 * in case multiple are available. 1361 */ 1362 if (GNUNET_OK != TALER_TESTING_get_trait_amount_obj (cmd2, 0, &a)) 1363 return GNUNET_SYSERR; 1364 ... 1365 1366 use(a); /* 'a' points straight into the internal state of CMD2 */ 1367 1368 In the Taler realm, there is also the possibility to alter the behaviour 1369 of supposedly well-behaved components. This is needed when, for example, 1370 we want the exchange to return some corrupted signature in order to 1371 check if the merchant backend detects it. 1372 1373 This alteration is accomplished by another service called *twister*. The 1374 twister acts as a proxy between service A and B, and can be programmed 1375 to tamper with the data exchanged by A and B. 1376 1377 Please refer to the Twister codebase (under the ``test`` directory) in 1378 order to see how to configure it. 1379 1380 1381 User-Facing Terminology 1382 ======================= 1383 1384 This section contains terminology that should be used and that should not be 1385 used in the user interface and help materials. 1386 1387 Terms to Avoid 1388 -------------- 1389 1390 Refreshing 1391 Refreshing is the internal technical terminology for the protocol to 1392 give change for partially spent coins 1393 1394 **Use instead**: "Obtaining change" 1395 1396 Charge 1397 Charge has two opposite meanings (charge to a credit card vs. charge a battery). 1398 This can confuse users. 1399 1400 **Use instead**: "Obtain", "Credit", "Debit", "Withdraw", "Top up" 1401 1402 Coin 1403 Coins are an internal construct, the user should never be aware that their balance 1404 is represented by coins of different denominations. 1405 1406 **Use instead**: "(Digital) Cash" or "(Wallet) Balance" 1407 1408 Consumer 1409 Has bad connotation of consumption. 1410 1411 **Use instead**: Customer or user. 1412 1413 Proposal 1414 The term used to describe the process of the merchant facilitating the download 1415 of the signed contract terms for an order. 1416 1417 **Avoid**. Generally events that relate to proposal downloads 1418 should not be shown to normal users, only developers. Instead, use 1419 "communication with merchant failed" if a proposed order can't be downloaded. 1420 1421 Anonymous E-Cash 1422 Should be generally avoided, since Taler is only anonymous for 1423 the customer. Also some people are scared of anonymity (which as 1424 a term is also way too absolute, as anonymity is hardly ever perfect). 1425 1426 **Use instead**: "Privacy-preserving", "Privacy-friendly" 1427 1428 Payment Replay 1429 The process of proving to the merchant that the customer is entitled 1430 to view a digital product again, as they already paid for it. 1431 1432 **Use instead**: In the event history, "re-activated digital content purchase" 1433 could be used. (FIXME: this is still not nice.) 1434 1435 Session ID 1436 See Payment Replay. 1437 1438 Order 1439 Too ambiguous in the wallet. 1440 1441 **Use instead**: Purchase 1442 1443 Fulfillment URL 1444 URL that serves the digital content that the user purchased 1445 with their payment. Can also be something like a donation receipt. 1446 1447 Donau 1448 Developer-internal name for the tax authority component. 1449 1450 **Use instead**: Tax authority 1451 1452 Terms to Use 1453 ------------ 1454 1455 Auditor 1456 Regulatory entity that certifies exchanges and oversees their operation. 1457 1458 Exchange Operator 1459 The entity/service that gives out digital cash in exchange for some 1460 other means of payment. 1461 1462 In some contexts, using "Issuer" could also be appropriate. 1463 When showing a balance breakdown, 1464 we can say "100 Eur (issued by exchange.euro.taler.net)". 1465 Sometimes we may also use the more generic term "Payment Service Provider" 1466 when the concept of an "Exchange" is still unclear to the reader. 1467 1468 Refund 1469 A refund is given by a merchant to the customer (rather the customer's wallet) 1470 and "undoes" a previous payment operation. 1471 1472 Payment 1473 The act of sending digital cash to a merchant to pay for an order. 1474 1475 Purchase 1476 Used to refer to the "result" of a payment, as in "view purchase". 1477 Use sparingly, as the word doesn't fit for all payments, such as donations. 1478 1479 Contract Terms 1480 Partially machine-readable representation of the merchant's obligation after the 1481 customer makes a payment. 1482 1483 Merchant 1484 Party that receives a payment. 1485 1486 Wallet 1487 Also "Taler Wallet". Software component that manages the user's digital cash 1488 and payments. 1489 1490 1491 Developer Glossary 1492 ================== 1493 1494 This glossary is meant for developers. It contains some terms that we usually do not 1495 use when talking to end users or even system administrators. 1496 1497 .. glossary:: 1498 :sorted: 1499 1500 absolute time 1501 method of keeping time in :term:`GNUnet` where the time is represented 1502 as the number of microseconds since 1.1.1970 (UNIX epoch). Called 1503 absolute time in contrast to :term:`relative time`. 1504 1505 aggregate 1506 the :term:`exchange` combines multiple payments received by the 1507 same :term:`merchant` into one larger :term:`wire transfer` to 1508 the respective merchant's :term:`bank` account 1509 1510 auditor 1511 trusted third party that verifies that the :term:`exchange` is operating correctly 1512 1513 bank 1514 traditional financial service provider who offers 1515 :term:`wire transfers <wire transfer>` between accounts 1516 1517 buyer 1518 individual in control of a Taler :term:`wallet`, usually using it to 1519 :term:`spend` the :term:`coins <coin>` on :term:`contracts <contract>` (see also :term:`customer`). 1520 1521 close 1522 operation an :term:`exchange` performs on a :term:`reserve` that has not been 1523 :term:`emptied <empty>` by :term:`withdraw` operations. When closing a reserve, the 1524 exchange wires the remaining funds back to the customer, minus a :term:`fee` 1525 for closing 1526 1527 customer 1528 individual that directs the buyer (perhaps the same individual) to make a purchase 1529 1530 coin 1531 coins are individual tokens representing a certain amount of value, also known as the :term:`denomination` of the coin 1532 1533 refresh commitment 1534 data that the wallet commits to during the :term:`melt` stage of the 1535 :term:`refresh` protocol where it 1536 has to prove to the :term:`exchange` that it is deriving the :term:`fresh` 1537 coins as specified by the Taler protocol. The commitment is verified 1538 probabilistically (see: :term:`kappa`) during the :term:`reveal` stage. 1539 1540 contract 1541 formal agreement between :term:`merchant` and :term:`customer` specifying the 1542 :term:`contract terms` and signed by the merchant and the :term:`coins <coin>` of the 1543 customer 1544 1545 contract terms 1546 the individual clauses specifying what the buyer is purchasing from the 1547 :term:`merchant` 1548 1549 denomination 1550 unit of currency, specifies both the currency and the face value of a :term:`coin`, 1551 as well as associated fees and validity periods 1552 1553 denomination key 1554 (RSA) key used by the exchange to certify that a given :term:`coin` is valid and of a 1555 particular :term:`denomination` 1556 1557 deposit 1558 operation by which a merchant passes coins to an exchange, expecting the 1559 exchange to credit his bank account in the future using an 1560 :term:`aggregate` :term:`wire transfer` 1561 1562 drain 1563 process by which an exchange operator takes the profits 1564 (from :term:`fees <fee>`) out of the escrow account and moves them into 1565 their regular business account 1566 1567 dirty 1568 a :term:`coin` is dirty if its public key may be known to an entity other than 1569 the customer, thereby creating the danger of some entity being able to 1570 link multiple transactions of coin's owner if the coin is not refreshed 1571 1572 empty 1573 a :term:`reserve` is being emptied when a :term:`wallet` is using the 1574 reserve's private key to :term:`withdraw` coins from it. This reduces 1575 the balance of the reserve. Once the balance reaches zero, we say that 1576 the reserve has been (fully) emptied. Reserves that are not emptied 1577 (which is the normal process) are :term:`closed <close>` by the exchange. 1578 1579 exchange 1580 Taler's payment service operator. Issues electronic coins during 1581 withdrawal and redeems them when they are deposited by merchants 1582 1583 expired 1584 Various operations come with time limits. In particular, denomination keys 1585 come with strict time limits for the various operations involving the 1586 coin issued under the denomination. The most important limit is the 1587 deposit expiration, which specifies until when wallets are allowed to 1588 use the coin in deposit or refreshing operations. There is also a "legal" 1589 expiration, which specifies how long the exchange keeps records beyond the 1590 deposit expiration time. This latter expiration matters for legal disputes 1591 in courts and also creates an upper limit for refreshing operations on 1592 special zombie coin 1593 1594 GNUnet 1595 Codebase of various libraries for a better Internet, some of which 1596 GNU Taler depends upon. 1597 1598 fakebank 1599 implementation of the :term:`bank` API in memory to be used only for test 1600 cases. 1601 1602 fee 1603 an :term:`exchange` charges various fees for its service. The different 1604 fees are specified in the protocol. There are fees per coin for 1605 :term:`withdrawing <withdraw>`, :term:`depositing <deposit>`, :term:`melting <melt>`, and 1606 :term:`refunding <refund>`. Furthermore, there are fees per wire transfer 1607 when a :term:`reserve` is :term:`closed <close>` 1608 and for :term:`aggregate` :term:`wire transfers <wire transfer>` 1609 to the :term:`merchant`. 1610 1611 fresh 1612 a :term:`coin` is fresh if its public key is only known to the customer 1613 1614 JSON 1615 JavaScript Object Notation (JSON) is a 1616 serialization format derived from the JavaScript language which is 1617 commonly used in the Taler protocol as the payload of HTTP requests 1618 and responses. 1619 1620 kappa 1621 security parameter used in the :term:`refresh` protocol. Defined to be 3. 1622 The probability of successfully evading the income transparency with the 1623 refresh protocol is 1:kappa. 1624 1625 libeufin 1626 Kotlin component that implements a regional currency bank and an 1627 adapter to communicate via EBICS with European core banking systems. 1628 1629 link 1630 specific step in the :term:`refresh` protocol that an exchange must offer 1631 to prevent abuse of the :term:`refresh` mechanism. The link step is 1632 not needed in normal operation, it just must be offered. 1633 1634 master key 1635 offline key used by the exchange to certify denomination keys and 1636 message signing keys 1637 1638 melt 1639 step of the :term:`refresh` protocol where a :term:`dirty` :term:`coin` 1640 is invalidated to be reborn :term:`fresh` in a subsequent 1641 :term:`reveal` step. 1642 1643 merchant 1644 party receiving payments (usually in return for goods or services) 1645 1646 message signing key 1647 key used by the exchange to sign online messages, other than coins 1648 1649 order 1650 offer made by the merchant to a wallet; pre-cursor to 1651 a contract where the wallet is not yet fixed. Turns 1652 into a :term:`contract` when a wallet claims the order. 1653 1654 owner 1655 a coin is owned by the entity that knows the private key of the coin 1656 1657 relative time 1658 method of keeping time in :term:`GNUnet` where the time is represented 1659 as a relative number of microseconds. Thus, a relative time specifies 1660 an offset or a duration, but not a date. Called relative time in 1661 contrast to :term:`absolute time`. 1662 1663 recoup 1664 Operation by which an exchange returns the value of coins affected 1665 by a :term:`revocation <revoke>` to their :term:`owner`, either by allowing the owner to 1666 withdraw new coins or wiring funds back to the bank account of the :term:`owner`. 1667 1668 planchet 1669 precursor data for a :term:`coin`. A planchet includes the coin's internal 1670 secrets (coin private key, blinding factor), but lacks the RSA signature 1671 of the :term:`exchange`. When :term:`withdrawing <withdraw>`, a :term:`wallet` 1672 creates and persists a planchet before asking the exchange to sign it to 1673 get the coin. 1674 1675 purchase 1676 Refers to the overall process of negotiating a :term:`contract` and then 1677 making a payment with :term:`coins <coin>` to a :term:`merchant`. 1678 1679 privacy policy 1680 Statement of an operator how they will protect the privacy of users. 1681 1682 proof 1683 Message that cryptographically demonstrates that a particular claim is correct. 1684 1685 proposal 1686 a list of :term:`contract terms` that has been completed and signed by the 1687 merchant backend. 1688 1689 refresh 1690 operation by which a :term:`dirty` :term:`coin` is converted into one or more 1691 :term:`fresh` coins. Involves :term:`melting <melt>` the :term:`dirty` coins and 1692 then :term:`revealing <reveal>` so-called :term:`transfer keys <transfer key>`. 1693 1694 refund 1695 operation by which a merchant steps back from the right to funds that he 1696 obtained from a :term:`deposit` operation, giving the right to the funds back 1697 to the customer 1698 1699 refund transaction id 1700 unique number by which a merchant identifies a :term:`refund`. Needed 1701 as refunds can be partial and thus there could be multiple refunds for 1702 the same :term:`purchase`. 1703 1704 reserve 1705 accounting mechanism used by the exchange to track customer funds 1706 from incoming :term:`wire transfers <wire transfer>`. A reserve is created whenever 1707 a customer wires money to the exchange using a well-formed public key 1708 in the subject. The exchange then allows the customer's :term:`wallet` 1709 to :term:`withdraw` up to the amount received in :term:`fresh` 1710 :term:`coins <coin>` from the reserve, thereby emptying the reserve. If a 1711 reserve is not emptied, the exchange will eventually :term:`close` it. 1712 1713 Other definition: Funds set aside for future use; either the balance of a customer at the 1714 exchange ready for withdrawal, or the funds kept in the exchange;s bank 1715 account to cover obligations from coins in circulation. 1716 1717 reveal 1718 step in the :term:`refresh` protocol where some of the transfer private 1719 keys are revealed to prove honest behavior on the part of the wallet. 1720 In the reveal step, the exchange returns the signed :term:`fresh` coins. 1721 1722 revoke 1723 exceptional operation by which an exchange withdraws a denomination from 1724 circulation, either because the signing key was compromised or because 1725 the exchange is going out of operation; unspent coins of a revoked 1726 denomination are subjected to recoup. 1727 1728 sharing 1729 users can share ownership of a :term:`coin` by sharing access to the coin's 1730 private key, thereby allowing all co-owners to spend the coin at any 1731 time. 1732 1733 spend 1734 operation by which a customer gives a merchant the right to deposit 1735 coins in return for merchandise 1736 1737 transfer key 1738 special cryptographic key used in the :term:`refresh` protocol, some of which 1739 are revealed during the :term:`reveal` step. Note that transfer keys have, 1740 despite the name, no relationship to :term:`wire transfers <wire transfer>`. They merely 1741 help to transfer the value from a :term:`dirty` coin to a :term:`fresh` coin 1742 1743 terms 1744 the general terms of service of an operator, possibly including 1745 the :term:`privacy policy`. Not to be confused with the 1746 :term:`contract terms` which are about the specific purchase. 1747 1748 transaction 1749 method by which ownership is exclusively transferred from one entity 1750 1751 user 1752 any individual using the Taler payment system 1753 (see :term:`customer`, :term:`buyer`, :term:`merchant`). 1754 1755 version 1756 Taler uses various forms of versioning. There is a database 1757 schema version (stored itself in the database, see \*-0000.sql) describing 1758 the state of the table structure in the database of an :term:`exchange`, 1759 :term:`auditor` or :term:`merchant`. There is a protocol 1760 version (CURRENT:REVISION:AGE, see GNU libtool) which specifies 1761 the network protocol spoken by an :term:`exchange` or :term:`merchant` 1762 including backwards-compatibility. And finally there is the software 1763 release version (MAJOR.MINOR.PATCH, see https://semver.org/) of 1764 the respective code base. 1765 1766 wallet 1767 software running on a customer's computer; withdraws, stores and 1768 spends coins 1769 1770 WebExtension 1771 Cross-browser API used to implement the GNU Taler wallet browser extension. 1772 1773 wire gateway 1774 API used by the exchange to talk with some real-time gross settlement system 1775 (core banking system, blockchain) to notice inbound credits wire transfers 1776 (during withdraw) and to trigger outbound debit wire transfers (primarily 1777 for deposits). 1778 1779 wire transfer 1780 a wire transfer is a method of sending funds between :term:`bank` accounts 1781 1782 wire transfer identifier 1783 Subject of a wire transfer from the exchange to a merchant; 1784 set by the aggregator to a random nonce which uniquely 1785 identifies the transfer. 1786 1787 withdraw 1788 operation by which a :term:`wallet` can convert funds from a :term:`reserve` to 1789 fresh coins 1790 1791 zombie 1792 :term:`coin` where the respective :term:`denomination key` is past its 1793 :term:`deposit` :term:`expiration <expired>` time, but which is still (again) valid 1794 for an operation because it was :term:`melted <melt>` while it was still 1795 valid, and then later again credited during a :term:`recoup` process 1796 1797 1798 1799 Developer Tools 1800 =============== 1801 1802 This section describes various internal programs to make life easier for the 1803 developer. 1804 1805 1806 taler-harness 1807 ------------- 1808 1809 **taler-harness deployment gen-coin-config** is a tool to simplify Taler configuration generation. 1810 1811 1812 **taler-harness deployment gen-coin-config** 1813 [**-min-amount**=**\ \ *VALUE*] 1814 [**-max-amount**=**\ \ *VALUE*]