ebisync-manual.rst (12666B)
1 .. 2 This file is part of GNU TALER. 3 Copyright (C) 2025 Taler Systems SA 4 5 TALER is free software; you can redistribute it and/or modify it under the 6 terms of the GNU Affero General Public License as published by the Free Software 7 Foundation; either version 3.0, or (at your option) any later version. 8 9 TALER is distributed in the hope that it will be useful, but WITHOUT ANY 10 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR 11 A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. 12 13 You should have received a copy of the GNU Affero General Public License along with 14 TALER; see the file COPYING. If not, see <http://www.gnu.org/licenses/> 15 16 @author Antoine d'Aligny 17 18 .. target audience: operator, developer 19 20 .. _libeufin-ebisync: 21 22 EbiSync Manual 23 ############## 24 25 LibEuFin EbiSync is an EBICS synchronization tool. It offers a command line interface to setup EBICS access and download & submit ISO20022 files. 26 27 It addresses the problem that interacting with banks over EBICS requires 28 implementing a complex cryptographic handshake to download transaction data 29 (CAMT files) or to upload instructions for payment initiation (PAIN files). 30 With LibEuFin EbiSync, existing systems can use LibEuFin to talk with 31 core banking systems over EBICS without having to implement EBICS or the 32 :ref:`Taler Wire Gateway API <taler-wire-gateway-http-api>` themselves. 33 Instead, existing systems can simply use an HTTP POST request to submit 34 PAIN files. Incoming CAMT files can be automatically placed into 35 Cloud BLOB storage and processed from there. 36 37 .. image:: ../images/libeufin-ebisync.png 38 39 Today, the LibEuFin implementation supports EBICS 3.0 and has been tested with 40 Postfinance (CHF), GLS (EUR), Raiffeisen (CHF), Maerki Baumann (CHF) and Valiant (CHF). Contact 41 us if you need support for another bank. 42 43 In this manual, we explain how to setup an EBICS subscriber. We assume that 44 the bank has already granted EBICS access to the subscriber. 45 46 Installing EbiSync 47 ================== 48 49 The following section was tested on an *OpenJDK 17* environment. 50 51 52 Installing the libeufin-ebisync binary packages on Debian 53 --------------------------------------------------------- 54 55 .. include:: ../frags/installing-debian.rst 56 57 58 To install libeufin-ebisync, you can now simply run: 59 60 .. code-block:: console 61 62 # apt install libeufin-ebisync 63 64 Installing the libeufin-ebisync binary packages on Ubuntu 65 --------------------------------------------------------- 66 67 .. include:: ../frags/installing-ubuntu.rst 68 69 To install libeufin-ebisync, you can now simply run: 70 71 .. code-block:: console 72 73 # apt install libeufin-ebisync 74 75 76 Building from source 77 -------------------- 78 79 Ebisync belongs to the LibEuFin project, and can be downloaded via Git: 80 81 .. code-block:: console 82 83 $ git clone git://git.taler.net/libeufin 84 85 Note that Kotlin and Gradle should already work on the host system. 86 87 Navigate into the *libeufin* local repository, and from top-level run: 88 89 .. code-block:: console 90 91 $ ./bootstrap 92 93 Install deb package 94 ~~~~~~~~~~~~~~~~~~~ 95 96 To install EbiSync as a Debian/Ubuntu package with an automated secure setup and systemd services: 97 98 .. code-block:: console 99 100 $ sudo apt install debhelper 101 $ make deb 102 $ sudo dpkg -i ../libeufin-ebisync*.deb 103 104 If the previous steps succeeded, the ``libeufin-ebisync`` command should be found in the $PATH. 105 106 107 Services, users, groups and file system hierarchy 108 ------------------------------------------------- 109 110 The *libeufin-ebisync* package will create several system users 111 to compartmentalize different parts of the system: 112 113 * ``libeufin-ebisync``: runs all LibEuFin EbiSync components. 114 * ``postgres``: runs the PostgreSQL database (from *postgresql* package). 115 116 The package will deploy systemd service files in 117 ``/usr/lib/systemd/system/`` for the various components: 118 119 * ``libeufin-ebisync-httpd.service``: http server for the Sync API. 120 * ``libeufin-ebisync-fetch.service``: worker daemon that retrieves files via EBICS and uploads them to a destination. 121 * ``libeufin-ebisync.target``: main target for EbiSync to be operational. 122 123 The deployment creates the following key locations in the system: 124 125 * ``/etc/libeufin-ebisync/``: configuration files. 126 * ``/run/libeufin-ebisync/``: contains the UNIX domain sockets for inter-process communication (IPC). 127 * ``/var/lib/libeufin-ebisync/``: serves as the $HOME for libeufin-ebisync and contains the EBICS authentication keys. 128 129 Database setup 130 ============== 131 132 The configuration file must include a connection string that 133 tells EbiSync how it should connect to the database. The default 134 is: 135 136 .. code-block:: ini 137 :caption: /etc/libeufin-ebisync/secrets/ebisync-db.secret.conf 138 139 [ebisyncdb-postgres] 140 config = postgres:///libeufin-ebisync 141 142 For a local PostgreSQL installation, the following helper creates the database 143 and role, initializes (or upgrades) the schema, and grants service access: 144 145 .. code-block:: console 146 147 $ sudo libeufin-ebisync-dbconfig 148 149 EBICS setup 150 =========== 151 152 When you sign up for an EBICS-enabled bank account, the bank will provide you 153 with various credentials. Those must be provided in the 154 ``/etc/libeufin-ebisync/libeufin-ebisync.conf`` configuration file. 155 156 The following snippet shows the mandatory configuration values: 157 158 .. _ebisync-core-config: 159 160 .. code-block:: ini 161 :caption: /etc/libeufin-ebisync/libeufin-ebisync.conf 162 163 [ebisync] 164 # Bank 165 HOST_BASE_URL = https://ebics.postfinance.ch/ebics/ebics.aspx 166 167 # EBICS IDs 168 HOST_ID = PFEBICS 169 USER_ID = PFC00563 170 PARTNER_ID = PFC00563 171 172 .. note:: 173 Refer to the manpage ``libeufin-ebisync.conf(5)`` 174 for the full array of configuration values. 175 176 .. warning:: 177 This combination of HOST_ID, USER_ID and PARTNER_ID must never be used by another instance of libeufin-ebisync or by other EBICS clients, otherwise data will be lost. 178 179 Reuse existing client keys 180 -------------------------- 181 182 If you have client keys from a previous EBICS setup you can copy the JSON file to the configured path ``CLIENT_PRIVATE_KEYS_FILE`` (``/var/lib/libeufin-ebisync/client-ebics-keys.json`` with the default config). 183 184 Make sure this file is accessible to the user running ``libeufin-ebisync``, for the default services you should run: 185 186 .. code-block:: console 187 188 $ chown libeufin-ebisync:libeufin-ebisync /var/lib/libeufin-ebisync/client-ebics-keys.json 189 190 Create new client keys 191 ---------------------- 192 193 The following command should start the EBICS setup process: 194 195 .. code-block:: console 196 197 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 198 199 If the previous command failed when running EBICS INI with an error code of 200 ``EBICS_INVALID_USER_OR_USER_STATE``, you need to confirm your keys to your bank to 201 activate your account. 202 203 To that end, the previous run should have left a PDF document that you can 204 print, sign and send to the bank. Look for the message that looks like ``PDF 205 file with keys created at '/tmp/libeufin-ebics-keys/$TIMESTAMP.pdf'``. 206 207 Once the bank has received and processed this document you can continue. 208 209 Get bank keys 210 ------------- 211 212 The following command will finish the EBICS setup process: 213 214 .. code-block:: console 215 216 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 217 218 The EBICS setup is finished once the bank keys have been accepted. 219 220 Fetch destination setup 221 ======================= 222 223 Azure Blob Storage 224 ------------------ 225 226 The following snippet shows the mandatory configuration values: 227 228 .. code-block:: ini 229 :caption: /etc/libeufin-ebisync/conf.d/ebisync-fetch.conf 230 231 [ebisync-fetch] 232 DESTINATION = azure-blob-storage 233 234 Some configuration values are sensitive and must be written to a secret configuration file that is only accessible to the ``libeufin-ebisync`` user. 235 236 .. code-block:: ini 237 :caption: /etc/libeufin-ebisync/secrets/ebisync-fetch.secret.conf 238 239 [ebisync-fetch] 240 AZURE_API_URL = https://myaccount.blob.core.windows.net/ 241 AZURE_ACCOUNT_NAME = myaccount 242 AZURE_ACCOUNT_KEY = Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw== 243 AZURE_CONTAINER = mycontainer 244 245 The following command will finish the fetcher setup process: 246 247 .. code-block:: console 248 249 $ sudo -u libeufin-ebisync libeufin-ebisync setup -c /etc/libeufin-ebisync/libeufin-ebisync.conf 250 251 Submit source setup 252 =================== 253 254 EbiSync API 255 ----------- 256 257 The following snippet shows the mandatory configuration values: 258 259 .. code-block:: ini 260 :caption: /etc/libeufin-ebisync/conf.d/ebisync-submit.conf 261 262 [ebisync-submit] 263 SOURCE = ebisync-api 264 265 Some configuration values are sensitive and must be written to a secret configuration file that is only accessible to the ``libeufin-ebisync`` user. 266 267 .. code-block:: ini 268 :caption: /etc/libeufin-ebisync/secrets/ebisync-submit.secret.conf 269 270 [ebisync-submit] 271 AUTH_METHOD = basic 272 USERNAME = admin 273 PASSWORD = password 274 275 Check the server configuration: 276 277 .. code-block:: console 278 279 $ sudo -u libeufin-ebisync libeufin-ebisync serve --check -c /etc/libeufin-ebisync/libeufin-ebisync.conf 280 281 Fetching files 282 ============== 283 284 The responsible subcommand for fetching and uploading files is 285 ``fetch``, and its configuration is a **superset** of ebisync-core-config_. On 286 top of that, it expects the database connection string, the destination config and *optionally* a 287 frequency and a checkpoint time of day. 288 289 ``fetch`` will run a periodical fetch of pending documents (documents 290 that have never been fetched before). If the bank server supports it, it will 291 also listen to real-time document availability notifications and fetch 292 documents as soon as they are announced. This enables instant registration of 293 instant transactions. If your bank server does not support real-time 294 notifications, it is recommended that you fetch more often; otherwise, you can 295 reduce the fetch frequency. 296 297 Only fetching pending documents is not always reliable, if other EBICS clients 298 use the same connection settings as ``libeufin-ebisync``, they can consume documents before they are ingested. In case of bank downtime, the status of 299 documents can be lost. This is why ``fetch`` will run a checkpoint every 300 day to download all documents generated since the last checkpoint. 301 302 If the bank supports real-time notifications, you can expect transactions to be registered as soon as they are booked: ~10s for instant transactions and 24 to 72h for other transactions. Otherwise, expect a latency corresponding to the fetch frequency for instant transactions. Thanks to checkpointing, latency should not exceed one day. 303 304 The frequency **may** be specified as 305 306 .. code-block:: ini 307 :caption: /etc/libeufin-ebisync/conf.d/ebisync-fetch.conf 308 309 [ebisync-fetch] 310 FREQUENCY = 5m 311 312 The supported time units are ``s`` (seconds), ``m`` (minutes), ``h`` (hours). 313 314 The checkpoint time of day **may** be specified as 315 316 .. code-block:: ini 317 :caption: /etc/libeufin-ebisync/conf.d/ebisync-fetch.conf 318 319 [ebisync-fetch] 320 CHECKPOINT_TIME_OF_DAY = 19:00 321 322 The following command would download any unseen EBICS files: 323 324 .. code-block:: console 325 326 $ sudo -u libeufin-ebisync libeufin-ebisync fetch -c /etc/libeufin-ebisync/libeufin-ebisync.conf --transient 327 328 The ``fetch`` subcommand will always cause libeufin-ebisync to download EBICS files and upload them to the configured destination. 329 330 The ``--transient`` flag makes the command sync only 331 once and return immediately afterwards. Without the flag, EbiSync 332 will download at the frequency specified in the configuration or when it receives real-time notifications. 333 334 The ``--debug-ebics $SAVEDIR`` flag will cause EBICS transactions steps and payloads to be stored in ``$SAVEDIR/$YYYY-MM-DD`` where ``$YYYY-MM-DD`` represents the date when the download took place. This is mostly useful for debugging. 335 336 Submitting files 337 ================= 338 339 If the ``ebisync-api`` source is configured, submission is triggered by using the API. 340 341 Deployment 342 ========== 343 344 Reverse Proxy Setup 345 ------------------- 346 347 By default if ``ebisync-api`` is used as a source, the ``libeufin-ebisync-httpd`` service listens for HTTP connections 348 on localhost. To make the service publicly available, a reverse 349 proxy such as nginx should be used. We strongly recommend to configure nginx 350 to use TLS. 351 352 The ``libeufin-ebisync`` package ships with a sample configuration that can be 353 enabled in nginx: 354 355 .. code-block:: shell-session 356 357 # vim /etc/nginx/sites-available/libeufin-ebisync 358 < ... customize configuration ... > 359 # ln -s /etc/nginx/sites-available/libeufin-ebisync /etc/nginx/sites-enabled/libeufin-ebisync 360 # systemctl reload nginx 361 362 Launching 363 --------- 364 365 .. code-block:: console 366 367 $ sudo systemctl enable --now libeufin-ebisync.target