path: root/doc/scheme-gnunet.tm

<TeXmacs|2.1>

<style|<tuple|tmmanual|british|doc>>

<\body>
  <doc-data|<doc-title|Scheme-GNUnet (0.1)
  manual>|<doc-author|<author-data|<author-name|Maxime
  Devos>|<author-email|maximedevos@telenet.be>>>>

  Copyright (C) 2021 GNUnet e.V.

  Permission is granted to copy, distribute and/or modify this document under
  the terms of the GNU Free Documentation License, Version 1.3 or any later
  version published by the Free Software Foundation; \ with no Invariant
  Sections, no Front-Cover Texts, and no Back-Cover Texts.<space|1em>A copy
  of the license is included in the section entitled \<#2018\>GNU Free
  Documentation License\<#2019\>.

  \;

  <\table-of-contents|toc>
    <vspace*|1fn><with|font-series|bold|math-font-series|bold|font-shape|small-caps|1.<space|2spc>Installation
    and contributing guide> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <pageref|auto-1><vspace|0.5fn>

    1.1.<space|2spc>Building from source <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-2>

    <with|par-left|1tab|1.1.1.<space|2spc>Authenticating new source code
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-3>>

    1.2.<space|2spc>Writing tests <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-4>

    1.3.<space|2spc>Contact <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-6>

    1.4.<space|2spc>License <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-7>

    <vspace*|1fn><with|font-series|bold|math-font-series|bold|font-shape|small-caps|2.<space|2spc>Application
    guide> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <pageref|auto-8><vspace|0.5fn>

    <vspace*|1fn><with|font-series|bold|math-font-series|bold|font-shape|small-caps|3.<space|2spc>Programming
    guide> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <pageref|auto-9><vspace|0.5fn>

    3.1.<space|2spc>Concurrency <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-10>

    <with|par-left|4tab|Repeated conditions
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-11><vspace|0.15fn>>

    3.2.<space|2spc>Configuration <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-12>

    <with|par-left|1tab|3.2.1.<space|2spc>Locating configuration files
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-13>>

    <with|par-left|1tab|3.2.2.<space|2spc>Loading configuration files
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-14>>

    3.3.<space|2spc>Manipulation of network structures
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-15>

    <with|par-left|1tab|3.3.1.<space|2spc>Documentation
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-21>>

    <with|par-left|1tab|3.3.2.<space|2spc>Reading and writing
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-25>>

    <with|par-left|1tab|3.3.3.<space|2spc>Primitive types
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-30>>

    <with|par-left|1tab|3.3.4.<space|2spc>Packing
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-31>>

    3.4.<space|2spc>Communication with services
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-32>

    <with|par-left|1tab|3.4.1.<space|2spc>Asynchronuously connecting
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-33>>

    <with|par-left|1tab|3.4.2.<space|2spc>Message handler
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-34>>

    <with|par-left|1tab|3.4.3.<space|2spc>Message type database
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-38>>

    <with|par-left|1tab|3.4.4.<space|2spc>Error handler
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-39>>

    <with|par-left|1tab|3.4.5.<space|2spc>Ordering of injected errors and
    messages and sent messages <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-40>>

    <with|par-left|1tab|3.4.6.<space|2spc>Disconnecting
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-41>>

    3.5.<space|2spc>Error reporting <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-42>

    3.6.<space|2spc>Estimation of the size of the network
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-44>

    <vspace*|1fn><with|font-series|bold|math-font-series|bold|font-shape|small-caps|4.<space|2spc>Implementation
    details> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <pageref|auto-46><vspace|0.5fn>

    <vspace*|1fn><with|font-series|bold|math-font-series|bold|font-shape|small-caps|Appendix
    A.<space|2spc>GNU Free Documentation License>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <pageref|auto-47><vspace|0.5fn>

    0. <with|font-shape|small-caps|Preamble>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-48>

    1. <with|font-shape|small-caps|Applicability and definitions>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-49>

    2. <with|font-shape|small-caps|Verbatim copying>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-50>

    3. <with|font-shape|small-caps|Copying in quantity>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-51>

    4. <with|font-shape|small-caps|Modifications>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-52>

    5. <with|font-shape|small-caps|Combining documents>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-53>

    6. <with|font-shape|small-caps|Collections of documents>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-54>

    7. <with|font-shape|small-caps|Aggregation with independent works>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-55>

    8. <with|font-shape|small-caps|Translation>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-56>

    9. <with|font-shape|small-caps|Termination>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-57>

    10. <with|font-shape|small-caps|Future revisions of this license>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-58>

    11. <with|font-shape|small-caps|Relicensing>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-59>

    <with|font-shape|small-caps|Addendum>: How to use this License for your
    documents <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <no-break><pageref|auto-60>

    <vspace*|1fn><with|font-series|bold|math-font-series|bold|font-shape|small-caps|Index>
    <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
    <pageref|auto-61><vspace|0.5fn>
  </table-of-contents>

  <chapter|Installation and contributing guide>

  <section|Building from source>

  The latest \<#2018\>official\<#2019\> development version of scheme-GNUnet
  can be found at <hlink|https://notabug.org/maximed/scheme-gnunet|>. It can
  be downloaded with <hlink|git|https://en.wikipedia.org/wiki/Git>. The
  following software needs to be installed first:

  <\itemize>
    <item>The Autotools (<hlink|autoconf|https://www.gnu.org/software/autoconf/>
    and <hlink|automake|https://www.gnu.org/software/automake/>)

    <item><hlink|GNU Guile|https://www.gnu.org/software/guile/> (at least
    version 3)

    <item><hlink|Purely Functional Data Structures in Scheme
    (pfds)|https://github.com/ijp/pfds/>

    <item><hlink|(Guile) Fibers|https://github.com/wingo/fibers/>

    <item><hlink|Guile-QuickCheck|https://ngyro.com/software/guile-quickcheck.html>
  </itemize>

  For the benefit of dead tree readers, the invisible hyperlinks above are
  reproduced as visible URLs below.

  <\itemize>
    <item><slink|https://www.gnu.org/software/autoconf/>

    <item><slink|https://www.gnu.org/software/guile/>

    <item><slink|https://github.com/ijp/pfds/>

    <item><slink|https://github.com/wingo/fibers/>

    <item><slink|https://ngyro.com/software/guile-quickcheck.html>
  </itemize>

  A few patches to guile and guile-fibers are required (some bug fixes, some
  extra functionality), see <verbatim|guix.scm>.

  Users of <hlink|GNU Guix|https://guix.gnu.org> can run <shell|guix
  environment -l guix.scm> in the checkout to create an environment where
  these dependencies are all present.<space|1em>Scheme-GNUnet uses the
  standard GNU build system, so to build Scheme-Gnunet, you only need to run

  <\shell-code>
    autoreconf -vif

    ./configure

    make

    make check
  </shell-code>

  After building, the documentation is available at
  <verbatim|doc/scheme-gnunet.pdf> and <verbatim|doc/scheme-gnunet.html> in
  PDF and HTML formats.<space|1em>To get started, you can run the example
  mini-application at <verbatim|examples/nse-web.scm> and point your browser
  at <slink|http://localhost:8089>:

  <\shell-code>
    $ guile -L . -C . -l examples/nse-web.scm
  </shell-code>

  <subsection|Authenticating new source code>

  When GNU Guix is present, after pulling the latest Scheme-GNUnet commit,
  the following command can be run to verify it is authentic:

  <\shell-code>
    guix git authenticate 088ea8b27b95143584cbc36202c5a02dfe95796c "C1F3 3EE2
    0C52 8FDB 7DD7 \ 011F 49E3 EE22 1917 25EE"
  </shell-code>

  If it isn't authentic, an error message such as the following will be
  written:

  <\shell-code>
    Authenticating commits 54a74dc to 431f336 (1 new commits)...

    [#########################################################]

    guix git: error: commit 431f336edd51e1f0fe059a6f6f2d4c3e9267b7bc not
    signed by an authorized key: C1F3 3EE2 0C52 8FDB 7DD7 \ 011F 49E3 EE22
    1917 25EE
  </shell-code>

  <section|Writing tests>

  <index|tests>\<#2018\><hlink|How SQLite Is
  Tested|https://sqlite.org/testing.html>\<#2019\> is a recommended
  read.<space|1em>Scheme-GNUnet isn't that well-tested but still aims for
  being free of bugs and having many tests to prevents bugs from being
  introduced.<space|1em>When adding new code, consider writing test
  cases.<space|1em>Some things that can be tested and few methods for testing
  things:

  <\itemize>
    <item>Run mutation tests.<space|1em>That is, replace in the source code
    <scm|\<less\>> with <scm|\<less\>=>, <scm|0> with <scm|1>, a variable
    reference <scm|i> with a variable reference <scm|j>, swap destination and
    source arguments <text-dots> and verify whether the tests detect these
    little mutations.

    <item>Be exhaustive.<space|1em>If a procedure handles both foos and bars,
    write test cases that pass the procedure a foo and test cases that pass
    the procedure a bar.<space|1em>Sometimes Guile-QuickCheck can help with
    generating many test cases if the input has a regular structure yet many
    edge cases, see e.g. <verbatim|tests/cmsg.scm>.

    <item>Verify exception mechanisms!<space|1em>If a procedure is expected
    to handle I/O errors, simulate I/O errors and end-of-files in all the
    wrong places.<space|1em>If the procedure can raise exceptions, make sure
    these exceptions are raised when necessary.
  </itemize>

  Tests are added in the directory <scm|tests> and to the variable
  <verbatim|SCM_TESTS> in <verbatim|Makefile.am> and use <scm|srfi
  :64>.<space|1em>To run the test suite, run <verbatim|make check>.

  <section|Contact>

  Scheme-GNUnet is currently maintained on NotABug:
  <slink|https://notabug.org/maximed/scheme-gnunet/>.<space|1em>Issues and
  pull requests can be reported and submitted here.<space|1em>Alternatively,
  for discussion about developing Scheme-GNUnet, you can send mails to
  <hlink|gnunet-devel@gnu.org|mailto:gnunet-devel@gnu.org> and for help about
  how to use Scheme-GNUnet, you can contact
  <hlink|help-gnunet@gnu.org|mailto:help-gnunet@gnu.org>.<space|1em>These are
  public mailing lists, so don't send anything there you wouldn't mind the
  whole world to know.

  For security-sensitive issues, you can send a mail directly to the
  maintainer, <hlink|Maxime Devos \<less\>maximedevos@telenet.be\<gtr\>|mailto:maximedevos@telenet.be>,
  optionally encrypted and signed with a GnuPG-compatible
  system.<space|1em>The maintainer's key fingerprint is C1F3 3EE2 0C52 8FDB
  7DD7 011F 49E3 EE22 1917 25EE and a copy of the key can be downloaded from
  <slink|https://notabug.org/maximed/things/raw/master/Maxime_Devos.pub>.

  <section|License>

  The code of Scheme-GNUnet is available under the Affero General Public
  License (AGPL), version 3 or later; see individual source files for
  details.<space|1em>The documentaton is available under the GNU Free
  Documentation License, see the start of this manual and the likewise-named
  appendix for details.<space|1em>The AGPL has some unusual conditions w.r.t.
  applications interacting with the network, please read it carefully.

  <chapter|Application guide>

  Scheme-GNUnet doesn't have any example applications, except the half-baked
  <verbatim|examples/nse-web.scm>, <verbatim|gnu/gnunet/scripts/download-store.scm>
  and <verbatim|gnu/gnunet/scripts/publish-store.scm>.<space|1em>Over time,
  we hope we have something to write here, but for now, this chapter is
  empty.

  <chapter|Programming guide>

  <section|Concurrency>

  Scheme-GNUnet uses <code*|guile-fibers> for concurrency, but supports
  POSIX-style threading as well, using the <code*|(ice-9 threads)> Guile
  module. More concretely, this means <code*|spawn-fiber> is used by default
  for starting asynchronuous computations and public procedures accept an
  optional <scm|#:spawn> argument accepting a procedure like
  <code*|spawn-fiber> or <code*|call-with-new-thread>.
  \<#2018\>Conditions\<#2019\> can be used for synchronising concurrent
  computations, see the documentation of <code*|guile-fibers> for details.

  <paragraph|Repeated conditions>Scheme-GNUnet has a variant of fibers
  conditions, named \<#2018\>repeated conditions\<#2019\>, in the module
  <scm|(gnu gnunet concurrency repeated-condition)>.<space|1em>It is
  unfortunately ill-documented.

  <section|Configuration>

  There are a number of modules for accessing GNUnet
  configurations.<space|1em>Firstly, there is <scm|(gnu gnunet config db)>,
  which is the module library code would typically use.<space|1em>For
  testing, one can create an empty configuration with the procedure
  <scm|hash-\<gtr\>configuration> from that module and <scm|make-hashtable>
  from <scm|(rnrs hashtables)>, using <scm|hash-key> as hash function and
  <scm|key=?> as comparison function:

  <\scm-code>
    (import (gnu gnunet config db)

    \ \ \ \ \ \ \ \ (gnu gnunet config value-parser)

    \ \ \ \ \ \ \ \ (rnrs hashtables))

    (define config (hash-\<gtr\>configuration (make-hashtable hash-key
    key=?))
  </scm-code>

  The resulting configuration <scm|config> is initially empty, so set some
  <with|font-shape|italic|keys> in the <with|font-shape|italic|section>
  <verbatim|nse>, to configure the network-size estimation service:

  <\scm-code>
    (set-value! identity config "nse" "UNIXPATH" "/tmp/nse.sock")

    (set-value! number-\<gtr\>string config "cadet" "MAX_ROUTES" 5000)

    ;; TODO: IP address, time durations, booleans, ...
  </scm-code>

  Now read these values back:

  <\scm-code>
    (read-value value-\<gtr\>file-name config "nse" "UNIXPATH")

    ;; -\<gtr\> /tmp/nse.sock

    (read-value value-\<gtr\>natural config "cadet" "MAX_ROUTES")

    ;; -\<gtr\> 5000
  </scm-code>

  What if the configuration doesn't have a value for the specified section
  and key?<space|1em>Then an <scm|&undefined-key-error> results:

  <\scm-code>
    (read-value value-\<gtr\>natural config "kated" "MAX_ROUTES")

    ;; -\<gtr\>

    ;; ice-9/boot-9.scm:1685:16: In procedure raise-exception:

    ;; ERROR:

    ;; \ \ 1. &undefined-key-error:

    ;; \ \ \ \ \ \ section: "kated"

    ;; \ \ \ \ \ \ key: "MAX_ROUTES"
  </scm-code>

  <subsection|Locating configuration files>

  There are two \U possibly non-existent \U configuration files: the
  <with|font-shape|italic|user> configuration and the
  <with|font-shape|italic|system> configuration.<space|1em>The
  <with|font-shape|italic|system> configuration typically contains the paths
  for services like NSE, CORE, <text-dots><space|1em>A user may choose not to
  use the services by the system and instead use their own.<space|1em>To do
  so, the user needs to override the paths in the
  <with|font-shape|italic|user> configuration.<space|1em><todo|defaults?> The
  module <scm|(gnu gnunet config fs)> is responsible for determining the
  location of the configuration files to load and actually load configuration
  files.<space|1em>For determining the location of the configuration files,
  the procedures <scm|locate-user-configuration> and
  <scm|locate-system-configuration> can be used.

  <\warning>
    The C implementation's mechanism for user-system separation seems to work
    differently.
  </warning>

  <\explain>
    <scm|(locate-user-configuration <scm|#:getenv=getenv>)>
  <|explain>
    This procedure determines the location of the user configuration file, as
    a string, or <scm|#false> if it could not be determined.<space|1em>If the
    location of the user configuration file is known, but the file does not
    exist, it is returned anyway, as a string.

    If the environment variable <shell|XDG_CONFIG_HOME> is set, the location
    of the file <verbatim|gnunet.conf> in the directory
    <shell|$XDG_CONFIG_HOME> is returned.<space|1em>If the environment
    variable is not set, the location of the file at
    <verbatim|.config/gnunet.conf> in the home directory specified by the
    environment variable <verbatim|HOME> is returned, if that environment
    variable exists.<space|1em>If both are unset, <scm|#false> is returned.

    The values of environment variables is determined with the procedure
    <scm|getenv>.
  </explain>

  <\explain>
    <scm|(locate-system-configuration)>
  <|explain>
    This procedure determines the location of the system configuration file,
    as a string.

    Currently, this is always <verbatim|/etc/gnunet.conf>.
  </explain>

  <subsection|Loading configuration files>

  Once the location of the configuration file is known, the file can be
  opened with the Scheme procedure <scm|open-input-file>, which returns an
  input port.<space|1em>Then the procedure <scm|load-configuration/port!> can
  be used to determine all section-key-values triples in the configuration.

  <\explain>
    <scm|(load-configuration/port! <var|set-value!> <var|port>)>
  <|explain>
    Load the configuration from the input port <var|port>.

    For each variable, call <var|set-value!> with the section name, variable
    name and a vector of the form <scm|#(line line-number value)>, where
    <var|value> a list of expansible objects.

    <todo|document expansible objects><todo|error reporting>
  </explain>

  A variable assignment <verbatim|[section] key=value${var}> can refer to
  variables defined in the <verbatim|PATHS> section and variables from the
  environment.<space|1em>The previously described procedure
  <scm|load-configuration/port!> will <em|not> expand such assignements. \ To
  expand variable assignments, use the procedure
  <scm|make-expanded-configuration> instead.

  <\explain>
    <scm|(make-expanded-configuration <var|load!> #:getenv=<var|getenv>)>
  <|explain>
    Make a configuration object.<space|1em>To populate the configuration, all
    the procedure <var|load!> with a <scm|set-value!> procedure as expected
    by <scm|load-configuration/port!>.<space|1em>The values from
    <scm|set-value!> are added to the confoiguration and every variable is
    expanded.
  </explain>

  To automatically load the defaults, the system configuration and the user
  configuration, use the thunk <scm|load-configuration>:

  <\explain>
    <scm|(load-configuration #:getenv=<var|getenv> #:files=<text-dots>)>
  </explain|Load the defaults, the system configuration and the user
  configuration and return the resulting configuration object.<space|1em>The
  list of files to load can be overriden by setting the undocumented
  <var|files> keyword argument.>

  Applications (whether graphical or textual) are recommended to use
  <scm|load-configuration> by default, as it largely just works.

  <section|Manipulation of network structures>

  <index|network structure><index|netstruct>The modules <scm|(gnu gnunet
  netstruct procedural)> and <scm|(gnu gnunet netstruct syntactic)> can be
  used for formatting messages to be sent over the network or to a
  service.<space|1em>The macros <scm|define-type><index|define-type> and
  <scm|structure/packed><index|structure/packed> can be used to define new
  structures, like this:

  <\scm-code>
    (define-type /:msg:nse:estimate/example

    \ \ (structure/packed

    \ \ \ (properties '((message-symbol msg:nse:estimate)))

    \ \ \ (synopsis "Network size estimate")

    \ \ \ (documentation "Some explanation")

    \ \ \ (field (header /:message-header))

    \ \ \ (field (size-estimate ieee-double/big)

    \ \ \ \ \ \ \ \ \ \ (synopsis "Timestamp for when the estimate was
    made"))))
  </scm-code>

  This example is taken from the <scm|(gnu gnunet nse struct)> module and
  oversimplified.<space|1em>All its components will gradually
  explained.<space|1em>First, what actually is a network
  structure?<space|1em>This question is ambigious, because \<#2018\>network
  structure\<#2019\> can refer to either the <with|font-shape|italic|value>
  or the <with|font-shape|italic|type>.<space|1em>The
  <with|font-shape|italic|value> is a sequence of octets, i.e., a sequence of
  numbers in the closed range 0\U255.<space|1em>The
  <with|font-shape|italic|type> describes how the
  <with|font-shape|italic|value> is structured.

  As an example, consider figure <reference|networkstructex>.<space|1em>There,
  the value is <scm|0 12 1 65 64 51 238 123 71 27 58 149> and the type is
  <scm|/:msg:nse:estimate/example>.

  <big-figure|<tree|<scm|/:msg:nse:estimate/example>|<tree|<scm|header>
  <scm|/:message-header>|<tree|<scm|size> <scm|u16/big>|0
  12>|<tree|<scm|type> <scm|u16/big>|1 65>>|<tree|<scm|size-estimate>
  <scm|ieee-double/big>|64 51 238 123 71 27 58 149>>|A network structure,
  both <with|font-shape|italic|value> and
  <with|font-shape|italic|type>.<label|networkstructex>>

  This value has a <with|font-shape|italic|header> with a
  <with|font-shape|italic|size> \<#2018\>0 0 0 12\<#2019\> of type
  <scm|u32/big>, so the <with|font-shape|italic|size> is 12.<space|1em>The
  <with|font-shape|italic|header> also has a <with|font-shape|italic|type>
  \<#2018\>0 0 1 65\<#2019\> of type <scm|u32/big>, so the type is
  <math|256\<times\>1+65=321>.<space|1em>The value also has a
  <with|font-shape|italic|size estimate> of type
  <scm|ieee-double/big>.<space|1em>The octets <scm|64 51 238 123 71 27 58
  149>, \ interpreted as an IEEE double, form the number
  <math|19.93\<ldots\>>.

  <subsection|Documentation>

  A network structure can optionally have embedded
  documentation.<space|1em>More specifically, network structures can
  optionally have the <with|font-shape|italic|synopsis><index|synopsis>,
  <with|font-shape|italic|documentation><index|documentation> and
  <with|font-shape|italic|properties><index|properties> set.<space|1em>The
  <with|font-shape|italic|synopsis> is a short description of what the
  network structure represents, typically a one-liner.<space|1em>The
  <with|font-shape|italic|documentation> can be more detailed, explaining how
  the structure works, can be used, is used and should be used.<space|1em>The
  <with|font-shape|italic|properties> form a free-formed association list.

  The synopsis, documentation and properties can be set on structured created
  with <scm|structure/packed> and individual fields and can be accessed with
  the procedures <scm|documentation>, <scm|synopsis> and <scm|properties>.

  <subsection|Reading and writing>

  The procedures <scm|read%><index|read%>, <scm|set%!><index|set%!>,
  <scm|sizeof><index|sizeof> and <scm|select><index|select> from <scm|(gnu
  gnunet netstruct procedural)> or the like-named macros from <scm|(gnu
  gnunet netstruct syntactic)> can be used for reading and writing network
  structures.<space|1em>The macros from <scm|syntactic> behave like the
  procedures from <scm|procedural> with some optimisations at expansion
  time.<space|1em>The procedures will be demonstrated with the
  <scm|/:msg:nse:estimate/example> network structure defined previously.

  First, create a memory slice with <scm|make-slice/read-write> from
  <scm|(gnu gnunet utils bv-slice)>.<space|1em>The required size can be
  determinded with <scm|(sizeof /:msg:nse:estimate/example
  '())>.<space|1em>The role of <scm|'()> will be explained later.

  <\scm-code>
    (import (gnu gnunet netstruct procedural)

    \ \ \ \ \ \ \ \ (gnu gnunet netstruct syntactic)

    \ \ \ \ \ \ \ \ (gnu gnunet utils bv-slice)

    \ \ \ \ \ \ \ \ (gnu gnunet util struct))

    \;

    (define-type /:msg:nse:estimate/example

    \ \ (structure/packed

    \ \ \ (field (header /:message-header))

    \ \ \ (field (size-estimate ieee-double/big))))

    \;

    (define message

    \ \ (make-slice/read-write (sizeof /:msg:nse:estimate/example '())))
  </scm-code>

  The fields of <scm|message> can be set with <scm|(set%! netstruct '(field
  ...) slice value)>.<space|1em>The following code sets all the fields:

  <\scm-code>
    (set%! /:msg:nse:estimate/example '(header size) message

    \ \ \ \ \ \ \ (sizeof /:msg:nse:estimate/example '()))

    (set%! /:msg:nse:estimate/example '(header type) message 165)

    (set%! /:msg:nse:estimate/example '(size-estimate) message 19.2)
  </scm-code>

  The size of an individual field can be determined with <scm|(sizeof
  netstruct '(field ...))>.<space|1em>For example, the following code
  determines the size of the \<#2018\>size\<#2019\> field in the header:

  <\scm-code>
    (sizeof /:msg:nse:estimate/example '(header size)) ; 12
  </scm-code>

  The fields can also be read:

  <\scm-code>
    (read% /:msg:nse:estimate/example '(header size) message) ; 12

    (read% /:msg:nse:estimate/example '(header type) message) ; 165

    (read% /:msg:nse:estimate/example '(size-estimate) message) ; 19.2
  </scm-code>

  <subsection|Primitive types>

  There are a number of pre-defined types.<space|1em>First, there is
  <scm|u8>, a single octet that is interpreted as an integer in the closed
  range <math|<around*|[|0,255|]>>.<space|1em>There are also types
  <scm|uN/endian> for <math|N\<in\><around*|{|16,32,64|}>> and
  <math|endian\<in\><around*|{|little,big|}>>, which interprets <math|N/8>
  octets as integers in the closed range <math|<around*|[|0,2<rsup|N>-1|]>>.<space|1em>The
  types <scm|ieee-double/big> and <scm|ieee-double/little> are 8 octets long
  and represent floating-point numbers in IEEE 754 format
  (\<#2018\>binary64\<#2019\>).

  <subsection|Packing>

  In contrast to C structures, Scheme-GNUnet network structures are always
  packed \V there are no \<#2018\>gaps\<#2019\> between fields.

  <section|Communication with services>

  To connect with a GNUnet service \V this applies to both the C and Scheme
  implementation, the GNUnet service must bind a local domain
  socket<\footnote>
    The C implementation supports Internet sockets as well.
  </footnote> somewhere on the file system and the client (possibly another
  service) must connect to it.<space|1em>Connections to a service can be made
  with the <scm|connect/fibers> procedure from <scm|(gnu gnunet mq-impl
  stream)>, like this:

  <\scm-code>
    (define mq (connect/fibers config "nse" handlers error-handler))
  </scm-code>

  <subsection|Asynchronuously connecting>

  This is an asynchronuous operation: it will \<#2018\>complete\<#2019\>
  immediately and the connection will actually be formed in the
  background.<space|1em>When the connection has actually be formed, the
  <scm|error-handler> is called with the symbol
  <scm|connection:connected>.<space|1em>To demonstrate, the following code
  asynchronuously connects to the NSE service, and prints the text
  <scm|"connected!"> when the connection has actually been formed.

  <\scm-code>
    ;; XXX test this, explain 'config' ...

    (define (error-handler error . args)

    \ \ (case error

    \ \ \ \ ((connection:connected)

    \ \ \ \ \ (format #t "connected!~%"))

    \ \ \ \ (else (format #t "unknown error: ~a ~a~%" error args))))

    \;

    (define mq

    \ \ (connect/fibers config "nse" (message-handlers) error-handler))
  </scm-code>

  <subsection|Message handler>

  <index|message handler>When a message is received by the message queue, the
  corresponding message handler is invoked.<space|1em>Message handlers can be
  constructed with the <scm|message-handler><index|message-handler> macro and
  the <scm|make-message-handler><index|make-message-handler> procedure from
  <scm|(gnu gnunet nse client)>, as follows:

  <\scm-code>
    (import (gnu gnunet mq handler)

    \ \ \ \ \ \ \ \ (gnu extractor enum)

    \ \ \ \ \ \ \ \ (gnu gnunet message protocols)

    \ \ \ \ \ \ \ \ (gnu gnunet util struct)

    \ \ \ \ \ \ \ \ (gnu gnunet utils bv-slice)

    \ \ \ \ \ \ \ \ (gnu gnunet netstruct syntactic))

    \;

    (define handler/syntactic

    \ \ (message-handler

    \ \ \ (type (symbol-value message-type msg:util:dummy))

    \ \ \ ((interpose code) code)

    \ \ \ ((well-formed? slice)

    \ \ \ \ (= (slice-length slice)

    \ \ \ \ \ \ \ (sizeof /:message-header '())))

    \ \ \ ((handle! slice)

    \ \ \ \ (pk 'message: slice))))

    \;

    (define handler/procedural

    \ \ (make-message-handler

    \ \ \ (symbol-value message-type msg:util:dummy)

    \ \ \ (lambda (thunk) (thunk))

    \ \ \ (lambda (slice)

    \ \ \ \ \ (= (slice-length slice)

    \ \ \ \ \ \ \ \ (sizeof /:message-header '())))

    \ \ \ (lambda (slice)

    \ \ \ \ \ (pk 'message: slice))))
  </scm-code>

  As illustrated in the example code above, a message handler has four
  components: the <with|font-shape|italic|type> of message it handles, an
  <with|font-shape|italic|interposer> which will be explained later, the
  <with|font-shape|italic|verifier> deciding if a message is well-formed and
  the <with|font-shape|italic|handler procedure>.

  The verifier is passed a bytevector slice with the message and should
  return <scm|#true> if the message is well-formed and <scm|#false> if it
  isn't.<space|1em>It may assume that the length of the slice corresponds to
  the length <em|in> the message header and is at least the length <em|of>
  the message header and that the type in the message header corresponds to
  the type of the message handler.<space|1em>Messages will only be passed to
  the handler procedue if the verifiers returns <scm|#true>.

  The handler procedure is passed a bytevector slice with the message, but
  only if the verifier considers it well-formed.<space|1em>The handler
  procedure and verifier are run from the
  <with|font-shape|italic|interposer>.<space|1em>The interposer is passed a
  thunk to execute and may e.g. install exception handlers and parameterise
  parameters.<space|1em>It can change the current input, output and error
  ports for example.

  <todo|document the message type database, various procedures>

  <subsection|Message type database>

  The module <scm|(gnu gnunet message protocols)> has a mapping of symbolic
  names of every message type known to scheme-GNUnet to their numeric
  value.<space|1em>To use it, the macro <scm|symbol-value> from <scm|(gnu
  extractor enum)> is required and possibly <scm|value-\<gtr\>index> as
  well.<space|1em>To determine the numeric value of the message type
  <scm|msg:nse:estimate>, one would write:

  <\scm-code>
    (define numeric-type

    \ \ (value-\<gtr\>index (symbol-value message-type msg:nse:estimate)))
  </scm-code>

  <todo|other various enum procedures for introspection, documentation,
  <text-dots>?>

  <todo|how to define new message types>

  <subsection|Error handler>

  The message queue implementation usually just sends and receives messages,
  but some exceptional situations cannot be communicated with
  <scm|send-message!> or <scm|inject-message!>.<space|1em>For those, there is
  the <scm|inject-error!> procedure.<space|1em>This variadic procedure
  accepts a message queue to inject the error into, a
  <with|font-shape|italic|key> (usually a symbol) describing the exceptional
  situation and rest arguments.<space|1em>It calls the
  <with|font-shape|italic|error handler> of the message queue with the key
  and rest arguments.<space|1em>The following errors can currently be
  reported by the built-in message queue implementations:

  <\explain>
    <scm|connection:connected>
  <|explain>
    The connection to the server has been established.
  </explain>

  <\explain>
    <scm|connection:interrupted>
  </explain|The message queue has been closed before the connection to the
  server could be established.>

  <\explain>
    <scm|input:regular-end-of-file>
  <|explain>
    The connection has been closed by the server.

    For people wondering about what happens if a connection becomes
    half-duplex: GNUnet does not have a notion of half-duplex message
    streams.<space|1em>If it is detected the underlying stream became
    half-duplex anyways, it will be treated as closed by scheme-GNUnet,
    resulting in this error.<space|1em>However, note that currently broken
    pipes cannot be reliably detected.
  </explain>

  <\explain>
    <scm|input:premature-end-of-file>
  </explain|The connection was closed by the server while a message was still
  being read.<space|1em>This can happen if the server was stopped while it
  was still sending the rest of the message.>

  <\explain>
    <scm|input:overly-small> <var|type> <var|size>
  </explain|The message size in the header was smaller than the minimal
  message size.<space|1em>Sometimes, but not always, the message type
  <var|type> and message size <var|size> are available (as exact
  naturals).<space|1em>When they are not available, <var|type> and <var|size>
  are <scm|#false> instead.<space|1em>This can only happen if the server or
  connection to the server is buggy.>

  <\explain>
    <scm|logic:no-handler> <var|type> . <var|rest>
  <|explain>
    The received message of type <var|type> (as an integer) does not have a
    corresponding message handler.<space|1em><var|rest> is currently
    unspecified.
  </explain>

  <\explain>
    <scm|logic:ill-formed> <var|type> . <var|rest>
  </explain|The received message of type (as an integer) is ill-formed
  according to the message handler.<space|1em><var|rest> is currently
  unspecified.>

  Consider automatically reconnecting after
  <scm|<scm|input:regular-end-of-file>> and
  <scm|<scm|input:premature-end-of-file>>, to allow the server to restart
  without having to manually restart every individual
  application.<space|1em>To report errors, see the section
  <reference|sec:error reporting> Error reporting.

  <subsection|Ordering of injected errors and messages and sent messages>

  This section describes how injected errors and messages and sent messages
  are ordered with respect to each other in the default message queue
  implementation.<space|1em>Messages are handled or corresponding
  <scm|logic:no-handler> or <scm|logic:ill-formed> errors are injected in the
  order that the messages are received.<space|1em>Before messages are read,
  <scm|connection:connected> is injected.<space|1em>This error is injected at
  most once.

  <em|Soon> after all messages are read (and therefore
  <with|font-shape|italic|soon> after all handled messages or corresponding
  errors), the error <scm|input:regular-end-of-file>,
  <scm|input:overly-small> or <scm|input:premature-end-of-file> is
  injected.<space|1em>Only one of those errors can be injected for the entire
  lifetime of the message queue.

  Be aware that <em|soon> is not <em|immediate> here!<space|1em>For example,
  it is possible for a message to be received, the port closed, a message
  queued for sending, the closing of the port being detected by the write
  fiber, <scm|input:regular-end-of-file> being injected from the write fiber
  and the read fiber handling the received message, and the read fiber
  exiting because the port is closed, in that order.

  Messages are sent (and received on the other side) in the order they were
  enqueued for sending.<space|1em>Likewise, the notify-sent callback of
  enqueued messages are called in order.<space|1em>If the notify-sent
  callback is called, it is before the message is received by the other
  side.<space|1em>The message and its notify-sent callback are only received
  by the other side and called after the message has been injected and
  <scm|connection:connected> has been injected.<space|1em>It is possible for
  the notify-sent callback to be called without the message being received by
  the other side, e.g. if the port was closed during the notify-sent
  callback.

  If a message is received by the other side, all previously-sent messages
  have be received before.<space|1em>If a notify-sent callback is invoked,
  all notify-sent callbacks of previous messages have been invoked before,
  except the messages that are eventually cancelled.

  The errors <scm|logic:no-handler> and <scm|logic:ill-formed> are not fatal:
  later messages can still be read and handled.<space|1em>If
  <scm|connection:interrupted> is injected, no other errors are ever
  injected, whether in the past or in the future.<space|1em>This error can
  only be injected once.

  <todo|I/O errors>

  <todo|envelopes>

  <subsection|Disconnecting>

  A message queue can be closed with the <scm|close-queue!> procedure from
  <scm|(gnu gnunet mq)>.<space|1em>In the default message queue
  implementation, this asynchronuously closes the port and stops associated
  fibers.<space|1em>Closing ports when they won't be used anymore is
  important for limiting resource consumption, especially for servers that
  can have many connections.<space|1em>Closing message queues is an
  idempotent operation: closing a message queue twice is the same as closing
  it once.<space|1em> If a message queue is closed before a connection could
  be formed, <scm|connection:interrupted> is injected instead of
  <scm|connection:connected> and <scm|connection:regular-end-of-file>.

  <section|Error reporting><label|sec:error reporting>

  <index|error reporting>Errors can be reported with the procedure
  <scm|report-error> from the module <scm|(gnu gnunet mq
  error-reporting)>.<space|1em>It can be called as <scm|(report-error key
  argument ...)>, e.g. <scm|(report-error 'logic:no-handler 3)>.<space|1em>By
  default, it reports the error to the current error port.<space|1em>If this
  is not desired, the output can be sent to another port by setting the
  parameter <scm|textual-error-reporting-port>.<space|1em>If textual error
  reporting is not desired, the parameter <scm|error-reporter> can be set to
  a procedure with the same interface as <scm|report-error>.<space|1em>Such a
  procedure could e.g. open a GUI dialog, sent the message to the system
  logger or ignore the error.

  Error messages are translated for the current locale.<todo|TODO actually
  call bindtextdomain>

  <section|Estimation of the size of the network>

  <index|network size estimation>GNUnet has a service that roughly estimates
  the size of the network \U i.e., the number of peers.<space|1em>The module
  <scm|(gnu gnunet nse client)> can be used to interact with this
  service.<space|1em>The connection is made with the procedure <scm|connect>,
  which is accepts a <with|font-shape|italic|configuration> (see
  <todo|reference>) and some optional keyword arguments.<space|1em>This
  procedure can be called as <scm|(connect config #:updated updated
  #:connected connected #:disconnected disconnected)>.<space|1em>It returns a
  <with|font-shape|italic|NSE server object>.

  The connection is made asynchronuously; the thunk <var|connected> will be
  called when the connection has actually been made.<space|1em>Whenever a new
  estimate becomes available, the (optional) procedure <var|updated> is
  called with the new <with|font-shape|italic|estimate>.<space|1em>Alternatively,
  the procedure <scm|estimate> can be called on the server object to return
  the latest available estimate.<space|1em>If the
  <with|font-shape|italic|server object> doesn't have an estimate yet, that
  procedure will return <scm|#false> instead of an estimate.

  When the connection is lost, the (optional) thunk <var|disconnected> is
  called and <scm|(gnu gnunet nse client)> will retry
  connecting.<space|1em>To close the current connection, if any, and stop
  reconnecting, the idempotent procedure <scm|disconnect!> can be called on
  the server object.

  <todo|input, validation, I/O errors?>

  The estimate object has a number of accessors:

  <\explain>
    <scm|(estimate:logarithmic-number-peers <var|estimate>)>
  </explain|The base-2 logarithm of the number of peers (estimated), as a
  positive flonum, possibly zero or infinite>

  <\explain>
    <scm|(estimate:number-peers <var|estimate>)>
  </explain|The number of peers (estimated), as a flonum, at least <scm|1.0>
  and possibly infinite.<space|1em>This is not necessarily an (inexact)
  <scm|integer?>, as it is only an estimate.>

  <\explain>
    <scm|(estimate:timestamp estimate)>
  </explain|A timestamp when the estimate was made <todo|something about
  epoch?>>

  <\explain>
    <scm|(estimate:standard-deviation <var|estimate>)>
  </explain|The estimated standard deviation on the base-2 logarithm of
  peers, calculated over the last 64 rounds, with the <math|<frac|N|N-1>>
  correction.<space|1em>This is a positive flonum, possibly zero or
  infinite.>

  Assuming the network size is stable and the errors on the logarithmic
  estimate are normally distributed, the procedure
  <scm|estimate:standard-deviation> can be used to put probablistic error
  bounds on the number of peers on the network. <todo|example>

  <chapter|Implementation details>

  TODO<appendix|GNU Free Documentation License>

  <include|fdl.tm>

  <\the-index|idx>
    <index+1|define-type|<pageref|auto-18>>

    <index+1|documentation|<pageref|auto-23>>

    <index+1|error reporting|<pageref|auto-43>>

    <index+1|make-message-handler|<pageref|auto-37>>

    <index+1|message handler|<pageref|auto-35>>

    <index+1|message-handler|<pageref|auto-36>>

    <index+1|netstruct|<pageref|auto-17>>

    <index+1|network size estimation|<pageref|auto-45>>

    <index+1|network structure|<pageref|auto-16>>

    <index+1|properties|<pageref|auto-24>>

    <index+1|read%|<pageref|auto-26>>

    <index+1|select|<pageref|auto-29>>

    <index+1|set%!|<pageref|auto-27>>

    <index+1|sizeof|<pageref|auto-28>>

    <index+1|structure/packed|<pageref|auto-19>>

    <index+1|synopsis|<pageref|auto-22>>

    <index+1|tests|<pageref|auto-5>>
  </the-index>
</body>

<\initial>
  <\collection>
    <associate|page-medium|paper>
    <associate|save-aux|false>
  </collection>
</initial>