commit a47ea0eb76677b0171a2bb65bb43fae1e4e3df75
parent 8f6f148595f2fa25709cfac840e3a02644c35034
Author: Martin Schanzenbach <schanzen@gnunet.org>
Date: Tue, 18 Oct 2022 11:10:56 +0900
Cleanup; start improving GNS/namestore
Diffstat:
4 files changed, 99 insertions(+), 36 deletions(-)
diff --git a/developers/index.rst b/developers/index.rst
@@ -42,6 +42,7 @@ new chapters, sections or insightful comments.
.. toctree::
contributing.rst
style.rst
+ repo.rst
util/index.rst
subsystems/index.rst
diff --git a/developers/rest/namestore.rst b/developers/rest/namestore.rst
@@ -1,3 +1,5 @@
+.. _namestore_rest_api:
+
Namestore
=========
diff --git a/developers/subsystems/namestore/namestore.rst b/developers/subsystems/namestore/namestore.rst
@@ -17,19 +17,23 @@ the service: GNS information from other users obtained from the DHT are
stored by the NAMECACHE subsystem.
NAMESTORE uses a plugin-based database backend to store GNS information
-with good performance. Here sqlite, MySQL and PostgreSQL are supported
+with good performance. Here sqlite and PostgreSQL are supported
database backends. NAMESTORE clients interact with the IDENTITY
subsystem to obtain cryptographic information about zones based on egos
as described with the IDENTITY subsystem, but internally NAMESTORE
-refers to zones using the ECDSA private key. In addition, it
+refers to zones using the respective private key.
+
+NAMESTORE is queried and monitored by the ZONEMASTER service which periodically
+publishes public records of GNS zones. ZONEMASTER also
collaborates with the NAMECACHE subsystem and stores zone information
-when local information are modified in the GNS cache to increase look-up
-performance for local information.
+when local information are modified in the NAMECACHE cache to increase look-up
+performance for local information and to enable local access to private records
+in zones through GNS.
NAMESTORE provides functionality to look-up and store records, to
iterate over a specific or all zones and to monitor zones for changes.
-NAMESTORE functionality can be accessed using the NAMESTORE api or the
-NAMESTORE command line tool.
+NAMESTORE functionality can be accessed using the NAMESTORE C API, the NAMESTORE
+REST API, or the NAMESTORE command line tool.
:index:`libgnunetnamestore <single: libgnunet; namestore>`
----------------------------------------------------------
@@ -42,7 +46,7 @@ operations, or NULL is returned if the connection failed.
To disconnect from NAMESTORE, clients use
``GNUNET_NAMESTORE_disconnect`` and specify the handle to disconnect.
-NAMESTORE internally uses the ECDSA private key to refer to zones. These
+NAMESTORE internally uses the private key to refer to zones. These
private keys can be obtained from the IDENTITY subsystem. Here *egos*
*can be used to refer to zones or the default ego assigned to the GNS
subsystem can be used to obtained the master zone's private key.*
@@ -65,13 +69,18 @@ provided callback function with the result GNUNET_SYSERR on failure
was already there or not found GNUNET_YES (or other positive value) on
success plus an additional error message.
+In addition, ``GNUNET_NAMESTORE_records_store2`` can be used to store multiple
+record sets using a single API call. This allows the caller to import
+a large number of (label, records) tuples in a single database transaction.
+This is useful for large zone imports.
+
Records are deleted by using the store command with 0 records to store.
It is important to note, that records are not merged when records exist
with the label. So a client has first to retrieve records, merge with
existing records and then store the result.
To perform a lookup operation, the client uses the
-``GNUNET_NAMESTORE_records_store`` function. Here it has to pass the
+``GNUNET_NAMESTORE_records_lookup`` function. Here it has to pass the
namestore handle, the private key of the zone and the label. It also has
to provide a callback function which will be called with the result of
the lookup operation: the zone for the records, the label, and the
@@ -90,8 +99,8 @@ Iterating Zone Information
^^^^^^^^^^^^^^^^^^^^^^^^^^
A client can iterate over all information in a zone or all zones managed
-by NAMESTORE. Here a client uses the
-``GNUNET_NAMESTORE_zone_iteration_start`` function and passes the
+by NAMESTORE. Here a client uses one of the
+``GNUNET_NAMESTORE_zone_iteration_start(2)`` functions and passes the
namestore handle, the zone to iterate over and a callback function to
call with the result. To iterate over all the zones, it is possible to
pass NULL for the zone. A ``GNUNET_NAMESTORE_ZoneIterator`` handle is
@@ -109,7 +118,7 @@ Monitoring Zone Information
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Clients can also monitor zones to be notified about changes. Here the
-clients uses the ``GNUNET_NAMESTORE_zone_monitor_start`` function and
+clients uses one of the ``GNUNET_NAMESTORE_zone_monitor_start(2)`` functions and
passes the private key of the zone and and a callback function to call
with updates for a zone. The client can specify to obtain zone
information first by iterating over the zone and specify a
@@ -122,5 +131,3 @@ the zone, the label and the records and their number.
To stop monitoring, the client calls
``GNUNET_NAMESTORE_zone_monitor_stop`` and passes the handle obtained
from the function to start the monitoring.
-
-
diff --git a/users/gns.rst b/users/gns.rst
@@ -53,8 +53,12 @@ public key (you will learn how below) and extend the configuration:
$ gnunet-config -s gns -o .myfriend -V PUBLIC_KEY
-Zones and Egos
-~~~~~~~~~~~~~~
+
+Maintaining Zones
+~~~~~~~~~~~~~~~~~
+
+Creation and deletion
+"""""""""""""""""""""
In GNUnet, identity management is about managing egos. Egos can
correspond to pseudonyms or real-world identities. If you value your
@@ -68,14 +72,20 @@ identity. The IDENTITY service essentially stores the private keys under
human-readable names, and keeps a mapping of which private key should be
used for particular important system functions.
-You probably should create at least one zone of your own. You can create
-any number of zones using the gnunet-identity tool using:
+You can create any number of zones using the gnunet-identity tool using:
::
$ gnunet-identity --create="myzone"
-Henceforth, on your system you control the TLD “myzone”.
+Henceforth, on your system you control the TLD ``myzone``.
+Note that by default, this will create ECDSA key pairs.
+GNS also supports the use of EdDSA keys.
+In order to create a EdDSA zone, execute:
+
+::
+
+ $ gnunet-identity --create="myedzone" --eddsa
All of your zones can be listed (displayed) using the gnunet-identity
command line tool as well:
@@ -84,16 +94,41 @@ command line tool as well:
$ gnunet-identity --display
-Maintaining Zones
-~~~~~~~~~~~~~~~~~
+One you have created an identity, you can configure and initialize your NAMESTORE
+database.
+You have the choice between ``sqlite`` and ``postgres`` as databases.
+Most users will be served well with the ``sqlite`` default.
+To set the database backend of your NAMESTORE to ``postgres`` execute:
+
+::
+
+ $ gnunet-config -s namestore -o DATABASE -V postgres
+
+You can now configure the ``postgres`` database backend to find your database.
+See the ``namestore-postgres`` configuration section on possible options:
+
+::
+
+ $ gnunet-config -s namestore-postgres
+
+If you ever need to reset your zone database, you can use ``gnunet-namestore-dbtool``.
+**DANGER: This command will delete ALL of your records in all zones!**
+
+::
+
+ $ gnunet-namestore-dbtool --plugin=postgres --reset
+
-Now you can add (or edit, or remove) records in your GNS zone using the
-gnunet-namestore-gtk GUI or using the gnunet-namestore command-line
-tool. In either case, your records will be stored in an SQL database
-under control of the gnunet-service-namestore. Note that if multiple
-users use one peer, the namestore database will include the combined
-records of all users. However, users will not be able to see each
-other’s records if they are marked as private.
+Record management
+"""""""""""""""""
+
+At this point you can add (or edit, or remove) records in your GNS zone using the
+using the gnunet-namestore command-line tool.
+Your records will be stored in a database
+of the NAMESTORE service. Note that for multi-user setups, the NAMESTORE
+database will include the combined records of all users.
+However, users will not be able to see each other’s records if they are marked
+as private.
To provide a short example for editing your own zone, suppose you have
your own web server with the IP 1.2.3.4. Then you can put an A record (A
@@ -144,8 +179,28 @@ under “carol”, you can access Carol’s records under
“NAME.carol.bob.myzone” (where “NAME” is the name of Carol’s record you
want to access).
-Resolving GNS records
-~~~~~~~~~~~~~~~~~~~~~
+Large Zones
+"""""""""""
+
+Large zones in DNS such as Top-Level Domains can grow to millions of names.
+For example, the ``.fr`` DNS zone contained over 3.9 million entries in 2022.
+Such use cases require good performance and careful consideration of a couple
+of choices in GNS and NAMESTORE.
+
+The cryptographic operations of EdDSA are significantly faster than their
+ECDSA counterparts.
+Most users will be unable to observe this difference.
+But, if you plan on managing large zones, you may want to consider using EdDSA
+zones.
+
+For large zones, using the PostgreSQL NAMESTORE database backend is a must.
+Further, when importing large numbers of records at once, it is advised to make
+use of the bulk import functionality of the NAMESTORE API.
+There is a :ref:`C API for bulk imports <NAMESTORE-Subsystem>`.
+There is also a :ref:`REST API endpoint <namestore_rest_api>` which calls this API.
+
+Name resolution
+~~~~~~~~~~~~~~~
Next, you should try resolving your own GNS records. The method we found
to be the most uncomplicated is to do this by explicitly resolving using
@@ -159,14 +214,12 @@ In the shell, type:
::
$ gnunet-gns -u www.gnunet.org
- www.gnunet.org:
- Got `A' record: ...
That shows that resolution works, once GNS is integrated with the
application.
Integration with Browsers (DNS2GNS service)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+"""""""""""""""""""""""""""""""""""""""""""
Most OSes allow you to either modify your ``/etc/resolv.conf`` directly
or through ``resolvectl``. We are going to configure the dns2gns service
@@ -182,7 +235,7 @@ should be a DNS server you trust such as your local router:
::
$ gnunet-config -s dns2gns -o OPTIONS -V "-d $FALLBACK_DNS -p 5252"
- $ gnunet-arm -i dns2gns # Make sure the service is started
+ $ gnunet-arm -i dns2gns
If you edit your resolv.conf directly, it should contain and entry like
this:
@@ -214,7 +267,7 @@ already you can use NetworkManager-dispatcher. First, enable it:
Then, create a script ``/etc/NetworkManager/dispatch.h/10-dns2-gns.sh``:
-::
+.. code-block:: bash
#!/bin/sh
interface=$1
@@ -252,7 +305,7 @@ accessing HTTPS websites with browsers. In order learn how to provide
relief for this issue, read on.
Integration with Browsers (SOCKS proxy)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+"""""""""""""""""""""""""""""""""""""""
While we recommend integrating GNS using the DNS2GNS service or the
NSSwitch plugin, you can also integrate GNS directly with your browser
@@ -335,7 +388,7 @@ and please be not surprised if there is no output:
::
- $ gnunet-bcd # does not return
+ $ gnunet-bcd
Then, start a browser and point it to http://localhost:8888/ where
gnunet-bcd is running a Web server!
