The R5N Distributed Hash Table

Introduction Distributed Hash Tables (DHTs) are a key data structure for the construction of completely decentralized applications. DHTs are important because they generally provide a robust and efficient means to distribute the storage and retrieval of key-value pairs. While already provides a peer-to-peer (P2P) signaling protocol with extensible routing and topology mechanisms, it also relies on strict admission control through the use of either centralized enrollment servers or pre-shared keys. Modern decentralized applications require a more open system that enables ad-hoc participation and other means to prevent common attacks on P2P overlays. This document contains the technical specification of the R5N DHT , a secure DHT routing algorithm and data structure for decentralized applications. R5N is an open P2P overlay routing mechanism which supports ad-hoc participation and security properties including support for topologies in restricted-route environments and path signatures. This document defines the normative wire format of peer-to-peer messages, routing algorithms, cryptographic routines and security considerations for use by implementors. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

Architecture R5N is an overlay network with a pluggable transport layer. The following figure shows the R5N architecture.

Applications: Applications are components which directly use the DHT overlay interfaces. Possible applications include the GNU Name System or the CADET transport system .
Overlay Interface: The Overlay Interface exposes the core operations of the DHT overlay to applications. This includes querying and retrieving data from the DHT.
Block Storage: The Block Storage component is used to persist and manage data by peers. It includes logic for quotas, caching stragegies and data validation.
Message Processing: The Message Processing component processes requests from and responses to applications as well as messages from the underlay network.
Routing: The Routing component includes the routing table as well as routing and peer selection logic. It facilitates the R5N routing algorithm with required data structures and algorithms.
Underlay Interface: The DHT Underlay Interface is an abstraction layer on top of the supported links of a peer. Peers may be linked by a variety of different transports, including "classical" protocols such as TCP, UDP and TLS or advanced protocols such as GNUnet, L2P or Tor.

Underlay In the network underlay, a peer is addressable by traditional means out of scope of this document. For example, the peer may have a TCP/IP address, or a HTTPS endpoint. While the specific addressing options and mechanisms are out of scope for this document, it is necessary to define a universal addressing format in order to facilitate the distribution of connectivity information to other peers in the DHT overlay. This format is the "HELLO" message. A "HELLO" is a human-readable UTF-8 string consisting of the peer public key and the HELLO URI .

peer-public-key := [A-HJ-NP-Z1-9]+ ]]>

For the string representation of the peer public key, the base-32 encoding "StringEncode" is used. However, instead of following the character map is based on the optical character recognition friendly proposal of Crockford . The only difference to Crockford is that the letter "U" decodes to the same base-32 value as the letter "V" (27). The "scheme" part of the HELLO URI defined the addressing scheme which is used. An example of an addressing scheme used throughout this document is "ip+tcp", which refers to a standard TCP/IP socket connection. The "hier"-part of the URI must provide a suitable address for the given addressing scheme. The following is a non-normative example of a HELLO containing three HELLO URIs:

It is expected that there are basic mechanisms available to manage peer connectivity and addressing. The required functionality are abstracted through the following procedures and events:

PEER_CONNECTED(phash,address): is a signal that allows the DHT to react to peers which connect. Such an event triggers, for example, updates in the routing table.
PEER_DISCONNECTED(phash,address): is a signal that allows the DHT to react to peers which disconnect. Such an event triggers, for example, updates in the routing table.
TRY_CONNECT(pid, address): A function which allows a peer to attempt the establishment of a connection to another peer using an address.
HOLD(pash): A function which tells the underlay to keep a hold on the connection to another peer.
DROP(pash): A function which tells the underlay to drop the connection to another peer.
RECEIVE(source, message): A function or event that allows the peer to receive protocol messages as defined in this document from a connected peer.
SEND(target, message): A function that allows a peer to send protocol messages as defined in this document to a connected peer. If call to SEND fails, the message has not been sent.
NETWORK_SIZE_ESTIMATE(N): A function or event that provides estimates on the network size for use in the DHT routing algorithms.
ADDRESS_ADD(pk, address): The underlay signals us that an address was added. This information is used, for example, to publish connectivity as part of the bootstrapping and overlay creation.
ADDRESS_DELETE(pk, address): The underlay signals us that an address was removed. This information is used, for example, to publish connectivity as part of the bootstrapping and overlay creation.
VERIFY(blob): Signature verification by underlay.

Overlay In the DHT overlay, a peer is addressable by its Peer ID. The Peer ID is the 256-bit hash of the peer public key. The peer public key is the public key of the corresponding Ed25519 peer private key.

Block Storage

Routing

Peer selection In order to select peers from the routing table which are suitable destinations for sending messages, R5N uses a hybrid approach: Given an estimated network size N, the peer selection for the first N hops is random. After the initial N hops, peer selection follows an XOR-based peer distance calculation. As the message traverses a random path through the network for the first N hops, it is essential that routing loops are avoided. In R5N, a bloomfilter is used as part of the routing metadata in messages. The bloomfilter is updates at each hop with the hops peer identity. For the next hop selection in both the random and the deterministic case, any peer which is in the bloomfilter for the respective message is not included in the peer selection process. The procedure to select a peer for a given message key and bloomfilter is defined as follows:

IF hops >= N dist := MAX_VALUE FOR EACH p IN peers IF XOR(p, key) < dist dist := XOR(p, key) target := p END END ELSE r := rand() target := peers[r] END END ]]>

Message Processing

Bloomfilter In order to prevent circular routes, GET and PUT messages contain a 128-bit Bloom filter (m=128). The Bloom filter is used to detect duplicate peer IDs along the route. A Bloom filter "bf" is initially empty, consisting only of zeroes. There are two functions which can be invoked on the Bloom filter: ADD(bf, e) and TEST(bf, e) where "e" is an element which is to added to the Bloom filter or queried against the set. Any bloom filter uses k=16 different hash functions each of which is defined as follows:

Processing options In order to indicate certain processing requirements for messages a number of processing options may be specificied in the respective field of the signalling messages. The options field is 8 octets in length and each options is encoded in a single bit.

Demultiplex everywhere (0): Each peer along the way should look at 'enc'. Otherwise only the k-peers closest to the key should look at it.
Record route (1): Indicates to keep track of the route that the message took in the P2P network.
Find peer (2): Indicates a 'FIND-PEER' request. Implies that approximate results are acceptable.
Random key (3): Option for query key randomization. (TODO)
Last hop (4): Indicates if this was the last hop for a GET/PUT.

Extended query TODO: What is this for? Not documented anywhere

PUT message

where:

MSIZE: denotes the size of this message in network byte order.
MTYPE: is the 16-bit message type. This type can be one of the DHT message types but for put messages it must be set to the value 146 in network byte order.
OPTIONS: is a 32-bit options field (see below).
BTYPE: is a 32-bit block type field. The block type indicates the content type of the payload. In network byte order.
HOPCOUNT: is a 32-bit number indicating how many hops this message has traversed to far. In network byte order.
REPLICATIONLVL: is a 32-bit number indicating the desired replication level of the data. In network byte order.
PATH_LEN: is a 32-bit number indicating the length of the PUT path recorded in PUTPATH. As PUTPATH is optiona, this value may be zero. In network byte order.
EXPIRATION: denotes the absolute 64-bit expiration date of the content. In microseconds since midnight (0 hour), January 1, 1970 in network byte order.
BLOOMFILTER: A bloomfilter (for peer identities) to stop circular routes.
KEY: The key under which the PUT request wants to store content under.
PUTPATH: the variable-length PUT path. The path consists of a list of PATH_LEN peer IDs.
PAYLOAD: the variable-length resource record data payload. The contents are defined by the respective type of the resource record.

GET message

where:

MSIZE: denotes the size of this message in network byte order.
MTYPE: is the 16-bit message type. This type can be one of the DHT message types but for put messages it must be set to the value 147 in network byte order.
OPTIONS: is a 32-bit options field (see below).
BTYPE: is a 32-bit block type field. The block type indicates the content type of the payload. In network byte order.
HOPCOUNT: is a 32-bit number indicating how many hops this message has traversed to far. In network byte order.
REPLICATIONLVL: is a 32-bit number indicating the desired replication level of the data. In network byte order.
XQUERY_SIZE: is a 32-bit number indicating the length of the optional extended query XQUERY. In network byte order.
BF_MUTATOR: The bloomfilter mutator.
BLOOMFILTER: A bloomfilter (for peer identities) to stop circular routes.
KEY: The key under which the PUT request wants to store content under.
XQUERY: the variable-length extended query. Optional.
BF_RESULT: the variable-length result bloomfilter.

RESULT message

where:

MSIZE: denotes the size of this message in network byte order.
MTYPE: is the 16-bit message type. This type can be one of the DHT message types but for put messages it must be set to the value 148 in network byte order.
OPTIONS: is a 32-bit options field (see below).
BTYPE: is a 32-bit block type field. The block type indicates the content type of the payload. In network byte order.
PUT_PATH_LEN: is a 32-bit number indicating the length of the PUT path recorded in PUTPATH. As PUTPATH is optiona, this value may be zero. In network byte order.
GET_PATH_LEN: is a 32-bit number indicating the length of the GET path recorded in GETPATH. As PUTPATH is optiona, this value may be zero. In network byte order.
EXPIRATION: denotes the absolute 64-bit expiration date of the content. In microseconds since midnight (0 hour), January 1, 1970 in network byte order.
KEY: The key under which the PUT request wants to store content under.
PUTPATH: the variable-length PUT path. The path consists of a list of PATH_LEN peer IDs.
GETPATH: the variable-length PUT path. The path consists of a list of PATH_LEN peer IDs.
PAYLOAD: the variable-length resource record data payload. The contents are defined by the respective type of the resource record.

Security Considerations

GANA Considerations GANA is requested to create a "DHT Block Types" registry. The registry shall record for each entry:

Name: The name of the record type (case-insensitive ASCII string, restricted to alphanumeric characters
Number: 32-bit, above 65535
Comment: Optionally, a brief English text describing the purpose of the record type (in UTF-8)
Contact: Optionally, the contact information of a person to contact for further information
References: Optionally, references describing the record type (such as an RFC)

The registration policy for this sub-registry is "First Come First Served", as described in . GANA is requested to populate this registry as follows:

GANA is requested to amend the "GNUnet Signature Purpose" registry as follows:

Test Vectors