The R5N Distributed Hash Table

Introduction Distributed Hash Tables (DHTs) are a key data structure for the construction of completely decentralized applications. DHTs are important because they generally provide a robust and efficient means to distribute the storage and retrieval of key-value pairs. This document contains the technical specification of the R5N DHT , a secure DHT routing algorithm and data structure for decentralized applications. This document defines the normative wire format of peer-to-peer messages, routing algorithms, cryptographic routines and security considerations for use by implementors. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

Connectivity and addressing In the DHT overlay, a peer is addressable by its Peer ID. The Peer ID is the 256-bit hash of the peer public key. The peer public key is the public key of the corresponding Ed25519 peer private key. In the network underlay, a peer is addressable by traditional means out of scope of this document. For example, the peer may have a TCP/IP address, or a HTTPS endpoint. While the specific addressing options and mechanisms are out of scope for this document, it is necessary to define a universal addressing format in order to facilitate the distribution of connectivity information to other peers in the DHT overlay. This format is the "HELLO" message. A "HELLO" is a human-readable UTF-8 string consisting of the peer public key and the HELLO URI .

peer-public-key := [A-HJ-NP-Z1-9]+ ]]>

For the string representation of the peer public key, the base-32 encoding "StringEncode" is used. However, instead of following the character map is based on the optical character recognition friendly proposal of Crockford . The only difference to Crockford is that the letter "U" decodes to the same base-32 value as the letter "V" (27). The "scheme" part of the HELLO URI defined the addressing scheme which is used. An example of an addressing scheme used throughout this document is "ip+tcp", which refers to a standard TCP/IP socket connection. The "hier"-part of the URI must provide a suitable address for the given addressing scheme. The following is a non-normative example of a HELLO containing three HELLO URIs:

It is expected that there are basic mechanisms available to manage peer connectivity and addressing. The required functionality are abstracted through the following procedures and events:

PEER_CONNECTED(phash,address): is a signal that allows the DHT to react to peers which connect. Such an event triggers, for example, updates in the routing table.
PEER_DISCONNECTED(phash,address): is a signal that allows the DHT to react to peers which disconnect. Such an event triggers, for example, updates in the routing table.
TRY_CONNECT(pid, address): A function which allows a peer to attempt the establishment of a connection to another peer using an address.
HOLD(pash): A function which tells the underlay to keep a hold on the connection to another peer.
DROP(pash): A function which tells the underlay to drop the connection to another peer.
RECEIVE(source, message): A function or event that allows the peer to receive protocol messages as defined in this document from a connected peer.
SEND(target, message): A function that allows a peer to send protocol messages as defined in this document to a connected peer. If call to SEND fails, the message has not been sent.
NETWORK_SIZE_ESTIMATE(N): A function or event that provides estimates on the network size for use in the DHT routing algorithms.
ADDRESS_ADD(pk, address): The underlay signals us that an address was added. This information is used, for example, to publish connectivity as part of the bootstrapping and overlay creation.
ADDRESS_DELETE(pk, address): The underlay signals us that an address was removed. This information is used, for example, to publish connectivity as part of the bootstrapping and overlay creation.
VERIFY(blob): Signature verification by underlay.

Routing

Peer distance

Peer selection

Peer-to-peer wire formats

Bloomfilter

Processing options

Extended query

PUT message

where:

MSIZE: denotes the size of this message in network byte order.
MTYPE: is the 16-bit message type. This type can be one of the DHT message types but for put messages it must be set to the value 146 in network byte order.
OPTIONS: is a 32-bit options field (see below).
BTYPE: is a 32-bit block type field. The block type indicates the content type of the payload. In network byte order.
HOPCOUNT: is a 32-bit number indicating how many hops this message has traversed to far. In network byte order.
REPLICATIONLVL: is a 32-bit number indicating the desired replication level of the data. In network byte order.
PATH_LEN: is a 32-bit number indicating the length of the PUT path recorded in PUTPATH. As PUTPATH is optiona, this value may be zero. In network byte order.
EXPIRATION: denotes the absolute 64-bit expiration date of the content. In microseconds since midnight (0 hour), January 1, 1970 in network byte order.
BLOOMFILTER: A bloomfilter (for peer identities) to stop circular routes.
KEY: The key under which the PUT request wants to store content under.
PUTPATH: the variable-length PUT path. The path consists of a list of PATH_LEN peer IDs.
PAYLOAD: the variable-length resource record data payload. The contents are defined by the respective type of the resource record.

GET message

where:

MSIZE: denotes the size of this message in network byte order.
MTYPE: is the 16-bit message type. This type can be one of the DHT message types but for put messages it must be set to the value 147 in network byte order.
OPTIONS: is a 32-bit options field (see below).
BTYPE: is a 32-bit block type field. The block type indicates the content type of the payload. In network byte order.
HOPCOUNT: is a 32-bit number indicating how many hops this message has traversed to far. In network byte order.
REPLICATIONLVL: is a 32-bit number indicating the desired replication level of the data. In network byte order.
XQUERY_SIZE: is a 32-bit number indicating the length of the optional extended query XQUERY. In network byte order.
BF_MUTATOR: The bloomfilter mutator.
BLOOMFILTER: A bloomfilter (for peer identities) to stop circular routes.
KEY: The key under which the PUT request wants to store content under.
XQUERY: the variable-length extended query. Optional.
BF_RESULT: the variable-length result bloomfilter.

RESULT message

where:

MSIZE: denotes the size of this message in network byte order.
MTYPE: is the 16-bit message type. This type can be one of the DHT message types but for put messages it must be set to the value 148 in network byte order.
OPTIONS: is a 32-bit options field (see below).
BTYPE: is a 32-bit block type field. The block type indicates the content type of the payload. In network byte order.
PUT_PATH_LEN: is a 32-bit number indicating the length of the PUT path recorded in PUTPATH. As PUTPATH is optiona, this value may be zero. In network byte order.
GET_PATH_LEN: is a 32-bit number indicating the length of the GET path recorded in GETPATH. As PUTPATH is optiona, this value may be zero. In network byte order.
EXPIRATION: denotes the absolute 64-bit expiration date of the content. In microseconds since midnight (0 hour), January 1, 1970 in network byte order.
KEY: The key under which the PUT request wants to store content under.
PUTPATH: the variable-length PUT path. The path consists of a list of PATH_LEN peer IDs.
GETPATH: the variable-length PUT path. The path consists of a list of PATH_LEN peer IDs.
PAYLOAD: the variable-length resource record data payload. The contents are defined by the respective type of the resource record.

Security Considerations

GANA Considerations GANA is requested to create a "DHT Block Types" registry. The registry shall record for each entry:

Name: The name of the record type (case-insensitive ASCII string, restricted to alphanumeric characters
Number: 32-bit, above 65535
Comment: Optionally, a brief English text describing the purpose of the record type (in UTF-8)
Contact: Optionally, the contact information of a person to contact for further information
References: Optionally, references describing the record type (such as an RFC)

The registration policy for this sub-registry is "First Come First Served", as described in . GANA is requested to populate this registry as follows:

GANA is requested to amend the "GNUnet Signature Purpose" registry as follows:

Test Vectors