commit 82163ff25bfb6da30609991c4066dd969a49a202
parent 9d78bfa2413fcdb1b42ae4bbaee26e91edd47ac2
Author: Christian Grothoff <christian@grothoff.org>
Date: Sat, 13 Jul 2024 17:02:05 +0200
work on Block section
Diffstat:
1 file changed, 127 insertions(+), 108 deletions(-)
diff --git a/draft-schanzen-r5n.xml b/draft-schanzen-r5n.xml
@@ -2568,12 +2568,12 @@ BEGIN
<dt>SetupResultFilter(FilterSize, Mutator) -> RF</dt>
<dd>
is used to setup an empty result filter. The arguments
- are the size of the set of results that must be filtered
- at the initiator, and a <tt>Mutator</tt> value which
- <bcp14>MAY</bcp14> be used to deterministically
- re-randomize probabilistic data structures. <tt>RF</tt>
- <bcp14>MUST</bcp14> be a byte sequence suitable for
- transmission over the network.
+ are typically the size of the set of results that must
+ be filtered at the initiator, and a <tt>Mutator</tt>
+ value which <bcp14>MAY</bcp14> be used to
+ deterministically re-randomize probabilistic data
+ structures. <tt>RF</tt> <bcp14>MUST</bcp14> be a byte
+ sequence suitable for transmission over the network.
</dd>
<dt>FilterResult(Block, Key, RF, XQuery) -> (FilterEvaluationResult, RF')</dt>
<dd>
@@ -2581,7 +2581,7 @@ BEGIN
is used to filter results against specific queries.
This function does not check the validity of
<tt>Block</tt> itself or that it matches the given key,
- as this must have been checked earlier. Thus, locally
+ as this must have been checked earlier. Locally
stored blocks from previously observed
<tt>ResultMessages</tt> and <tt>PutMessages</tt> use
this function to perform filtering based on the request
@@ -2632,20 +2632,25 @@ BEGIN
<section anchor="hello_block">
<name>HELLO Blocks</name>
<t>
- For bootstrapping and peer discovery, the DHT implementation uses
- its own block type called "HELLO". <tt>HELLO</tt> blocks are the only type
- of block that <bcp14>MUST</bcp14> be supported by every
- R<sup>5</sup>N implementation. A block with this block type
- contains the peer public key of the peer that published the <tt>HELLO</tt> together
- with a set of addresses of this peer. The key of a <tt>HELLO</tt> block
- is the SHA-512 of the peer public key and thus the peer's identity in the DHT.
+ For bootstrapping and peer discovery, the DHT
+ implementation uses its own block type called "HELLO".
+ <tt>HELLO</tt> blocks are the only type of block that
+ <bcp14>MUST</bcp14> be supported by every R<sup>5</sup>N
+ implementation. A block with this block type contains the
+ peer public key of the peer that published the
+ <tt>HELLO</tt> together with a set of addresses of this
+ peer. The key of a <tt>HELLO</tt> block is the SHA-512
+ hash <xref target="RFC4634"/> of the peer public key and
+ thus the peer's identity in the DHT.
</t>
<t>
- The <tt>HELLO</tt> block type wire format is illustrated in
- <xref target="figure_hello"/>. A query for block of type <tt>HELLO</tt> <bcp14>MUST NOT</bcp14>
- include extended query data (XQuery). Any implementation
- encountering a request for a <tt>HELLO</tt> with non-empty XQuery
- data <bcp14>MUST</bcp14> consider the request invalid and ignore it.
+ The <tt>HELLO</tt> block type wire format is illustrated
+ in <xref target="figure_hello"/>. A query for block of
+ type <tt>HELLO</tt> <bcp14>MUST NOT</bcp14> include
+ extended query data (XQuery). Any implementation
+ encountering a request for a <tt>HELLO</tt> with non-empty
+ XQuery data <bcp14>MUST</bcp14> consider the request
+ invalid and ignore it.
</t>
<figure anchor="figure_hello" title="The HELLO Block Format.">
<artwork name="" type="" align="left" alt=""><![CDATA[
@@ -2675,35 +2680,38 @@ BEGIN
<dl>
<dt>PEER PUBLIC KEY</dt>
<dd>
- is the public key of the peer which has generated this HELLO.
+ is the public key of the peer to which the
+ <tt>ADDRESSES</tt> belong. This is also the public key
+ needed to verify the <tt>SIGNATURE</tt>.
</dd>
<dt>EXPIRATION</dt>
<dd>
- denotes the absolute 64-bit expiration date of the content.
- The value specified is microseconds since midnight (0 hour),
- January 1, 1970, but must be a multiple of one million (so that it
+ denotes the absolute 64-bit expiration date of the
+ content. The value specified is microseconds since
+ midnight (0 hour), January 1, 1970 in network byte
+ order, but must be a multiple of one million (so that it
can be represented in seconds in a <tt>HELLO</tt> URL).
- Stored in network byte order.
</dd>
<dt>ADDRESSES</dt>
<dd>
is a list of UTF-8 addresses (<xref target="terminology"/>)
which can be used to contact the peer.
Each address <bcp14>MUST</bcp14> be 0-terminated.
- The set of addresses MAY be empty.
+ The set of addresses <bcp14>MAY</bcp14> be empty.
</dd>
<dt>SIGNATURE</dt>
<dd>
<t>
- is the signature of the HELLO.
- It covers a 64-bit pseudo header
- derived from the information in the <tt>HELLO</tt> block.
- The pseudo header includes
- the expiration time, a constant that uniquely
- identifies the purpose of the signature,
- and a hash over the addresses.
- The wire format is illustrated
- in <xref target="figure_hellowithpseudo"/>.
+ is the EdDSA signature <xref target="ed25519"/> of the
+ <tt>HELLO</tt> block. The signature covers various
+ information derived from the actual data in the
+ <tt>HELLO</tt> block. The data signed over includes the
+ block expiration time, a constant that uniquely
+ identifies the purpose of the signature, and a hash of
+ the addresses with 0-terminators in the same order as
+ they are present in the <tt>HELLO</tt> block. The
+ format is illustrated in <xref
+ target="figure_hellowithpseudo"/>.
</t>
<figure anchor="figure_hellowithpseudo" title="The Wire Format of the HELLO for Signing.">
<artwork name="" type="" align="left" alt=""><![CDATA[
@@ -2727,88 +2735,97 @@ BEGIN
<dl>
<dt>SIZE</dt>
<dd>
- A 32-bit value containing the length of the signed data in bytes
- in network byte order.
- The length of the signed data <bcp14>MUST</bcp14> be 80 bytes.
+ A 32-bit value containing the length of the signed data
+ in bytes in network byte order. The length of the
+ signed data <bcp14>MUST</bcp14> be 80 bytes.
</dd>
<dt>PURPOSE</dt>
<dd>
- A 32-bit signature purpose flag. This field <bcp14>MUST</bcp14> be 7 (in network
- byte order).
+ A 32-bit signature purpose flag. This field
+ <bcp14>MUST</bcp14> be 7 in network byte order.
</dd>
<dt>EXPIRATION</dt>
<dd>
- denotes the absolute 64-bit expiration date of the HELLO
- in microseconds since midnight (0 hour), January 1, 1970
- in network byte order.
+ denotes the absolute 64-bit expiration date of the
+ <tt>HELLO</tt> in microseconds since midnight (0 hour),
+ January 1, 1970 in network byte order.
</dd>
<dt>H_ADDRS</dt>
<dd>
- a SHA-512 hash over the addresses in the HELLO.
- H_ADDRS is generated over the ADDRESSES field
- as provided in the <tt>HELLO</tt> block using SHA-512 <xref target="RFC4634"/>.
+ a SHA-512 hash over the addresses in the <tt>HELLO</tt>.
+ <tt>H_ADDRS</tt> is generated over the
+ <tt>ADDRESSES</tt> field as provided in the
+ <tt>HELLO</tt> block using SHA-512 <xref
+ target="RFC4634"/>.
</dd>
</dl>
</dd>
</dl>
<t>
- The <tt>HELLO</tt> block functions <bcp14>MUST</bcp14> be implemented
- as follows:
+ The <tt>HELLO</tt> block operations <bcp14>MUST</bcp14> be
+ implemented as follows:
</t>
<dl>
<dt>ValidateBlockQuery(Key, XQuery)
-> RequestEvaluationResult</dt>
<dd>
To validate a block query for a <tt>HELLO</tt> is to
- simply check that the XQuery is empty. If it is empty,
+ simply check that the <tt>XQuery</tt> is empty. If it is empty,
<tt>REQUEST_VALID</tt> ist returned. Otherwise,
<tt>REQUEST_INVALID</tt> is returned.
</dd>
<dt>DeriveBlockKey(Block) -> Key | NONE</dt>
<dd>
To derive a block key for a <tt>HELLO</tt> is to simply
- hash the peer public key from the HELLO. The result of this function
- is always the SHA-512 hash over the PEER PUBLIC KEY.
+ hash the <tt>PEER PUBLIC KEY</tt> from the <tt>HELLO</tt>. The
+ result of this function is thus always the SHA-512 hash over
+ the <tt>PEER PUBLIC KEY</tt>.
</dd>
<dt>ValidateBlockStoreRequest(Block)
-> BlockEvaluationResult</dt>
<dd>
- To validate a block store request is to verify
- the EdDSA <tt>SIGNATURE</tt> over the hashed <tt>ADDRESSES</tt>
- against the public key from the PEER PUBLIC KEY field.
- If the signature is valid BLOCK_VALID is returned.
- Otherwise BLOCK_INVALID.
+ To validate a block store request is to verify the EdDSA
+ <tt>SIGNATURE</tt> over the hashed <tt>ADDRESSES</tt>
+ against the public key from the <tt>PEER PUBLIC KEY</tt>
+ field. If the signature is valid <tt>BLOCK_VALID</tt> is
+ returned. Otherwise <tt>BLOCK_INVALID</tt> is returned.
</dd>
<dt>SetupResultFilter(FilterSize, Mutator) -> RF</dt>
<dd>
<t>
- The RESULT_FILTER for <tt>HELLO</tt> blocks is implemented using a
- Bloom filter following the definition from <xref target="bloom_filters"/>
- and consists of a variable number of bits <tt>L</tt>.
- <tt>L</tt> depends on the number of connected peers <tt>|E|</tt> known to
- the peer creating a <tt>HELLO</tt> block from its own addresses:
- <tt>L</tt> is set to the minimum of
- 2<sup>18</sup> bits (2<sup>15</sup> bytes) and the lowest power
- of 2 that is strictly larger than <tt>2*K*|E|</tt> bits (<tt>K*|E|/4</tt> bytes).
+ The <tt>RF</tt> for <tt>HELLO</tt> blocks is implemented
+ using a Bloom filter following the definition from <xref
+ target="bloom_filters"/> and consists of a variable number
+ of bits <tt>L</tt>. <tt>L</tt> depends on the
+ <tt>FilterSize</tt> which will be the number of
+ connected peers <tt>|E|</tt> known to the peer creating a
+ <tt>HELLO</tt> block from its own addresses: <tt>L</tt> is
+ set to the minimum of 2<sup>18</sup> bits (2<sup>15</sup>
+ bytes) and the lowest power of 2 that is strictly larger
+ than <tt>2*K*|E|</tt> bits (<tt>K*|E|/4</tt> bytes).
</t>
<t>
- The <tt>k</tt>-value for the Bloom filter is 16.
- The elements used in the Bloom filter
- consist of an XOR between the <tt>H_ADDRS</tt> field (as computed using
- SHA-512 over the <tt>ADDRESSES</tt>) and the SHA-512
- hash of the <tt>MUTATOR</tt> field from a given <tt>HELLO</tt> block.
- The mapping function M(<tt>H_ADDRS XOR MUTATOR</tt>) is defined as follows:
+ The <tt>k</tt>-value for the Bloom filter
+ <bcp14>MUST</bcp14> be 16. The elements used in the Bloom
+ filter consist of an XOR between the <tt>H_ADDRS</tt>
+ field (as computed using SHA-512 over the
+ <tt>ADDRESSES</tt>) and the SHA-512 hash of the
+ <tt>MUTATOR</tt> field from a given <tt>HELLO</tt> block.
+ The mapping function M(<tt>H_ADDRS XOR MUTATOR</tt>) is
+ defined as follows:
</t>
<t>
<tt>M(e = H_ADDR XOR MUTATOR) -> e as uint32[]</tt>
</t>
<t>
- <tt>M</tt> is an identity function and returns the 512-bit XOR result unmodified.
- This resulting byte string is interpreted as k=16 32-bit
- integers in network byte order which are used to set and check the bits in
- <tt>B</tt> using <tt>BF-SET()</tt> and <tt>BF-TEST()</tt>.
- The 32-bit Mutator is prepended to the L-bit Bloom filter field <tt>HELLO_BF</tt> containing <tt>B</tt>
- to create the result filter for a <tt>HELLO</tt> block:
+ <tt>M</tt> is an identity function and returns the 512-bit
+ XOR result unmodified. This resulting byte string is
+ interpreted as k=16 32-bit integers in network byte order
+ which are used to set and check the bits in <tt>B</tt>
+ using <tt>BF-SET()</tt> and <tt>BF-TEST()</tt>. The
+ 32-bit <tt>MUTATOR</tt> is prepended to the L-bit Bloom filter
+ field <tt>HELLO_BF</tt> containing <tt>B</tt> to create
+ the result filter for a <tt>HELLO</tt> block:
</t>
<figure anchor="hello_rf" title="The HELLO Block Result Filter.">
<artwork name="" type="" align="left" alt=""><![CDATA[
@@ -2832,54 +2849,56 @@ BEGIN
</dd>
</dl>
<t>
- The <tt>MUTATOR</tt> value is used
- to additionally "randomize" the computation of the Bloom filter while
- remaining deterministic across peers.
- It is only ever set by the peer initiating the GET
- request, and changed every time the GET request is repeated.
- Peers forwarding GET requests <bcp14>MUST</bcp14> not change the
- mutator value included in the <tt>RESULT_FILTER</tt> as they might not
- be able to recalculate the result filter with a different <tt>MUTATOR</tt>
- value.
+ The <tt>MUTATOR</tt> value is used to additionally
+ "randomize" the computation of the Bloom filter while
+ remaining deterministic across peers. It is only ever set
+ by the peer initiating the GET request, and changed every
+ time the GET request is repeated. Peers forwarding GET
+ requests <bcp14>MUST</bcp14> not change the mutator value
+ included in the <tt>RESULT_FILTER</tt> as they might not
+ be able to recalculate the result filter with a different
+ <tt>MUTATOR</tt> value.
</t>
<t>
- Consequently, repeated
- requests have statistically independent probabilities of creating
- false-positives in a result filter. Thus, even if for one request
- a result filter may exclude a result as a false-positive
- match, subsequent requests are likely to not have the same
+ Consequently, repeated requests have statistically
+ independent probabilities of creating false-positives in a
+ result filter. Thus, even if for one request a result
+ filter may exclude a result as a false-positive match,
+ subsequent requests are likely to not have the same
false-positives.
</t>
<t>
- <tt>HELLO</tt> result filters can be merged if the
- Bloom filters have the same size and
- <tt>MUTATOR</tt> by setting all bits to 1 that are
- set in either Bloom filter. This is done whenever
- a peer receives a query with the same <tt>MUTATOR</tt>,
- predecessor and Bloom filter size.
+ <tt>HELLO</tt> result filters can be merged if the Bloom
+ filters have the same size and <tt>MUTATOR</tt> by setting
+ all bits to 1 that are set in either Bloom filter. This
+ is done whenever a peer receives a query with the same
+ <tt>MUTATOR</tt>, predecessor and Bloom filter size.
</t>
</dd>
<dt>FilterResult(Block, Key, RF, XQuery) -> (FilterEvaluationResult, RF')</dt>
<dd>
- The <tt>H_ADDRS</tt> field is XORed with the SHA-512
- hash of the <tt>MUTATOR</tt> field from the <tt>HELLO</tt> block and the resulting
- value is checked against the Bloom filter in RF.
- Consequently, HELLOs with completely identical sets of
- addresses will be filtered and FILTER_DUPLICATE is returned.
- Any small variation in the set of addresses will cause the block
- to no longer be filtered (with high probability) and
- FILTER_MORE is returned.
+ The <tt>H_ADDRS</tt> field is XORed with the SHA-512 hash
+ of the <tt>MUTATOR</tt> field from the <tt>HELLO</tt>
+ block and the resulting value is checked against the
+ Bloom filter in <tt>RF</tt>. Consequently,
+ <tt>HELLOs</tt> with completely identical sets of
+ addresses will be filtered and <tt>FILTER_DUPLICATE</tt>
+ is returned. Any small variation in the set of addresses
+ will cause the block to no longer be filtered (with high
+ probability) and <tt>FILTER_MORE</tt> is returned.
</dd>
</dl>
</section>
<section>
<name>Persistence</name>
<t>
- An implementation <bcp14>SHOULD</bcp14> provide a local persistence mechanism for
- blocks. Embedded systems that lack storage capability <bcp14>MAY</bcp14> use
- connection-level signalling to indicate that they are merely a client utilizing a
- DHT and are not able to participate with storage.
- The local storage <bcp14>MUST</bcp14> provide the following functionality:
+ An implementation <bcp14>SHOULD</bcp14> provide a local
+ persistence mechanism for blocks. Embedded systems that
+ lack storage capability <bcp14>MAY</bcp14> use
+ connection-level signalling to indicate that they are merely
+ a client utilizing a DHT and are not able to participate
+ with storage. The local storage <bcp14>MUST</bcp14> provide
+ the following functionality:
</t>
<dl>
<dt>Store(Key, Block)</dt>