commit 128a1dbad6d2fa1677674784d2cfe77f5c2b2284
parent ada4dc07acf571ab5c0dabb69c3dbc3303ca5474
Author: Martin Schanzenbach <schanzen@gnunet.org>
Date: Sun, 25 Dec 2022 17:52:45 +0900
First pass overhault message processing (with FIXMEs)
Diffstat:
1 file changed, 156 insertions(+), 48 deletions(-)
diff --git a/draft-schanzen-r5n.xml b/draft-schanzen-r5n.xml
@@ -924,11 +924,33 @@ BEGIN
<section anchor="p2p_messages" numbered="true" toc="default">
<name>Message Processing</name>
<t>
+ Further, the implementation <bcp14>MAY</bcp14> act as an initiator of
+ messages.
+ If instructed through an application-facing API such as the one outlined
+ in <xref target="overlay"/>, the peer may acts as an initiator of <tt>GetMessage</tt>s
+ or <tt>PutMessage</tt>s.
+ The status of initiator is relevant for peers when processing <tt>ResultMessages</tt>
+ and the potential handover of results to the application.
+ <!-- FIXME: Are hello messages should or must?
+ -->
+ Initiation of <tt>HelloMessages</tt> is <bcp14>RECOMMENDED</bcp14>.
+ <tt>HelloMessage</tt>s are used to inform neighbors of
+ a peer about the sender's available addresses. The
+ recipients use these messages to inform their respective
+ Underlays about ways to sustain the connections and to
+ generate HELLO blocks (see <xref target="hello_block"/>)
+ to answer peer discovery queries
+ from other peers.
+ </t>
+ <t>
The implementation <bcp14>MUST</bcp14> listen for <tt>RECEIVE(P, M)</tt> signals
from the Underlay and respond to the respective messages sent by
the peer <tt>P</tt>.
- In the following, the wire formats of the messages and the required
- processing are detailed.
+ </t>
+ <t>
+ Wheather initiated locally or received from a neighbour, the implementation
+ processes the messages according to the wire formats and the required
+ validations detailed in the following.
Where required, the local peer's ID is referred to as <tt>SELF</tt>.
</t>
<section anchor="message_components">
@@ -1229,13 +1251,7 @@ BEGIN
<!-- FIXME: While it is discussed here how to hangle HelloMessages
nowhere in the draft is a HelloMessage created at the
initiator so it is completely irrelevant atm -->
- <tt>HelloMessage</tt>s are used to inform neighbors of
- a peer about the sender's available addresses. The
- recipients use these messages to inform their respective
- Underlays about ways to sustain the connections and to
- generate HELLO blocks (see <xref target="hello_block"/>)
- to answer peer discovery queries
- from other peers. In contrast to a HELLO block, a
+ In contrast to a HELLO block, a
<tt>HelloMessage</tt> does not contain the ID of
the peer the address information is about: in a
<tt>HelloMessage</tt>, the address information is always
@@ -1338,6 +1354,12 @@ BEGIN
<name>PutMessage</name>
<t>
<tt>PutMessage</tt>s are used to store information at other peers in the DHT.
+ Any API which allows applications to initiate <tt>PutMessage</tt>s needs to
+ provide sufficient, implementation-specific information needed to construct
+ the initial <tt>PutMessage</tt>.
+ For example, implementations supporting multiple applications and blocks will
+ have block type and message flag parameters in addition to the actual data
+ payload and key.
</t>
<section anchor="p2p_put_wire">
<name>Wire Format</name>
@@ -1372,52 +1394,69 @@ BEGIN
<dt>MSIZE</dt>
<dd>
denotes the size of this message in network byte order.
+ Set by the initiator.
+ Modified by processing peers when message contents such as the path lengths change.
</dd>
<dt>MTYPE</dt>
<dd>
- is the 16-bit message type. It must be set to
- the value 146 in network byte order.
+ is the 16-bit message type. It is set by the initiator to
+ the value 146 in network byte order. Read-only.
</dd>
<dt>BTYPE</dt>
<dd>
- is a 32-bit block type. The block type indicates the content
- type of the payload. In network byte order.
+ is a 32-bit block type.
+ The block type indicates the content
+ type of the payload.
+ Set by the initiator. Read-only.
+ In network byte order.
</dd>
<dt>FLAGS</dt>
<dd>
is a 16-bit vector with binary options (see <xref target="route_flags"/>).
+ Set by the initiator. Read-only.
</dd>
<dt>HOPCOUNT</dt>
<dd>
is a 16-bit number indicating how many hops this message has
- traversed to far. In network byte order.
+ traversed to far.
+ Set by the initiator to 0.
+ Incremented by processing peers.
+ In network byte order.
</dd>
<dt>REPL_LVL</dt>
<dd>
is a 16-bit number indicating the desired replication level of
- the data. In network byte order.
+ the data.
+ Set by the initiator. Read-only.
+ In network byte order.
</dd>
<dt>PATH_LEN</dt>
<dd>
is a 16-bit number indicating the number of Path Elements
recorded in PUTPATH.
As PUTPATH is optional, this value may be zero.
+ If the PUTPATH is enabled, set initially to 0 by the initiator.
+ Incremented by processing peers.
In network byte order.
</dd>
<dt>EXPIRATION</dt>
<dd>
denotes the absolute 64-bit expiration date of the content.
+ Set by the initiator. Read-only.
In microseconds since midnight (0 hour), January 1, 1970 in network
byte order.
</dd>
<dt>PEER_BF</dt>
<dd>
A peer Bloom filter to stop circular routes (see <xref target="routing_bloomfilter"/>).
+ Set by the initiator to include itself. <!-- FIXME: Is that so? -->
+ Modified by processing peers to include their own peer ID.
</dd>
<dt>BLOCK_KEY</dt>
<dd>
The key under which the <tt>PutMessage</tt> wants to store content
under.
+ Set by the initiator. Read-only.
</dd>
<dt>TRUNCATED ORIGIN</dt>
<dd>
@@ -1431,11 +1470,14 @@ BEGIN
this public key must be used. Note that
due to the truncation, this last hop
cannot be verified to exist.
+ Value is modified by processing peers.
</dd>
<dt>PUTPATH</dt>
<dd>
the variable-length PUT path.
The path consists of a list of PATH_LEN Path Elements.
+ Set by the initiator to 0.
+ Incremented by processing peers.
</dd>
<dt>LAST HOP SIGNATURE</dt>
<dd>
@@ -1447,12 +1489,14 @@ BEGIN
the predecessor (all zeros if PATH_LEN is 0,
otherwise the last peer in PUTPATH) to
the target peer.
+ Modified by processing peers (if flag is set).
</dd>
<dt>BLOCK</dt>
<dd>
the variable-length block payload. The contents are determined
by the BTYPE field. The length is determined by MSIZE minus
the size of all of the other fields.
+ Set by the initiator. Read-only.
</dd>
</dl>
</section>
@@ -1460,6 +1504,7 @@ BEGIN
<name>Processing</name>
<t>
Upon receiving a <tt>PutMessage</tt> from a peer <tt>P</tt>
+ , or created through initiation by an overlay API,
an implementation <bcp14>MUST</bcp14> process it step by step as follows:
</t>
<ol>
@@ -1491,13 +1536,18 @@ BEGIN
The peer address of the sender peer <tt>P</tt> <bcp14>SHOULD</bcp14> be in <tt>PEER_BF</tt>.
If not, the implementation <bcp14>MAY</bcp14> log an error, but <bcp14>MUST</bcp14> continue.
</li>
- <li>
+ <!--<li>
+ FIXME: I do not know what is going on here. "local peer address"
+ is certainly wrong.
+ But then, looking at the PathElement description, so wold be local peer ID.
+ Also, no signatures are ever created in this processing
If the <tt>RecordRoute</tt> flag is set in FLAGS,
the local peer address <bcp14>MUST</bcp14> be appended to the <tt>PUTPATH</tt>
- of the message. If the flag is not set, the <tt>PATH_LEN</tt>
- <bcp14>MUST</bcp14> be set to zero.
- </li>
+ of the message.
+ </li>-->
<li>
+ If the flag is not set, the <tt>PATH_LEN</tt>
+ <bcp14>MUST</bcp14> be set to zero.
If the <tt>PATH_LEN</tt> is non-zero,
the local peer <bcp14>SHOULD</bcp14> verify the signatures from the <tt>PUTPATH</tt>.
Verification <bcp14>MAY</bcp14> involve checking all signatures or any random
@@ -1533,12 +1583,16 @@ BEGIN
number of peers to forward the message to. The implementation <bcp14>MAY</bcp14>
forward to fewer or no peers in order to handle resource constraints
such as limited bandwidth.
- Finally, the local peer address <bcp14>MUST</bcp14> be added to the
- <tt>PEER_BF</tt> before forwarding the message.
- For all peers with peer address <tt>P</tt> selected to forward the message
- to, <tt>SEND(P, PutMessage')</tt> is called. Here, <tt>PutMessage'</tt>
- is the original message with updated fields. In particular, <tt>HOPCOUNT</tt>
- <bcp14>MUST</bcp14> be incremented by 1.
+ <!-- FIXME: From above. it seems to me that here we need to add a new putpath
+ signature element (as we know the successor now)-->
+ For each selected peer with peer address <tt>P</tt> a dedicated <tt>PutMessage'</tt>
+ is created containing the original (and where applicable already updated) fields
+ of the received <tt>PutMessage</tt>.
+ In each message the local peer address <bcp14>MUST</bcp14> be added to the
+ <tt>PEER_BF</tt> and the <tt>HOPCOUNT</tt> is incremented by 1.
+ If the <tt>RecordRoute</tt> flag is set, a new Path Element is created and added
+ to the <tt>PUTPATH</tt> fields and the <tt>PATH_LEN</tt> field is incremented by 1.
+ Finally, <tt>SEND(P, PutMessage')</tt> is called.
</li>
</ol>
</section>
@@ -1547,7 +1601,11 @@ BEGIN
<name>GetMessage</name>
<t>
<tt>GetMessage</tt>s are used to request information from other peers in the DHT.
- </t>
+ Any overlay API which allows applications to initiate <tt>GetMessage</tt>s needs to provide
+ sufficient, implementation-specific information needed to construct the initial <tt>GetMessage</tt>.
+ For example, implementations supporting multiple applications and blocks will have block type and
+ message flag parameters.
+ </t>
<section anchor="p2p_get_wire">
<name>Wire Format</name>
<figure anchor="figure_getmsg" title="The GetMessage Wire Format.">
@@ -1576,54 +1634,71 @@ BEGIN
<dt>MSIZE</dt>
<dd>
denotes the size of this message in network byte order.
+ Set by the initiator.
+ <!-- FIXME: Is this not fixed-length once set by initiator?-->
+ Modified by processing peers when message contents change.
</dd>
<dt>MTYPE</dt>
<dd>
- is the 16-bit message type. It must be set to
- the value 147 in network byte order.
+ is the 16-bit message type. It is set by the initiator to
+ the value 147 in network byte order. Read-only.
</dd>
<dt>BTYPE</dt>
<dd>
is a 32-bit block type field. The block type indicates the content
- type of the payload. In network byte order.
+ type of the payload. Set by the initiator. Read-only. In network byte order.
</dd>
<dt>FLAGS</dt>
<dd>
is a 16-bit vector with binary options (see <xref target="route_flags"/>).
+ Set by the initiator. Read-only.
</dd>
<dt>HOPCOUNT</dt>
<dd>
is a 16-bit number indicating how many hops this message has
- traversed to far. In network byte order.
+ traversed to far.
+ Set by the initiator to 0.
+ Incremented by processing peers.
+ In network byte order.
</dd>
<dt>REPL_LVL</dt>
<dd>
is a 16-bit number indicating the desired replication level of
- the data. In network byte order.
+ the data.
+ Set by the initiator. Read-only.
+ In network byte order.
</dd>
<dt>RF_SIZE</dt>
<dd>
is a 16-bit number indicating the length of the
- result filter RESULT_FILTER. In network byte order.
+ result filter RESULT_FILTER.
+ Set by the initiator. Read-only. <!--FIXME is that so? -->
+ In network byte order.
</dd>
<dt>PEER_BF</dt>
<dd>
A peer Bloom filter to stop circular routes (see <xref target="routing_bloomfilter"/>).
+ Set by the initiator to include itself. <!-- Is that so? -->
+ Modified by processing peers to include their own peer address.
</dd>
<dt>QUERY_HASH</dt>
<dd>
- The query used to indicate what the key is under which the originator is looking
+ The query used to indicate what the key is under which the initiator is looking
for blocks with this request.
The block type may use a different evaluation logic to determine
applicable result blocks.
+ Set by the initiator. Read-only.
</dd>
<dt>RESULT_FILTER</dt>
<dd>
the variable-length result filter, described in <xref target="result_filter"/>.
+ Set by the initiator.
+ Modified by processing peers.
</dd>
<dt>XQUERY</dt>
<dd>
the variable-length extended query. Optional.
+ Set by the initiator. <!-- FIXME: Modified by processing peers? -->
</dd>
</dl>
</section>
@@ -1637,8 +1712,9 @@ BEGIN
which matches the query key <bcp14>MUST</bcp14> check the result filter
and only send a reply message if the result does not test positive
under the result filter. Before forwarding the <tt>GetMessage</tt>, the
- result filter <bcp14>MUST</bcp14> be updated to filter out all results
- already returned by the local peer.
+ result filter <bcp14>MUST</bcp14> be updated using the result of the <tt>BTYPE</tt>-specific
+ <tt>FilterResult</tt> (see <xref target="block_functions"/>) function to filter
+ out all results already returned by the local peer.
</t>
<t>
How a result filter is implemented depends on the block type
@@ -1647,16 +1723,17 @@ BEGIN
it is possible that a desireable result is filtered by a result
filter because of a false-positive test.
</t>
- <t>
+ <t>
+ <!-- FIXME: then this should be part of the registration policy -->
How exactly a block result is added to a result filter
- <bcp14>MUST</bcp14> be
- specified as part of the definition of a block type.
+ <bcp14>MUST</bcp14> be specified as part of the definition of a block type.
</t>
</section>
<section anchor="p2p_get_processing">
<name>Processing</name>
<t>
- Upon receiving a <tt>GetMessage</tt> from a peer an
+ Upon receiving a <tt>GetMessage</tt> from a peer <tt>P</tt>, or
+ created through initiation by the overlay API, an
implementation <bcp14>MUST</bcp14> process it step by step as follows:
</t>
<ol>
@@ -1667,7 +1744,7 @@ BEGIN
If validation
function yields <tt>REQUEST_INVALID</tt>, the message <bcp14>MUST</bcp14> be discarded.
If the <tt>BTYPE</tt> is not supported, the message <bcp14>MUST</bcp14>
- be forwarded.
+ be forwarded (Skip to step 4).
If the <tt>BTYPE</tt> is <tt>ANY</tt>, the message is processed
without validation.
</li>
@@ -1736,8 +1813,8 @@ BEGIN
peer address <tt>SELF</tt>.
For all peers with peer address <tt>P'</tt> chosen to forward the message
to, <tt>SEND(P', GetMessage')</tt> is called. Here, <tt>GetMessage'</tt>
- is the original message with updated fields. In particular, <tt>HOPCOUNT</tt>
- <bcp14>MUST</bcp14> be incremented by 1.
+ is the original message with the updated fields for <tt>HOPCOUNT</tt> (incremented
+ by 1), <tt>PEER_BF</tt> and <tt>RESULT_FILTER</tt>.
</li>
</ol>
</section>
@@ -1745,7 +1822,10 @@ BEGIN
<section anchor="p2p_result" numbered="true" toc="default">
<name>ResultMessage</name>
<t>
- <tt>ResultMessage</tt>s are used to return information to other peers in the DHT.
+ <tt>ResultMessage</tt>s are used to return information to other peers in the DHT
+ or to applications using the overlay API that previously initiated a <tt>GetMessage</tt>.
+ The initiator of a <tt>ResultMessage</tt> is a peer triggered through the processing
+ of a <tt>GetMessage</tt>.
</t>
<section anchor="p2p_result_wire">
<name>Wire Format</name>
@@ -1782,16 +1862,21 @@ BEGIN
<dt>MSIZE</dt>
<dd>
denotes the size of this message in network byte order.
+ Set by the initiator.
+ Updated by processing peers.
</dd>
<dt>MTYPE</dt>
<dd>
is the 16-bit message type. It must be set to
the value 148 in network byte order.
+ Set by the initiator. Read-only.
</dd>
<dt>BTYPE</dt>
<dd>
is a 32-bit block type field. The block type indicates the content
- type of the payload. In network byte order.
+ type of the payload.
+ Set by the initiator. Read-only.
+ In network byte order.
</dd>
<dt>RESERVED</dt>
<dd>
@@ -1803,12 +1888,16 @@ BEGIN
<dt>FLAGS</dt>
<dd>
is a 16-bit vector with binary options (see <xref target="route_flags"/>).
+ Set by the initiator. <!-- FIXME to what? -->
</dd>
<dt>PUTPATH_L</dt>
<dd>
is a 16-bit number indicating the number of Path Elements recorded
in <tt>PUTPATH</tt>. As <tt>PUTPATH</tt> is optional, this value may be zero
even if the message has traversed several peers.
+ Set by the initiator to the <tt>PATH_LEN</tt> of the <tt>PutMessage</tt>
+ from which the block originated.
+ Modified by processing peers in case of path truncation.
In network byte order.
</dd>
<dt>GETPATH_L</dt>
@@ -1816,6 +1905,8 @@ BEGIN
is a 16-bit number indicating the number of Path Elements
recorded in <tt>GETPATH</tt>. As <tt>GETPATH</tt> is optional, this value may be zero
even if the message has traversed several peers.
+ Set by the initiator to 0.
+ Modified by processing peers.
In network byte order.
</dd>
<dt>EXPIRATION</dt>
@@ -1823,11 +1914,16 @@ BEGIN
denotes the absolute 64-bit expiration date of the content.
In microseconds since midnight (0 hour), January 1, 1970 in network
byte order.
+ Set by the initiator to the expiration value as recorded from
+ the <tt>PutMessage</tt> from which the block originated.
+ Read-only.
</dd>
<dt>QUERY_HASH</dt>
<dd>
the query hash corresponding to the <tt>GetMessage</tt> which
caused this reply message to be sent.
+ Set by the initiator using the value of the <tt>GetMessage</tt>.
+ Read-only.
</dd>
<dt>TRUNCATED ORIGIN</dt>
<dd>
@@ -1841,20 +1937,25 @@ BEGIN
this public key must be used. Note that
due to the truncation, this last hop
cannot be verified to exist.
+ Set by processing peers.
</dd>
<dt>PUTPATH</dt>
<dd>
the variable-length PUT path.
The path consists of a list of <tt>PUTPATH_L</tt> Path Elements.
+ Set by the initiator to the the <tt>PUTPATH</tt> of the <tt>PutMessage</tt>
+ from which the block originated.
+ Modified by processing peers in case of path truncation.
</dd>
<dt>GETPATH</dt>
<dd>
the variable-length PUT path.
The path consists of a list of <tt>GETPATH_L</tt> Path Elements.
+ Set by processing peers.
</dd>
<dt>LAST HOP SIGNATURE</dt>
<dd>
- is only provided if the RECORD ROUTE flag
+ is only provided if the <tt>RecordRoute</tt> flag
is set in FLAGS. If present, this is
an EdDSA signature of the sender of this message
(using the same format as the signatures in PUTPATH)
@@ -1867,13 +1968,15 @@ BEGIN
<dd>
the variable-length resource record data payload.
The contents are defined by the respective type of the resource record.
+ Set by the initiator. Read-only.
</dd>
</dl>
</section>
<section anchor="p2p_result_processing">
<name>Processing</name>
<t>
- Upon receiving a <tt>ResultMessage</tt> from a connected peer
+ Upon receiving a <tt>ResultMessage</tt> from a connected peer or
+ triggered by the processing of a <tt>GetMessage</tt>,
an implementation <bcp14>MUST</bcp14> process it step by step as follows:
</t>
<ol>
@@ -1960,12 +2063,17 @@ BEGIN
</ol>
<t>
If the result is either <tt>FILTER_MORE</tt> or <tt>FILTER_LAST</tt>,
- the result is forwarded to the origin of the query. If the result
- was <tt>FILTER_LAST</tt>, the query is removed from the list of pending
+ the message is forwarded to the origin of the query through
+ either the overlay API or using <tt>SEND(P, ResultMessage')</tt> where
+ <tt>ResultMessage'</tt> is the now modified message.
+ If the result was <tt>FILTER_LAST</tt>, the query is removed from the list of pending
queries.
</t>
</li>
<li>
+ <!-- FIXME: If that is the case, then this also needs to be in the
+ processing of GetMessage: I think we shoul dmove this to some
+ performance considerations instead. -->
Finally, the implementation <bcp14>MAY</bcp14> choose to
cache data from <tt>ResultMessage</tt>s.
</li>
