summaryrefslogtreecommitdiff
path: root/draft-schanzen-r5n.xml
diff options
context:
space:
mode:
Diffstat (limited to 'draft-schanzen-r5n.xml')
-rw-r--r--draft-schanzen-r5n.xml345
1 files changed, 174 insertions, 171 deletions
diff --git a/draft-schanzen-r5n.xml b/draft-schanzen-r5n.xml
index 28610a6..4fbfd79 100644
--- a/draft-schanzen-r5n.xml
+++ b/draft-schanzen-r5n.xml
@@ -331,7 +331,7 @@ Connectivity | |Underlay| |Underlay|
331 required depending on the respective <tt>Block-Type</tt>. 331 required depending on the respective <tt>Block-Type</tt>.
332 A <tt>Block-Type</tt> must define if the <tt>XQuery</tt> can or must 332 A <tt>Block-Type</tt> must define if the <tt>XQuery</tt> can or must
333 be used and what the specific format of its contents should be. 333 be used and what the specific format of its contents should be.
334 See also <xref target="block_types"/>. 334 See also <xref target="blockstorage"/>.
335 </dd> 335 </dd>
336 <dt>Result-Filter:</dt> 336 <dt>Result-Filter:</dt>
337 <dd> 337 <dd>
@@ -1359,168 +1359,108 @@ Connectivity | |Underlay| |Underlay|
1359 </section> 1359 </section>
1360 <section anchor="blockstorage" numbered="true" toc="default"> 1360 <section anchor="blockstorage" numbered="true" toc="default">
1361 <name>Block Storage</name> 1361 <name>Block Storage</name>
1362 <t>
1363 An implementation MUST provide a local persistence mechanism for
1364 blocks.
1365 The local storage MUST provide the following API:
1366 </t>
1367 <dl>
1368 <dt>Store(Key, Block)</dt>
1369 <dd>
1370 Stores a block under the specified key.
1371 </dd>
1372 <dt>Lookup(Key) -&gt; List of Blocks</dt>
1373 <dd>
1374 Retrieves the blocks stored under the specified key.
1375 </dd>
1376 <dt>LookupApproximate(Key) -&gt; List of Blocks</dt>
1377 <dd>
1378 Retrieves the blocks stored under the specified key and
1379 any blocks under keys close to the specified key.
1380 </dd>
1381 </dl>
1382 <t>
1383 Over time a peer may accumulate a significant number of blocks
1384 which are stored locally in the persistence layer.
1385 Due to the expected high number of blocks, the method to
1386 retrieve blocks close to the specified lookup key in the
1387 <tt>LookupApproximate</tt> API must be implemented with care
1388 with respect to efficiency.
1389 </t>
1390 <t>
1391 It is <bcp14>RECOMMENDED</bcp14> to limit the number of results
1392 from the <tt>LookupApproximate</tt> procedure to a result size
1393 which is manageable by the local system.
1394 </t>
1395 <t>
1396 In order to efficiently find a suitable result set, the implementation
1397 SHOULD follow the following procedure:
1398 </t>
1399 <ol>
1400 <li>
1401 Sort all blocks by the block key in ascending (decending) order.
1402 The block keys are interpreted as integer.
1403 </li>
1404 <li>
1405 Alternatingly select a block with a key larger and smaller from
1406 the sortings.
1407 The resulting set is sorted by XOR distance.
1408 The selection process continues until the upper bound for the
1409 result set is reached and both sortings do not yield any closer
1410 blocks.
1411 </li>
1412 </ol>
1413 <t>
1414 An implementation MAY decide to use a custom algorithm in order to
1415 find the closest blocks in the local storage.
1416 But, especially for more primitive approaches, such as only
1417 comparing XOR distances for all blocks in the storage, the
1418 procedure may become ineffective for large storages.
1419 </t>
1420 <!-- FIXME the result set is then filtered again by the block
1421 plugin. But we should discuss this elsewhere -->
1422 <section> 1362 <section>
1423 <name>Block Processing</name> 1363 <name>Blocks</name>
1424 <t>
1425 Block validation may be necessary for both request as well as
1426 reply messages.
1427 When evaluating request messages and their metadata, the possible
1428 evaluation results are:
1429 </t>
1430 <!--<t>RequestEvaluationResult</t>-->
1431 <dl>
1432 <dt>REQUEST_VALID</dt>
1433 <dd>Query is valid, no reply given.</dd>
1434 <dt>REQUEST_INVALID</dt>
1435 <dd>
1436 Query format does not match block type. For example, XQuery not
1437 given or of size of XQuery is not appropriate for type.
1438 </dd>
1439 </dl>
1440 <t>
1441 When evaluating result messages, the possible evaluation results
1442 are:
1443 </t>
1444 <!-->t>ReplyEvaluationResult</t>-->
1445 <!-- FIXME this is not integrated into the message processing
1446 or block processing. We need to specifiy where this is used.
1447 My guess is that we should handle result types in the message
1448 processing descriptions -->
1449 <dl>
1450 <dt>OK_MORE</dt>
1451 <dd>Valid result, and there may be more.</dd>
1452 <dt>OK_LAST</dt>
1453 <dd>Last possible valid result.</dd>
1454 <dt>OK_DUPLICATE</dt>
1455 <dd>Valid result, but duplicate.</dd>
1456 <dt>RESULT_INVALID</dt>
1457 <dd>Invalid result. Block does not match query. Value = 4.</dd>
1458 <dt>RESULT_IRRELEVANT</dt>
1459 <dd>Block does not match xquery. Valid result, but not relevant for the request.</dd>
1460 </dl>
1461 </section>
1462 <section anchor="block_functions">
1463 <name>Block Functions</name>
1464 <t>
1465 Any block type implementation <bcp14>MUST</bcp14> implement the following functions.
1466 </t>
1467 <dl>
1468 <dt>ValidateBlockQuery(Key, XQuery) -&gt; RequestEvaluationResult</dt>
1469 <dd>
1470 is used to evaluate the request for a block. It is used as part of
1471 <tt>GetMessage</tt> processing, where the block payload is still unkown,
1472 but the block <tt>XQuery</tt> <!--(FIXME: Undefined here)-->
1473 and <tt>Key</tt> can and
1474 <bcp14>MUST</bcp14> be verified, if possible.
1475 </dd>
1476 <dt>ValidateBlockStoreRequest(Block, Key) -&gt; RequestEvaluationResult</dt>
1477 <dd>
1478 is used to evaluate a block including its key and payload.
1479 It is used as part of <tt>PutMessage</tt> processing.
1480 The validation <bcp14>MUST</bcp14> include a check of the block payload against the
1481 <tt>Key</tt> under which it is requested to be stored.
1482 </dd>
1483 <dt>ValidateBlockReply(Block, XQuery, Key) -&gt; ReplyEvaluationResult</dt>
1484 <dd>
1485 is used to evaluate a block including its <tt>Key</tt> and payload.
1486 It is used as part <tt>ResultMessage</tt> processing.
1487 The validation of the respective <tt>Block</tt> requires a pending local query
1488 or a previously routed request of another peer and its associated
1489 XQuery data and Key.
1490 The validation <bcp14>MUST</bcp14> include a check of the block payload against the
1491 key under which it is requested to be stored.
1492 </dd>
1493 <dt>DeriveBlockKey(Block) -&gt; Key</dt>
1494 <dd>
1495 is used to synthesize the block key from the block payload
1496 and metadata. It is used as part of FIND-peer message processing.
1497 </dd>
1498 <dt>FilterResult(Block, XQuery, Key) -&gt; ReplyEvaluationResult</dt>
1499 <dd>
1500 is used to filter results stored in the local block storage for local
1501 queries. Locally stored blocks from previously observed
1502 <tt>ResultMessages</tt> and <tt>PutMessages</tt> <bcp14>MAY</bcp14> use this
1503 function instead of <tt>ValidateBlockReply</tt> in order to
1504 avoid revalidation of the block and only perform filtering based on
1505 request parameters.
1506 </dd>
1507 </dl>
1508 </section>
1509 <section anchor="block_types">
1510 <name>Block Types</name>
1511 <t> 1364 <t>
1512 Applications can and should define their own block types. 1365 Applications can and should define their own block types.
1513 The block type determines the format and handling of the block 1366 The block type determines the format and handling of the block
1514 payload by peers in PUT and RESULT messages. 1367 payload by peers in PUT and RESULT messages.
1515 Block types <bcp14>MUST</bcp14> be registered with GANA <xref target="gana"/>. 1368 Block types <bcp14>MUST</bcp14> be registered with GANA <xref target="gana"/>.
1516 </t> 1369 </t>
1517 <t> 1370 <section>
1518 For bootstrapping and peer discovery, the DHT implementation uses 1371 <name>Block Processing</name>
1519 its own block type called "HELLO". A block with this block type 1372 <t>
1520 contains the peerID of the peer initiating the GET request. 1373 Block validation may be necessary for both request as well as
1521 </t> 1374 reply messages.
1375 When evaluating request messages and their metadata, the possible
1376 evaluation results are:
1377 </t>
1378 <!--<t>RequestEvaluationResult</t>-->
1379 <dl>
1380 <dt>REQUEST_VALID</dt>
1381 <dd>Query is valid, no reply given.</dd>
1382 <dt>REQUEST_INVALID</dt>
1383 <dd>
1384 Query format does not match block type. For example, XQuery not
1385 given or of size of XQuery is not appropriate for type.
1386 </dd>
1387 </dl>
1388 <t>
1389 When evaluating result messages, the possible evaluation results
1390 are:
1391 </t>
1392 <!-->t>ReplyEvaluationResult</t>-->
1393 <!-- FIXME this is not integrated into the message processing
1394 or block processing. We need to specifiy where this is used.
1395 My guess is that we should handle result types in the message
1396 processing descriptions -->
1397 <dl>
1398 <dt>OK_MORE</dt>
1399 <dd>Valid result, and there may be more.</dd>
1400 <dt>OK_LAST</dt>
1401 <dd>Last possible valid result.</dd>
1402 <dt>OK_DUPLICATE</dt>
1403 <dd>Valid result, but duplicate.</dd>
1404 <dt>RESULT_INVALID</dt>
1405 <dd>Invalid result. Block does not match query. Value = 4.</dd>
1406 <dt>RESULT_IRRELEVANT</dt>
1407 <dd>Block does not match xquery. Valid result, but not relevant for the request.</dd>
1408 </dl>
1409 </section>
1410 <section anchor="block_functions">
1411 <name>Block Functions</name>
1412 <t>
1413 Any block type implementation <bcp14>MUST</bcp14> implement the following functions.
1414 </t>
1415 <dl>
1416 <dt>ValidateBlockQuery(Key, XQuery) -&gt; RequestEvaluationResult</dt>
1417 <dd>
1418 is used to evaluate the request for a block. It is used as part of
1419 <tt>GetMessage</tt> processing, where the block payload is still unkown,
1420 but the block <tt>XQuery</tt> <!--(FIXME: Undefined here)-->
1421 and <tt>Key</tt> can and
1422 <bcp14>MUST</bcp14> be verified, if possible.
1423 </dd>
1424 <dt>ValidateBlockStoreRequest(Block, Key) -&gt; RequestEvaluationResult</dt>
1425 <dd>
1426 is used to evaluate a block including its key and payload.
1427 It is used as part of <tt>PutMessage</tt> processing.
1428 The validation <bcp14>MUST</bcp14> include a check of the block payload against the
1429 <tt>Key</tt> under which it is requested to be stored.
1430 </dd>
1431 <dt>ValidateBlockReply(Block, XQuery, Key) -&gt; ReplyEvaluationResult</dt>
1432 <dd>
1433 is used to evaluate a block including its <tt>Key</tt> and payload.
1434 It is used as part <tt>ResultMessage</tt> processing.
1435 The validation of the respective <tt>Block</tt> requires a pending local query
1436 or a previously routed request of another peer and its associated
1437 XQuery data and Key.
1438 The validation <bcp14>MUST</bcp14> include a check of the block payload against the
1439 key under which it is requested to be stored.
1440 </dd>
1441 <dt>DeriveBlockKey(Block) -&gt; Key</dt>
1442 <dd>
1443 is used to synthesize the block key from the block payload
1444 and metadata. It is used as part of FIND-peer message processing.
1445 </dd>
1446 <dt>FilterResult(Block, XQuery, Key) -&gt; ReplyEvaluationResult</dt>
1447 <dd>
1448 is used to filter results stored in the local block storage for local
1449 queries. Locally stored blocks from previously observed
1450 <tt>ResultMessages</tt> and <tt>PutMessages</tt> <bcp14>MAY</bcp14> use this
1451 function instead of <tt>ValidateBlockReply</tt> in order to
1452 avoid revalidation of the block and only perform filtering based on
1453 request parameters.
1454 </dd>
1455 </dl>
1456 </section>
1522 <section anchor="hello_block"> 1457 <section anchor="hello_block">
1523 <name>HELLO</name> 1458 <name>Hello Block</name>
1459 <t>
1460 For bootstrapping and peer discovery, the DHT implementation uses
1461 its own block type called "HELLO". A block with this block type
1462 contains the peerID of the peer initiating the GET request.
1463 </t>
1524 <t> 1464 <t>
1525 The HELLO block type wire format is illustrated in 1465 The HELLO block type wire format is illustrated in
1526 <xref target="figure_hello"/>. A query for block of type HELLO <bcp14>MUST NOT</bcp14> 1466 <xref target="figure_hello"/>. A query for block of type HELLO <bcp14>MUST NOT</bcp14>
@@ -1652,32 +1592,95 @@ gnunet+tcp://12.3.4.5/ \
1652 </section> 1592 </section>
1653 </section> 1593 </section>
1654 <section> 1594 <section>
1655 <name>Caching Strategy</name> 1595 <name>Persistence</name>
1656 <t> 1596 <t>
1657 An implementation <bcp14>MUST</bcp14> implement an eviction strategy 1597 An implementation MUST provide a local persistence mechanism for
1658 for blocks stored in the block storage layer. 1598 blocks.
1599 The local storage MUST provide the following API:
1659 </t> 1600 </t>
1601 <dl>
1602 <dt>Store(Key, Block)</dt>
1603 <dd>
1604 Stores a block under the specified key.
1605 </dd>
1606 <dt>Lookup(Key) -&gt; List of Blocks</dt>
1607 <dd>
1608 Retrieves the blocks stored under the specified key.
1609 </dd>
1610 <dt>LookupApproximate(Key) -&gt; List of Blocks</dt>
1611 <dd>
1612 Retrieves the blocks stored under the specified key and
1613 any blocks under keys close to the specified key.
1614 </dd>
1615 </dl>
1660 <t> 1616 <t>
1661 In order to ensure the freshness of blocks, an implementation 1617 Over time a peer may accumulate a significant number of blocks
1662 <bcp14>MUST</bcp14> evict expired blocks in favor of 1618 which are stored locally in the persistence layer.
1663 new blocks. 1619 Due to the expected high number of blocks, the method to
1620 retrieve blocks close to the specified lookup key in the
1621 <tt>LookupApproximate</tt> API must be implemented with care
1622 with respect to efficiency.
1664 </t> 1623 </t>
1665 <t> 1624 <t>
1666 An implementation <bcp14>MAY</bcp14> preserve blocks which are often 1625 It is <bcp14>RECOMMENDED</bcp14> to limit the number of results
1667 requested. 1626 from the <tt>LookupApproximate</tt> procedure to a result size
1668 This approach can be expensive as it requires the implementation 1627 which is manageable by the local system.
1669 to keep track of how often a block is requested.
1670 </t> 1628 </t>
1671 <t> 1629 <t>
1672 An implementation <bcp14>MAY</bcp14> preserve blocks which are close 1630 In order to efficiently find a suitable result set, the implementation
1673 to the local peer ID. 1631 SHOULD follow the following procedure:
1674 </t> 1632 </t>
1633 <ol>
1634 <li>
1635 Sort all blocks by the block key in ascending (decending) order.
1636 The block keys are interpreted as integer.
1637 </li>
1638 <li>
1639 Alternatingly select a block with a key larger and smaller from
1640 the sortings.
1641 The resulting set is sorted by XOR distance.
1642 The selection process continues until the upper bound for the
1643 result set is reached and both sortings do not yield any closer
1644 blocks.
1645 </li>
1646 </ol>
1675 <t> 1647 <t>
1676 An implementation <bcp14>MAY</bcp14> provide configurable storage 1648 An implementation MAY decide to use a custom algorithm in order to
1677 quotas and 1649 find the closest blocks in the local storage.
1678 adapt its eviction strategy based on the current storage size 1650 But, especially for more primitive approaches, such as only
1679 or other constrained resources. 1651 comparing XOR distances for all blocks in the storage, the
1652 procedure may become ineffective for large storages.
1680 </t> 1653 </t>
1654 <!-- FIXME the result set is then filtered again by the block
1655 plugin. But we should discuss this elsewhere -->
1656 <section>
1657 <name>Caching Strategy</name>
1658 <t>
1659 An implementation <bcp14>MUST</bcp14> implement an eviction strategy
1660 for blocks stored in the block storage layer.
1661 </t>
1662 <t>
1663 In order to ensure the freshness of blocks, an implementation
1664 <bcp14>MUST</bcp14> evict expired blocks in favor of
1665 new blocks.
1666 </t>
1667 <t>
1668 An implementation <bcp14>MAY</bcp14> preserve blocks which are often
1669 requested.
1670 This approach can be expensive as it requires the implementation
1671 to keep track of how often a block is requested.
1672 </t>
1673 <t>
1674 An implementation <bcp14>MAY</bcp14> preserve blocks which are close
1675 to the local peer ID.
1676 </t>
1677 <t>
1678 An implementation <bcp14>MAY</bcp14> provide configurable storage
1679 quotas and
1680 adapt its eviction strategy based on the current storage size
1681 or other constrained resources.
1682 </t>
1683 </section>
1681 </section> 1684 </section>
1682 </section> 1685 </section>
1683 <section anchor="security" numbered="true" toc="default"> 1686 <section anchor="security" numbered="true" toc="default">