diff options
Diffstat (limited to 'draft-schanzen-r5n.xml')
-rw-r--r-- | draft-schanzen-r5n.xml | 345 |
1 files changed, 174 insertions, 171 deletions
diff --git a/draft-schanzen-r5n.xml b/draft-schanzen-r5n.xml index 28610a6..4fbfd79 100644 --- a/draft-schanzen-r5n.xml +++ b/draft-schanzen-r5n.xml | |||
@@ -331,7 +331,7 @@ Connectivity | |Underlay| |Underlay| | |||
331 | required depending on the respective <tt>Block-Type</tt>. | 331 | required depending on the respective <tt>Block-Type</tt>. |
332 | A <tt>Block-Type</tt> must define if the <tt>XQuery</tt> can or must | 332 | A <tt>Block-Type</tt> must define if the <tt>XQuery</tt> can or must |
333 | be used and what the specific format of its contents should be. | 333 | be used and what the specific format of its contents should be. |
334 | See also <xref target="block_types"/>. | 334 | See also <xref target="blockstorage"/>. |
335 | </dd> | 335 | </dd> |
336 | <dt>Result-Filter:</dt> | 336 | <dt>Result-Filter:</dt> |
337 | <dd> | 337 | <dd> |
@@ -1359,168 +1359,108 @@ Connectivity | |Underlay| |Underlay| | |||
1359 | </section> | 1359 | </section> |
1360 | <section anchor="blockstorage" numbered="true" toc="default"> | 1360 | <section anchor="blockstorage" numbered="true" toc="default"> |
1361 | <name>Block Storage</name> | 1361 | <name>Block Storage</name> |
1362 | <t> | ||
1363 | An implementation MUST provide a local persistence mechanism for | ||
1364 | blocks. | ||
1365 | The local storage MUST provide the following API: | ||
1366 | </t> | ||
1367 | <dl> | ||
1368 | <dt>Store(Key, Block)</dt> | ||
1369 | <dd> | ||
1370 | Stores a block under the specified key. | ||
1371 | </dd> | ||
1372 | <dt>Lookup(Key) -> List of Blocks</dt> | ||
1373 | <dd> | ||
1374 | Retrieves the blocks stored under the specified key. | ||
1375 | </dd> | ||
1376 | <dt>LookupApproximate(Key) -> List of Blocks</dt> | ||
1377 | <dd> | ||
1378 | Retrieves the blocks stored under the specified key and | ||
1379 | any blocks under keys close to the specified key. | ||
1380 | </dd> | ||
1381 | </dl> | ||
1382 | <t> | ||
1383 | Over time a peer may accumulate a significant number of blocks | ||
1384 | which are stored locally in the persistence layer. | ||
1385 | Due to the expected high number of blocks, the method to | ||
1386 | retrieve blocks close to the specified lookup key in the | ||
1387 | <tt>LookupApproximate</tt> API must be implemented with care | ||
1388 | with respect to efficiency. | ||
1389 | </t> | ||
1390 | <t> | ||
1391 | It is <bcp14>RECOMMENDED</bcp14> to limit the number of results | ||
1392 | from the <tt>LookupApproximate</tt> procedure to a result size | ||
1393 | which is manageable by the local system. | ||
1394 | </t> | ||
1395 | <t> | ||
1396 | In order to efficiently find a suitable result set, the implementation | ||
1397 | SHOULD follow the following procedure: | ||
1398 | </t> | ||
1399 | <ol> | ||
1400 | <li> | ||
1401 | Sort all blocks by the block key in ascending (decending) order. | ||
1402 | The block keys are interpreted as integer. | ||
1403 | </li> | ||
1404 | <li> | ||
1405 | Alternatingly select a block with a key larger and smaller from | ||
1406 | the sortings. | ||
1407 | The resulting set is sorted by XOR distance. | ||
1408 | The selection process continues until the upper bound for the | ||
1409 | result set is reached and both sortings do not yield any closer | ||
1410 | blocks. | ||
1411 | </li> | ||
1412 | </ol> | ||
1413 | <t> | ||
1414 | An implementation MAY decide to use a custom algorithm in order to | ||
1415 | find the closest blocks in the local storage. | ||
1416 | But, especially for more primitive approaches, such as only | ||
1417 | comparing XOR distances for all blocks in the storage, the | ||
1418 | procedure may become ineffective for large storages. | ||
1419 | </t> | ||
1420 | <!-- FIXME the result set is then filtered again by the block | ||
1421 | plugin. But we should discuss this elsewhere --> | ||
1422 | <section> | 1362 | <section> |
1423 | <name>Block Processing</name> | 1363 | <name>Blocks</name> |
1424 | <t> | ||
1425 | Block validation may be necessary for both request as well as | ||
1426 | reply messages. | ||
1427 | When evaluating request messages and their metadata, the possible | ||
1428 | evaluation results are: | ||
1429 | </t> | ||
1430 | <!--<t>RequestEvaluationResult</t>--> | ||
1431 | <dl> | ||
1432 | <dt>REQUEST_VALID</dt> | ||
1433 | <dd>Query is valid, no reply given.</dd> | ||
1434 | <dt>REQUEST_INVALID</dt> | ||
1435 | <dd> | ||
1436 | Query format does not match block type. For example, XQuery not | ||
1437 | given or of size of XQuery is not appropriate for type. | ||
1438 | </dd> | ||
1439 | </dl> | ||
1440 | <t> | ||
1441 | When evaluating result messages, the possible evaluation results | ||
1442 | are: | ||
1443 | </t> | ||
1444 | <!-->t>ReplyEvaluationResult</t>--> | ||
1445 | <!-- FIXME this is not integrated into the message processing | ||
1446 | or block processing. We need to specifiy where this is used. | ||
1447 | My guess is that we should handle result types in the message | ||
1448 | processing descriptions --> | ||
1449 | <dl> | ||
1450 | <dt>OK_MORE</dt> | ||
1451 | <dd>Valid result, and there may be more.</dd> | ||
1452 | <dt>OK_LAST</dt> | ||
1453 | <dd>Last possible valid result.</dd> | ||
1454 | <dt>OK_DUPLICATE</dt> | ||
1455 | <dd>Valid result, but duplicate.</dd> | ||
1456 | <dt>RESULT_INVALID</dt> | ||
1457 | <dd>Invalid result. Block does not match query. Value = 4.</dd> | ||
1458 | <dt>RESULT_IRRELEVANT</dt> | ||
1459 | <dd>Block does not match xquery. Valid result, but not relevant for the request.</dd> | ||
1460 | </dl> | ||
1461 | </section> | ||
1462 | <section anchor="block_functions"> | ||
1463 | <name>Block Functions</name> | ||
1464 | <t> | ||
1465 | Any block type implementation <bcp14>MUST</bcp14> implement the following functions. | ||
1466 | </t> | ||
1467 | <dl> | ||
1468 | <dt>ValidateBlockQuery(Key, XQuery) -> RequestEvaluationResult</dt> | ||
1469 | <dd> | ||
1470 | is used to evaluate the request for a block. It is used as part of | ||
1471 | <tt>GetMessage</tt> processing, where the block payload is still unkown, | ||
1472 | but the block <tt>XQuery</tt> <!--(FIXME: Undefined here)--> | ||
1473 | and <tt>Key</tt> can and | ||
1474 | <bcp14>MUST</bcp14> be verified, if possible. | ||
1475 | </dd> | ||
1476 | <dt>ValidateBlockStoreRequest(Block, Key) -> RequestEvaluationResult</dt> | ||
1477 | <dd> | ||
1478 | is used to evaluate a block including its key and payload. | ||
1479 | It is used as part of <tt>PutMessage</tt> processing. | ||
1480 | The validation <bcp14>MUST</bcp14> include a check of the block payload against the | ||
1481 | <tt>Key</tt> under which it is requested to be stored. | ||
1482 | </dd> | ||
1483 | <dt>ValidateBlockReply(Block, XQuery, Key) -> ReplyEvaluationResult</dt> | ||
1484 | <dd> | ||
1485 | is used to evaluate a block including its <tt>Key</tt> and payload. | ||
1486 | It is used as part <tt>ResultMessage</tt> processing. | ||
1487 | The validation of the respective <tt>Block</tt> requires a pending local query | ||
1488 | or a previously routed request of another peer and its associated | ||
1489 | XQuery data and Key. | ||
1490 | The validation <bcp14>MUST</bcp14> include a check of the block payload against the | ||
1491 | key under which it is requested to be stored. | ||
1492 | </dd> | ||
1493 | <dt>DeriveBlockKey(Block) -> Key</dt> | ||
1494 | <dd> | ||
1495 | is used to synthesize the block key from the block payload | ||
1496 | and metadata. It is used as part of FIND-peer message processing. | ||
1497 | </dd> | ||
1498 | <dt>FilterResult(Block, XQuery, Key) -> ReplyEvaluationResult</dt> | ||
1499 | <dd> | ||
1500 | is used to filter results stored in the local block storage for local | ||
1501 | queries. Locally stored blocks from previously observed | ||
1502 | <tt>ResultMessages</tt> and <tt>PutMessages</tt> <bcp14>MAY</bcp14> use this | ||
1503 | function instead of <tt>ValidateBlockReply</tt> in order to | ||
1504 | avoid revalidation of the block and only perform filtering based on | ||
1505 | request parameters. | ||
1506 | </dd> | ||
1507 | </dl> | ||
1508 | </section> | ||
1509 | <section anchor="block_types"> | ||
1510 | <name>Block Types</name> | ||
1511 | <t> | 1364 | <t> |
1512 | Applications can and should define their own block types. | 1365 | Applications can and should define their own block types. |
1513 | The block type determines the format and handling of the block | 1366 | The block type determines the format and handling of the block |
1514 | payload by peers in PUT and RESULT messages. | 1367 | payload by peers in PUT and RESULT messages. |
1515 | Block types <bcp14>MUST</bcp14> be registered with GANA <xref target="gana"/>. | 1368 | Block types <bcp14>MUST</bcp14> be registered with GANA <xref target="gana"/>. |
1516 | </t> | 1369 | </t> |
1517 | <t> | 1370 | <section> |
1518 | For bootstrapping and peer discovery, the DHT implementation uses | 1371 | <name>Block Processing</name> |
1519 | its own block type called "HELLO". A block with this block type | 1372 | <t> |
1520 | contains the peerID of the peer initiating the GET request. | 1373 | Block validation may be necessary for both request as well as |
1521 | </t> | 1374 | reply messages. |
1375 | When evaluating request messages and their metadata, the possible | ||
1376 | evaluation results are: | ||
1377 | </t> | ||
1378 | <!--<t>RequestEvaluationResult</t>--> | ||
1379 | <dl> | ||
1380 | <dt>REQUEST_VALID</dt> | ||
1381 | <dd>Query is valid, no reply given.</dd> | ||
1382 | <dt>REQUEST_INVALID</dt> | ||
1383 | <dd> | ||
1384 | Query format does not match block type. For example, XQuery not | ||
1385 | given or of size of XQuery is not appropriate for type. | ||
1386 | </dd> | ||
1387 | </dl> | ||
1388 | <t> | ||
1389 | When evaluating result messages, the possible evaluation results | ||
1390 | are: | ||
1391 | </t> | ||
1392 | <!-->t>ReplyEvaluationResult</t>--> | ||
1393 | <!-- FIXME this is not integrated into the message processing | ||
1394 | or block processing. We need to specifiy where this is used. | ||
1395 | My guess is that we should handle result types in the message | ||
1396 | processing descriptions --> | ||
1397 | <dl> | ||
1398 | <dt>OK_MORE</dt> | ||
1399 | <dd>Valid result, and there may be more.</dd> | ||
1400 | <dt>OK_LAST</dt> | ||
1401 | <dd>Last possible valid result.</dd> | ||
1402 | <dt>OK_DUPLICATE</dt> | ||
1403 | <dd>Valid result, but duplicate.</dd> | ||
1404 | <dt>RESULT_INVALID</dt> | ||
1405 | <dd>Invalid result. Block does not match query. Value = 4.</dd> | ||
1406 | <dt>RESULT_IRRELEVANT</dt> | ||
1407 | <dd>Block does not match xquery. Valid result, but not relevant for the request.</dd> | ||
1408 | </dl> | ||
1409 | </section> | ||
1410 | <section anchor="block_functions"> | ||
1411 | <name>Block Functions</name> | ||
1412 | <t> | ||
1413 | Any block type implementation <bcp14>MUST</bcp14> implement the following functions. | ||
1414 | </t> | ||
1415 | <dl> | ||
1416 | <dt>ValidateBlockQuery(Key, XQuery) -> RequestEvaluationResult</dt> | ||
1417 | <dd> | ||
1418 | is used to evaluate the request for a block. It is used as part of | ||
1419 | <tt>GetMessage</tt> processing, where the block payload is still unkown, | ||
1420 | but the block <tt>XQuery</tt> <!--(FIXME: Undefined here)--> | ||
1421 | and <tt>Key</tt> can and | ||
1422 | <bcp14>MUST</bcp14> be verified, if possible. | ||
1423 | </dd> | ||
1424 | <dt>ValidateBlockStoreRequest(Block, Key) -> RequestEvaluationResult</dt> | ||
1425 | <dd> | ||
1426 | is used to evaluate a block including its key and payload. | ||
1427 | It is used as part of <tt>PutMessage</tt> processing. | ||
1428 | The validation <bcp14>MUST</bcp14> include a check of the block payload against the | ||
1429 | <tt>Key</tt> under which it is requested to be stored. | ||
1430 | </dd> | ||
1431 | <dt>ValidateBlockReply(Block, XQuery, Key) -> ReplyEvaluationResult</dt> | ||
1432 | <dd> | ||
1433 | is used to evaluate a block including its <tt>Key</tt> and payload. | ||
1434 | It is used as part <tt>ResultMessage</tt> processing. | ||
1435 | The validation of the respective <tt>Block</tt> requires a pending local query | ||
1436 | or a previously routed request of another peer and its associated | ||
1437 | XQuery data and Key. | ||
1438 | The validation <bcp14>MUST</bcp14> include a check of the block payload against the | ||
1439 | key under which it is requested to be stored. | ||
1440 | </dd> | ||
1441 | <dt>DeriveBlockKey(Block) -> Key</dt> | ||
1442 | <dd> | ||
1443 | is used to synthesize the block key from the block payload | ||
1444 | and metadata. It is used as part of FIND-peer message processing. | ||
1445 | </dd> | ||
1446 | <dt>FilterResult(Block, XQuery, Key) -> ReplyEvaluationResult</dt> | ||
1447 | <dd> | ||
1448 | is used to filter results stored in the local block storage for local | ||
1449 | queries. Locally stored blocks from previously observed | ||
1450 | <tt>ResultMessages</tt> and <tt>PutMessages</tt> <bcp14>MAY</bcp14> use this | ||
1451 | function instead of <tt>ValidateBlockReply</tt> in order to | ||
1452 | avoid revalidation of the block and only perform filtering based on | ||
1453 | request parameters. | ||
1454 | </dd> | ||
1455 | </dl> | ||
1456 | </section> | ||
1522 | <section anchor="hello_block"> | 1457 | <section anchor="hello_block"> |
1523 | <name>HELLO</name> | 1458 | <name>Hello Block</name> |
1459 | <t> | ||
1460 | For bootstrapping and peer discovery, the DHT implementation uses | ||
1461 | its own block type called "HELLO". A block with this block type | ||
1462 | contains the peerID of the peer initiating the GET request. | ||
1463 | </t> | ||
1524 | <t> | 1464 | <t> |
1525 | The HELLO block type wire format is illustrated in | 1465 | The HELLO block type wire format is illustrated in |
1526 | <xref target="figure_hello"/>. A query for block of type HELLO <bcp14>MUST NOT</bcp14> | 1466 | <xref target="figure_hello"/>. A query for block of type HELLO <bcp14>MUST NOT</bcp14> |
@@ -1652,32 +1592,95 @@ gnunet+tcp://12.3.4.5/ \ | |||
1652 | </section> | 1592 | </section> |
1653 | </section> | 1593 | </section> |
1654 | <section> | 1594 | <section> |
1655 | <name>Caching Strategy</name> | 1595 | <name>Persistence</name> |
1656 | <t> | 1596 | <t> |
1657 | An implementation <bcp14>MUST</bcp14> implement an eviction strategy | 1597 | An implementation MUST provide a local persistence mechanism for |
1658 | for blocks stored in the block storage layer. | 1598 | blocks. |
1599 | The local storage MUST provide the following API: | ||
1659 | </t> | 1600 | </t> |
1601 | <dl> | ||
1602 | <dt>Store(Key, Block)</dt> | ||
1603 | <dd> | ||
1604 | Stores a block under the specified key. | ||
1605 | </dd> | ||
1606 | <dt>Lookup(Key) -> List of Blocks</dt> | ||
1607 | <dd> | ||
1608 | Retrieves the blocks stored under the specified key. | ||
1609 | </dd> | ||
1610 | <dt>LookupApproximate(Key) -> List of Blocks</dt> | ||
1611 | <dd> | ||
1612 | Retrieves the blocks stored under the specified key and | ||
1613 | any blocks under keys close to the specified key. | ||
1614 | </dd> | ||
1615 | </dl> | ||
1660 | <t> | 1616 | <t> |
1661 | In order to ensure the freshness of blocks, an implementation | 1617 | Over time a peer may accumulate a significant number of blocks |
1662 | <bcp14>MUST</bcp14> evict expired blocks in favor of | 1618 | which are stored locally in the persistence layer. |
1663 | new blocks. | 1619 | Due to the expected high number of blocks, the method to |
1620 | retrieve blocks close to the specified lookup key in the | ||
1621 | <tt>LookupApproximate</tt> API must be implemented with care | ||
1622 | with respect to efficiency. | ||
1664 | </t> | 1623 | </t> |
1665 | <t> | 1624 | <t> |
1666 | An implementation <bcp14>MAY</bcp14> preserve blocks which are often | 1625 | It is <bcp14>RECOMMENDED</bcp14> to limit the number of results |
1667 | requested. | 1626 | from the <tt>LookupApproximate</tt> procedure to a result size |
1668 | This approach can be expensive as it requires the implementation | 1627 | which is manageable by the local system. |
1669 | to keep track of how often a block is requested. | ||
1670 | </t> | 1628 | </t> |
1671 | <t> | 1629 | <t> |
1672 | An implementation <bcp14>MAY</bcp14> preserve blocks which are close | 1630 | In order to efficiently find a suitable result set, the implementation |
1673 | to the local peer ID. | 1631 | SHOULD follow the following procedure: |
1674 | </t> | 1632 | </t> |
1633 | <ol> | ||
1634 | <li> | ||
1635 | Sort all blocks by the block key in ascending (decending) order. | ||
1636 | The block keys are interpreted as integer. | ||
1637 | </li> | ||
1638 | <li> | ||
1639 | Alternatingly select a block with a key larger and smaller from | ||
1640 | the sortings. | ||
1641 | The resulting set is sorted by XOR distance. | ||
1642 | The selection process continues until the upper bound for the | ||
1643 | result set is reached and both sortings do not yield any closer | ||
1644 | blocks. | ||
1645 | </li> | ||
1646 | </ol> | ||
1675 | <t> | 1647 | <t> |
1676 | An implementation <bcp14>MAY</bcp14> provide configurable storage | 1648 | An implementation MAY decide to use a custom algorithm in order to |
1677 | quotas and | 1649 | find the closest blocks in the local storage. |
1678 | adapt its eviction strategy based on the current storage size | 1650 | But, especially for more primitive approaches, such as only |
1679 | or other constrained resources. | 1651 | comparing XOR distances for all blocks in the storage, the |
1652 | procedure may become ineffective for large storages. | ||
1680 | </t> | 1653 | </t> |
1654 | <!-- FIXME the result set is then filtered again by the block | ||
1655 | plugin. But we should discuss this elsewhere --> | ||
1656 | <section> | ||
1657 | <name>Caching Strategy</name> | ||
1658 | <t> | ||
1659 | An implementation <bcp14>MUST</bcp14> implement an eviction strategy | ||
1660 | for blocks stored in the block storage layer. | ||
1661 | </t> | ||
1662 | <t> | ||
1663 | In order to ensure the freshness of blocks, an implementation | ||
1664 | <bcp14>MUST</bcp14> evict expired blocks in favor of | ||
1665 | new blocks. | ||
1666 | </t> | ||
1667 | <t> | ||
1668 | An implementation <bcp14>MAY</bcp14> preserve blocks which are often | ||
1669 | requested. | ||
1670 | This approach can be expensive as it requires the implementation | ||
1671 | to keep track of how often a block is requested. | ||
1672 | </t> | ||
1673 | <t> | ||
1674 | An implementation <bcp14>MAY</bcp14> preserve blocks which are close | ||
1675 | to the local peer ID. | ||
1676 | </t> | ||
1677 | <t> | ||
1678 | An implementation <bcp14>MAY</bcp14> provide configurable storage | ||
1679 | quotas and | ||
1680 | adapt its eviction strategy based on the current storage size | ||
1681 | or other constrained resources. | ||
1682 | </t> | ||
1683 | </section> | ||
1681 | </section> | 1684 | </section> |
1682 | </section> | 1685 | </section> |
1683 | <section anchor="security" numbered="true" toc="default"> | 1686 | <section anchor="security" numbered="true" toc="default"> |