fs.rst (5958B)
1 .. index:: 2 double: File sharing; subsystem 3 see: FS; File sharing 4 5 .. _FS-Subsystem-Dev: 6 7 FS 8 == 9 10 .. _Namespace-Advertisements: 11 12 Namespace Advertisements 13 ^^^^^^^^^^^^^^^^^^^^^^^^ 14 15 .. todo:: FIXME: all zeroses -> ? 16 17 An ``SBlock`` with identifier all zeros is a signed advertisement for a 18 namespace. This special ``SBlock`` contains metadata describing the 19 content of the namespace. Instead of the name of the identifier for a 20 potential update, it contains the identifier for the root of the 21 namespace. The URI should always be empty. The ``SBlock`` is signed with 22 the content provider's RSA private key (just like any other SBlock). 23 Peers can search for ``SBlock``\ s in order to find out more about a 24 namespace. 25 26 .. _KSBlocks: 27 28 KSBlocks 29 ^^^^^^^^ 30 31 GNUnet implements ``KSBlocks`` which are ``KBlocks`` that, instead of 32 encrypting a CHK and metadata, encrypt an ``SBlock`` instead. In other 33 words, ``KSBlocks`` enable GNUnet to find ``SBlocks`` using the global 34 keyword search. Usually the encrypted ``SBlock`` is a namespace 35 advertisement. The rationale behind ``KSBlock``\ s and ``SBlock``\ s is 36 to enable peers to discover namespaces via keyword searches, and, to 37 associate useful information with namespaces. When GNUnet finds 38 ``KSBlocks`` during a normal keyword search, it adds the information to 39 an internal list of discovered namespaces. Users looking for interesting 40 namespaces can then inspect this list, reducing the need for out-of-band 41 discovery of namespaces. Naturally, namespaces (or more specifically, 42 namespace advertisements) can also be referenced from directories, but 43 ``KSBlock``\ s should make it easier to advertise namespaces for the 44 owner of the pseudonym since they eliminate the need to first create a 45 directory. 46 47 Collections are also advertised using ``KSBlock``\ s. 48 49 .. https://old.gnunet.org/sites/default/files/ecrs.pdf 50 .. What is this? - WGL 51 52 .. _File_002dsharing-persistence-directory-structure: 53 54 File-sharing persistence directory structure 55 -------------------------------------------- 56 57 This section documents how the file-sharing library implements 58 persistence of file-sharing operations and specifically the resulting 59 directory structure. This code is only active if the 60 ``GNUNET_FS_FLAGS_PERSISTENCE`` flag was set when calling 61 ``GNUNET_FS_start``. In this case, the file-sharing library will try 62 hard to ensure that all major operations (searching, downloading, 63 publishing, unindexing) are persistent, that is, can live longer than 64 the process itself. More specifically, an operation is supposed to live 65 until it is explicitly stopped. 66 67 If ``GNUNET_FS_stop`` is called before an operation has been stopped, a 68 ``SUSPEND`` event is generated and then when the process calls 69 ``GNUNET_FS_start`` next time, a ``RESUME`` event is generated. 70 Additionally, even if an application crashes (segfault, SIGKILL, system 71 crash) and hence ``GNUNET_FS_stop`` is never called and no ``SUSPEND`` 72 events are generated, operations are still resumed (with ``RESUME`` 73 events). This is implemented by constantly writing the current state of 74 the file-sharing operations to disk. Specifically, the current state is 75 always written to disk whenever anything significant changes (the 76 exception are block-wise progress in publishing and unindexing, since 77 those operations would be slowed down significantly and can be resumed 78 cheaply even without detailed accounting). Note that if the process 79 crashes (or is killed) during a serialization operation, FS does not 80 guarantee that this specific operation is recoverable (no strict 81 transactional semantics, again for performance reasons). However, all 82 other unrelated operations should resume nicely. 83 84 Since we need to serialize the state continuously and want to recover as 85 much as possible even after crashing during a serialization operation, 86 we do not use one large file for serialization. Instead, several 87 directories are used for the various operations. When 88 ``GNUNET_FS_start`` executes, the master directories are scanned for 89 files describing operations to resume. Sometimes, these operations can 90 refer to related operations in child directories which may also be 91 resumed at this point. Note that corrupted files are cleaned up 92 automatically. However, dangling files in child directories (those that 93 are not referenced by files from the master directories) are not 94 automatically removed. 95 96 Persistence data is kept in a directory that begins with the 97 \"STATE_DIR\" prefix from the configuration file (by default, 98 \"$SERVICEHOME/persistence/\") followed by the name of the client as 99 given to ``GNUNET_FS_start`` (for example, \"gnunet-gtk\") followed by 100 the actual name of the master or child directory. 101 102 The names for the master directories follow the names of the operations: 103 104 - \"search\" 105 106 - \"download\" 107 108 - \"publish\" 109 110 - \"unindex\" 111 112 Each of the master directories contains names (chosen at random) for 113 each active top-level (master) operation. Note that a download that is 114 associated with a search result is not a top-level operation. 115 116 In contrast to the master directories, the child directories are only 117 consulted when another operation refers to them. For each search, a 118 subdirectory (named after the master search synchronization file) 119 contains the search results. Search results can have an associated 120 download, which is then stored in the general \"download-child\" 121 directory. Downloads can be recursive, in which case children are stored 122 in subdirectories mirroring the structure of the recursive download 123 (either starting in the master \"download\" directory or in the 124 \"download-child\" directory depending on how the download was 125 initiated). For publishing operations, the \"publish-file\" directory 126 contains information about the individual files and directories that are 127 part of the publication. However, this directory structure is flat and 128 does not mirror the structure of the publishing operation. Note that 129 unindex operations cannot have associated child operations.