aboutsummaryrefslogtreecommitdiff
path: root/doc/chapters
diff options
context:
space:
mode:
authorng0 <ng0@infotropique.org>2017-10-19 15:19:32 +0000
committerng0 <ng0@infotropique.org>2017-10-19 15:19:32 +0000
commit07e671228bea1e23ea07dedc3ce2eec049279871 (patch)
tree384380dcc3a33b706effaf60495955fbe8470297 /doc/chapters
parentfa780739b3ec7d22a98407797f20f2015f9190de (diff)
downloadgnunet-07e671228bea1e23ea07dedc3ce2eec049279871.tar.gz
gnunet-07e671228bea1e23ea07dedc3ce2eec049279871.zip
chapters/developer.texi: line length 74 for the first 19% of the file.
Diffstat (limited to 'doc/chapters')
-rw-r--r--doc/chapters/developer.texi1463
1 files changed, 770 insertions, 693 deletions
diff --git a/doc/chapters/developer.texi b/doc/chapters/developer.texi
index da8aa8a83..9a58e38cc 100644
--- a/doc/chapters/developer.texi
+++ b/doc/chapters/developer.texi
@@ -1,4 +1,4 @@
1@c *************************************************************************** 1@c ***********************************************************************
2@node GNUnet Developer Handbook 2@node GNUnet Developer Handbook
3@chapter GNUnet Developer Handbook 3@chapter GNUnet Developer Handbook
4 4
@@ -12,39 +12,40 @@ that believes in the GNU philosophy
12@item 12@item
13A set of standards, including coding conventions and architectural rules 13A set of standards, including coding conventions and architectural rules
14@item 14@item
15A set of layered protocols, both specifying the communication between peers as 15A set of layered protocols, both specifying the communication between
16well as the communication between components of a single peer. 16peers as well as the communication between components of a single peer.
17@item 17@item
18A set of libraries with well-defined APIs suitable for writing extensions 18A set of libraries with well-defined APIs suitable for writing extensions
19@end itemize 19@end itemize
20 20
21In particular, the architecture specifies that a peer consists of many 21In particular, the architecture specifies that a peer consists of many
22processes communicating via protocols. Processes can be written in almost 22processes communicating via protocols. Processes can be written in almost
23any language. C and Java APIs exist for accessing existing services and for 23any language. C and Java APIs exist for accessing existing services and
24writing extensions. It is possible to write extensions in other languages by 24for writing extensions. It is possible to write extensions in other
25implementing the necessary IPC protocols. 25languages by implementing the necessary IPC protocols.
26 26
27GNUnet can be extended and improved along many possible dimensions, and anyone 27GNUnet can be extended and improved along many possible dimensions, and
28interested in free software and freedom-enhancing networking is welcome to 28anyone interested in free software and freedom-enhancing networking is
29join the effort. This developer handbook attempts to provide an initial 29welcome to join the effort. This developer handbook attempts to provide
30introduction to some of the key design choices and central components of the 30an initial introduction to some of the key design choices and central
31system. This manual is far from complete, and we welcome informed 31components of the system. This manual is far from complete, and we
32contributions, be it in the form of new chapters or insightful comments. 32welcome informed contributions, be it in the form of new chapters or
33insightful comments.
33 34
34However, the website is experiencing a constant onslaught of sophisticated 35However, the website is experiencing a constant onslaught of sophisticated
35link-spam entered manually by exploited workers solving puzzles and 36link-spam entered manually by exploited workers solving puzzles and
36customizing text. To limit this commercial defacement, we are strictly 37customizing text. To limit this commercial defacement, we are strictly
37moderating comments and have disallowed "normal" users from posting new 38moderating comments and have disallowed "normal" users from posting new
38content. However, this is really only intended to keep the spam at bay. If 39content. However, this is really only intended to keep the spam at bay. If
39you are a real user or aspiring developer, please drop us a note (IRC, e-mail, 40you are a real user or aspiring developer, please drop us a note
40contact form) with your user profile ID number included. We will then relax 41(IRC, e-mail, contact form) with your user profile ID number included.
41these restrictions on your account. We're sorry for this inconvenience; 42We will then relax these restrictions on your account. We're sorry for
42however, few people would want to read this site if 99% of it was 43this inconvenience; however, few people would want to read this site
43advertisements for bogus websites. 44if 99% of it was advertisements for bogus websites.
44 45
45 46
46 47
47@c *************************************************************************** 48@c ***********************************************************************
48 49
49 50
50 51
@@ -95,15 +96,16 @@ advertisements for bogus websites.
95@node Developer Introduction 96@node Developer Introduction
96@section Developer Introduction 97@section Developer Introduction
97 98
98This developer handbook is intended as first introduction to GNUnet for new 99This developer handbook is intended as first introduction to GNUnet for
99developers that want to extend the GNUnet framework. After the introduction, 100new developers that want to extend the GNUnet framework. After the
100each of the GNUnet subsystems (directories in the @file{src/} tree) is (supposed to 101introduction, each of the GNUnet subsystems (directories in the
101be) covered in its own chapter. In addition to this documentation, GNUnet 102@file{src/} tree) is (supposed to be) covered in its own chapter. In
102developers should be aware of the services available on the GNUnet server to 103addition to this documentation, GNUnet developers should be aware of the
103them. 104services available on the GNUnet server to them.
104 105
105New developers can have a look a the GNUnet tutorials for C and java available 106New developers can have a look a the GNUnet tutorials for C and java
106in the @file{src/} directory of the repository or under the following links: 107available in the @file{src/} directory of the repository or under the
108following links:
107 109
108@c ** FIXME: Link to files in source, not online. 110@c ** FIXME: Link to files in source, not online.
109@c ** FIXME: Where is the Java tutorial? 111@c ** FIXME: Where is the Java tutorial?
@@ -114,45 +116,47 @@ in the @file{src/} directory of the repository or under the following links:
114 116
115In addition to this book, the GNUnet server contains various resources for 117In addition to this book, the GNUnet server contains various resources for
116GNUnet developers. They are all conveniently reachable via the "Developer" 118GNUnet developers. They are all conveniently reachable via the "Developer"
117entry in the navigation menu. Some additional tools (such as static analysis 119entry in the navigation menu. Some additional tools (such as static
118reports) require a special developer access to perform certain operations. If 120analysis reports) require a special developer access to perform certain
119you feel you need access, you should contact 121operations. If you feel you need access, you should contact
120@uref{http://grothoff.org/christian/, Christian Grothoff}, GNUnet's maintainer. 122@uref{http://grothoff.org/christian/, Christian Grothoff},
123GNUnet's maintainer.
121 124
122The public subsystems on the GNUnet server that help developers are: 125The public subsystems on the GNUnet server that help developers are:
123 126
124@itemize @bullet 127@itemize @bullet
125@item The Version control system keeps our code and enables distributed 128@item The Version control system keeps our code and enables distributed
126development. Only developers with write access can commit code, everyone else 129development. Only developers with write access can commit code, everyone
127is encouraged to submit patches to the 130else is encouraged to submit patches to the
128@uref{https://lists.gnu.org/mailman/listinfo/gnunet-developers, GNUnet-developers mailinglist}. 131@uref{https://lists.gnu.org/mailman/listinfo/gnunet-developers, GNUnet-developers mailinglist}.
129@item The GNUnet bugtracking system is used to track feature requests, open bug 132@item The GNUnet bugtracking system is used to track feature requests,
130reports and their resolutions. Anyone can report bugs, only developers can 133open bug reports and their resolutions. Anyone can report bugs, only
131claim to have fixed them. 134developers can claim to have fixed them.
132@item A buildbot is used to check GNUnet builds automatically on a range of 135@item A buildbot is used to check GNUnet builds automatically on a range
133platforms. Builds are triggered automatically after 30 minutes of no changes to 136of platforms. Builds are triggered automatically after 30 minutes of no
134Git. 137changes to Git.
135@item The current quality of our automated test suite is assessed using Code 138@item The current quality of our automated test suite is assessed using
136coverage analysis. This analysis is run daily; however the webpage is only 139Code coverage analysis. This analysis is run daily; however the webpage
137updated if all automated tests pass at that time. Testcases that improve our 140is only updated if all automated tests pass at that time. Testcases that
138code coverage are always welcome. 141improve our code coverage are always welcome.
139@item We try to automatically find bugs using a static analysis scan. This scan 142@item We try to automatically find bugs using a static analysis scan.
140is run daily; however the webpage is only updated if all automated tests pass 143This scan is run daily; however the webpage is only updated if all
141at the time. Note that not everything that is flagged by the analysis is a bug, 144automated tests pass at the time. Note that not everything that is
142sometimes even good code can be marked as possibly problematic. Nevertheless, 145flagged by the analysis is a bug, sometimes even good code can be marked
143developers are encouraged to at least be aware of all issues in their code that 146as possibly problematic. Nevertheless, developers are encouraged to at
144are listed. 147least be aware of all issues in their code that are listed.
145@item We use Gauger for automatic performance regression visualization. Details 148@item We use Gauger for automatic performance regression visualization.
146on how to use Gauger are here. 149Details on how to use Gauger are here.
147@item We use @uref{http://junit.org/, junit} to automatically test gnunet-java. 150@item We use @uref{http://junit.org/, junit} to automatically test
148Automatically generated, current reports on the test suite are here. 151gnunet-java. Automatically generated, current reports on the test suite
152are here.
149@item We use Cobertura to generate test coverage reports for gnunet-java. 153@item We use Cobertura to generate test coverage reports for gnunet-java.
150Current reports on test coverage are here. 154Current reports on test coverage are here.
151@end itemize 155@end itemize
152 156
153 157
154 158
155@c *************************************************************************** 159@c ***********************************************************************
156@menu 160@menu
157* Project overview:: 161* Project overview::
158@end menu 162@end menu
@@ -160,10 +164,11 @@ Current reports on test coverage are here.
160@node Project overview 164@node Project overview
161@subsection Project overview 165@subsection Project overview
162 166
163The GNUnet project consists at this point of several sub-projects. This section 167The GNUnet project consists at this point of several sub-projects. This
164is supposed to give an initial overview about the various sub-projects. Note 168section is supposed to give an initial overview about the various
165that this description also lists projects that are far from complete, including 169sub-projects. Note that this description also lists projects that are far
166even those that have literally not a single line of code in them yet. 170from complete, including even those that have literally not a single line
171of code in them yet.
167 172
168GNUnet sub-projects in order of likely relevance are currently: 173GNUnet sub-projects in order of likely relevance are currently:
169 174
@@ -173,14 +178,17 @@ GNUnet sub-projects in order of likely relevance are currently:
173chat applications; this is what the developer handbook covers mostly 178chat applications; this is what the developer handbook covers mostly
174@item gnunet-gtk Gtk+-based user interfaces, including gnunet-fs-gtk 179@item gnunet-gtk Gtk+-based user interfaces, including gnunet-fs-gtk
175(file-sharing), gnunet-statistics-gtk (statistics over time), 180(file-sharing), gnunet-statistics-gtk (statistics over time),
176gnunet-peerinfo-gtk (information about current connections and known peers), 181gnunet-peerinfo-gtk (information about current connections and known
177gnunet-chat-gtk (chat GUI) and gnunet-setup (setup tool for "everything") 182peers), gnunet-chat-gtk (chat GUI) and gnunet-setup (setup tool for
178@item gnunet-fuse Mounting directories shared via GNUnet's file-sharing on Linux 183"everything")
184@item gnunet-fuse Mounting directories shared via GNUnet's file-sharing
185on Linux
179@item gnunet-update Installation and update tool 186@item gnunet-update Installation and update tool
180@item gnunet-ext Template for starting 'external' GNUnet projects 187@item gnunet-ext Template for starting 'external' GNUnet projects
181@item gnunet-java Java APIs for writing GNUnet services and applications 188@item gnunet-java Java APIs for writing GNUnet services and applications
182@c ** FIXME: Point to new website repository once we have it: 189@c ** FIXME: Point to new website repository once we have it:
183@c ** @item svn/gnunet-www/ Code and media helping drive the GNUnet website 190@c ** @item svn/gnunet-www/ Code and media helping drive the GNUnet
191website
184@item eclectic Code to run 192@item eclectic Code to run
185GNUnet nodes on testbeds for research, development, testing and evaluation 193GNUnet nodes on testbeds for research, development, testing and evaluation
186@c ** FIXME: Solve the status and location of gnunet-qt 194@c ** FIXME: Solve the status and location of gnunet-qt
@@ -197,253 +205,266 @@ We are also working on various supporting libraries and tools:
197@item libmicrohttpd GNU libmicrohttpd (embedded HTTP(S) server library) 205@item libmicrohttpd GNU libmicrohttpd (embedded HTTP(S) server library)
198@item gauger Tool for performance regression analysis 206@item gauger Tool for performance regression analysis
199@item monkey Tool for automated debugging of distributed systems 207@item monkey Tool for automated debugging of distributed systems
200@item libmwmodem Library for accessing satellite connection quality reports 208@item libmwmodem Library for accessing satellite connection quality
209reports
201@end table 210@end table
202 211
203Finally, there are various external projects (see links for a list of those 212Finally, there are various external projects (see links for a list of
204that have a public website) which build on top of the GNUnet framework. 213those that have a public website) which build on top of the GNUnet
214framework.
205 215
206@c *************************************************************************** 216@c ***********************************************************************
207@node Code overview 217@node Code overview
208@section Code overview 218@section Code overview
209 219
210This section gives a brief overview of the GNUnet source code. Specifically, we 220This section gives a brief overview of the GNUnet source code.
211sketch the function of each of the subdirectories in the @file{gnunet/src/} 221Specifically, we sketch the function of each of the subdirectories in
212directory. The order given is roughly bottom-up (in terms of the layers of the 222the @file{gnunet/src/} directory. The order given is roughly bottom-up
213system). 223(in terms of the layers of the system).
214@table @asis
215 224
225@table @asis
216@item util/ --- libgnunetutil Library with general utility functions, all 226@item util/ --- libgnunetutil Library with general utility functions, all
217GNUnet binaries link against this library. Anything from memory allocation and 227GNUnet binaries link against this library. Anything from memory
218data structures to cryptography and inter-process communication. The goal is to 228allocation and data structures to cryptography and inter-process
219provide an OS-independent interface and more 'secure' or convenient 229communication. The goal is to provide an OS-independent interface and
220implementations of commonly used primitives. The API is spread over more than a 230more 'secure' or convenient implementations of commonly used primitives.
221dozen headers, developers should study those closely to avoid duplicating 231The API is spread over more than a dozen headers, developers should study
222existing functions. 232those closely to avoid duplicating existing functions.
223@item hello/ --- libgnunethello HELLO messages are used to 233@item hello/ --- libgnunethello HELLO messages are used to
224describe under which addresses a peer can be reached (for example, protocol, 234describe under which addresses a peer can be reached (for example,
225IP, port). This library manages parsing and generating of HELLO messages. 235protocol, IP, port). This library manages parsing and generating of HELLO
226@item block/ --- libgnunetblock The DHT and other components of GNUnet store 236messages.
227information in units called 'blocks'. Each block has a type and the type 237@item block/ --- libgnunetblock The DHT and other components of GNUnet
228defines a particular format and how that binary format is to be linked to a 238store information in units called 'blocks'. Each block has a type and the
229hash code (the key for the DHT and for databases). The block library is a 239type defines a particular format and how that binary format is to be
230wapper around block plugins which provide the necessary functions for each 240linked to a hash code (the key for the DHT and for databases). The block
231block type. 241library is a wapper around block plugins which provide the necessary
242functions for each block type.
232@item statistics/ The statistics service enables associating 243@item statistics/ The statistics service enables associating
233values (of type uint64_t) with a componenet name and a string. The main uses is 244values (of type uint64_t) with a componenet name and a string. The main
234debugging (counting events), performance tracking and user entertainment (what 245uses is debugging (counting events), performance tracking and user
235did my peer do today?). 246entertainment (what did my peer do today?).
236@item arm/ The automatic-restart-manager (ARM) service 247@item arm/ The automatic-restart-manager (ARM) service
237is the GNUnet master service. Its role is to start gnunet-services, to re-start 248is the GNUnet master service. Its role is to start gnunet-services, to
238them when they crashed and finally to shut down the system when requested. 249re-start them when they crashed and finally to shut down the system when
239@item peerinfo/ The peerinfo service keeps track of which peers are known to 250requested.
240the local peer and also tracks the validated addresses for each peer (in the 251@item peerinfo/ The peerinfo service keeps track of which peers are known
241form of a HELLO message) for each of those peers. The peer is not necessarily 252to the local peer and also tracks the validated addresses for each peer
242connected to all peers known to the peerinfo service. Peerinfo provides 253(in the form of a HELLO message) for each of those peers. The peer is not
243persistent storage for peer identities --- peers are not forgotten just because 254necessarily connected to all peers known to the peerinfo service.
244of a system restart. 255Peerinfo provides persistent storage for peer identities --- peers are
256not forgotten just because of a system restart.
245@item datacache/ --- libgnunetdatacache The datacache 257@item datacache/ --- libgnunetdatacache The datacache
246library provides (temporary) block storage for the DHT. Existing plugins can 258library provides (temporary) block storage for the DHT. Existing plugins
247store blocks in Sqlite, Postgres or MySQL databases. All data stored in the 259can store blocks in Sqlite, Postgres or MySQL databases. All data stored
248cache is lost when the peer is stopped or restarted (datacache uses temporary 260in the cache is lost when the peer is stopped or restarted (datacache
249tables). 261uses temporary tables).
250@item datastore/ The datastore service stores file-sharing blocks in 262@item datastore/ The datastore service stores file-sharing blocks in
251databases for extended periods of time. In contrast to the datacache, data is 263databases for extended periods of time. In contrast to the datacache, data
252not lost when peers restart. However, quota restrictions may still cause old, 264is not lost when peers restart. However, quota restrictions may still
253expired or low-priority data to be eventually discarded. Existing plugins can 265cause old, expired or low-priority data to be eventually discarded.
254store blocks in Sqlite, Postgres or MySQL databases. 266Existing plugins can store blocks in Sqlite, Postgres or MySQL databases.
255@item template/ Template 267@item template/ Template for writing a new service. Does nothing.
256for writing a new service. Does nothing.
257@item ats/ The automatic transport 268@item ats/ The automatic transport
258selection (ATS) service is responsible for deciding which address (i.e. which 269selection (ATS) service is responsible for deciding which address (i.e.
259transport plugin) should be used for communication with other peers, and at 270which transport plugin) should be used for communication with other peers,
260what bandwidth. 271and at what bandwidth.
261@item nat/ --- libgnunetnat Library that provides basic 272@item nat/ --- libgnunetnat Library that provides basic
262functions for NAT traversal. The library supports NAT traversal with manual 273functions for NAT traversal. The library supports NAT traversal with
263hole-punching by the user, UPnP and ICMP-based autonomous NAT traversal. The 274manual hole-punching by the user, UPnP and ICMP-based autonomous NAT
264library also includes an API for testing if the current configuration works and 275traversal. The library also includes an API for testing if the current
265the @code{gnunet-nat-server} which provides an external service to test the 276configuration works and the @code{gnunet-nat-server} which provides an
266local configuration. 277external service to test the local configuration.
267@item fragmentation/ --- libgnunetfragmentation Some 278@item fragmentation/ --- libgnunetfragmentation Some
268transports (UDP and WLAN, mostly) have restrictions on the maximum transfer 279transports (UDP and WLAN, mostly) have restrictions on the maximum
269unit (MTU) for packets. The fragmentation library can be used to break larger 280transfer unit (MTU) for packets. The fragmentation library can be used to
270packets into chunks of at most 1k and transmit the resulting fragments 281break larger packets into chunks of at most 1k and transmit the resulting
271reliabily (with acknowledgement, retransmission, timeouts, etc.). 282fragments reliabily (with acknowledgement, retransmission, timeouts,
272@item transport/ The transport service is responsible for managing the basic P2P 283etc.).
273communication. It uses plugins to support P2P communication over TCP, UDP, 284@item transport/ The transport service is responsible for managing the
274HTTP, HTTPS and other protocols.The transport service validates peer addresses, 285basic P2P communication. It uses plugins to support P2P communication
275enforces bandwidth restrictions, limits the total number of connections and 286over TCP, UDP, HTTP, HTTPS and other protocols.The transport service
276enforces connectivity restrictions (i.e. friends-only). 287validates peer addresses, enforces bandwidth restrictions, limits the
288total number of connections and enforces connectivity restrictions (i.e.
289friends-only).
277@item peerinfo-tool/ 290@item peerinfo-tool/
278This directory contains the gnunet-peerinfo binary which can be used to inspect 291This directory contains the gnunet-peerinfo binary which can be used to
279the peers and HELLOs known to the peerinfo service. 292inspect the peers and HELLOs known to the peerinfo service.
280@item core/ The core 293@item core/ The core
281service is responsible for establishing encrypted, authenticated connections 294service is responsible for establishing encrypted, authenticated
282with other peers, encrypting and decrypting messages and forwarding messages to 295connections with other peers, encrypting and decrypting messages and
283higher-level services that are interested in them. 296forwarding messages to higher-level services that are interested in them.
284@item testing/ --- 297@item testing/ ---
285libgnunettesting The testing library allows starting (and stopping) peers for 298libgnunettesting The testing library allows starting (and stopping) peers
286writing testcases.@ 299for writing testcases.@
287It also supports automatic generation of configurations for 300It also supports automatic generation of configurations for peers
288peers ensuring that the ports and paths are disjoint. libgnunettesting is also 301ensuring that the ports and paths are disjoint. libgnunettesting is also
289the foundation for the testbed service 302the foundation for the testbed service
290@item testbed/ The testbed service is 303@item testbed/ The testbed service is
291used for creating small or large scale deployments of GNUnet peers for 304used for creating small or large scale deployments of GNUnet peers for
292evaluation of protocols. It facilitates peer depolyments on multiple hosts (for 305evaluation of protocols. It facilitates peer depolyments on multiple
293example, in a cluster) and establishing varous network topologies (both 306hosts (for example, in a cluster) and establishing varous network
294underlay and overlay). 307topologies (both underlay and overlay).
295@item nse/ The network size estimation (NSE) service 308@item nse/ The network size estimation (NSE) service
296implements a protocol for (securely) estimating the current size of the P2P 309implements a protocol for (securely) estimating the current size of the
297network. 310P2P network.
298@item dht/ The distributed hash table (DHT) service provides a 311@item dht/ The distributed hash table (DHT) service provides a
299distributed implementation of a hash table to store blocks under hash keys in 312distributed implementation of a hash table to store blocks under hash
300the P2P network. 313keys in the P2P network.
301@item hostlist/ The hostlist service allows learning about 314@item hostlist/ The hostlist service allows learning about
302other peers in the network by downloading HELLO messages from an HTTP server, 315other peers in the network by downloading HELLO messages from an HTTP
303can be configured to run such an HTTP server and also implements a P2P protocol 316server, can be configured to run such an HTTP server and also implements
304to advertise and automatically learn about other peers that offer a public 317a P2P protocol to advertise and automatically learn about other peers
305hostlist server. 318that offer a public hostlist server.
306@item topology/ The topology service is responsible for 319@item topology/ The topology service is responsible for
307maintaining the mesh topology. It tries to maintain connections to friends 320maintaining the mesh topology. It tries to maintain connections to friends
308(depending on the configuration) and also tries to ensure that the peer has a 321(depending on the configuration) and also tries to ensure that the peer
309decent number of active connections at all times. If necessary, new connections 322has a decent number of active connections at all times. If necessary, new
310are added. All peers should run the topology service, otherwise they may end up 323connections are added. All peers should run the topology service,
311not being connected to any other peer (unless some other service ensures that 324otherwise they may end up not being connected to any other peer (unless
312core establishes the required connections). The topology service also tells the 325some other service ensures that core establishes the required
313transport service which connections are permitted (for friend-to-friend 326connections). The topology service also tells the transport service which
314networking) 327connections are permitted (for friend-to-friend networking)
315@item fs/ The file-sharing (FS) service implements GNUnet's 328@item fs/ The file-sharing (FS) service implements GNUnet's
316file-sharing application. Both anonymous file-sharing (using gap) and 329file-sharing application. Both anonymous file-sharing (using gap) and
317non-anonymous file-sharing (using dht) are supported. 330non-anonymous file-sharing (using dht) are supported.
318@item cadet/ The CADET 331@item cadet/ The CADET
319service provides a general-purpose routing abstraction to create end-to-end 332service provides a general-purpose routing abstraction to create
320encrypted tunnels in mesh networks. We wrote a paper documenting key aspects of 333end-to-end encrypted tunnels in mesh networks. We wrote a paper
321the design. 334documenting key aspects of the design.
322@item tun/ --- libgnunettun Library for building IPv4, IPv6 335@item tun/ --- libgnunettun Library for building IPv4, IPv6
323packets and creating checksums for UDP, TCP and ICMP packets. The header 336packets and creating checksums for UDP, TCP and ICMP packets. The header
324defines C structs for common Internet packet formats and in particular structs 337defines C structs for common Internet packet formats and in particular
325for interacting with TUN (virtual network) interfaces. 338structs for interacting with TUN (virtual network) interfaces.
326@item mysql/ --- 339@item mysql/ ---
327libgnunetmysql Library for creating and executing prepared MySQL statements and 340libgnunetmysql Library for creating and executing prepared MySQL
328to manage the connection to the MySQL database. Essentially a lightweight 341statements and to manage the connection to the MySQL database.
329wrapper for the interaction between GNUnet components and libmysqlclient. 342Essentially a lightweight wrapper for the interaction between GNUnet
330@item dns/ Service that allows intercepting and modifying DNS requests of the 343components and libmysqlclient.
331local machine. Currently used for IPv4-IPv6 protocol translation (DNS-ALG) as 344@item dns/ Service that allows intercepting and modifying DNS requests of
332implemented by "pt/" and for the GNUnet naming system. The service can also be 345the local machine. Currently used for IPv4-IPv6 protocol translation
333configured to offer an exit service for DNS traffic. 346(DNS-ALG) as implemented by "pt/" and for the GNUnet naming system. The
347service can also be configured to offer an exit service for DNS traffic.
334@item vpn/ The virtual 348@item vpn/ The virtual
335public network (VPN) service provides a virtual tunnel interface (VTUN) for IP 349public network (VPN) service provides a virtual tunnel interface (VTUN)
336routing over GNUnet. Needs some other peers to run an "exit" service to work. 350for IP routing over GNUnet. Needs some other peers to run an "exit"
337Can be activated using the "gnunet-vpn" tool or integrated with DNS using the 351service to work.
338"pt" daemon. 352Can be activated using the "gnunet-vpn" tool or integrated with DNS using
353the "pt" daemon.
339@item exit/ Daemon to allow traffic from the VPN to exit this 354@item exit/ Daemon to allow traffic from the VPN to exit this
340peer to the Internet or to specific IP-based services of the local peer. 355peer to the Internet or to specific IP-based services of the local peer.
341Currently, an exit service can only be restricted to IPv4 or IPv6, not to 356Currently, an exit service can only be restricted to IPv4 or IPv6, not to
342specific ports and or IP address ranges. If this is not acceptable, additional 357specific ports and or IP address ranges. If this is not acceptable,
343firewall rules must be added manually. exit currently only works for normal 358additional firewall rules must be added manually. exit currently only
344UDP, TCP and ICMP traffic; DNS queries need to leave the system via a DNS 359works for normal UDP, TCP and ICMP traffic; DNS queries need to leave the
345service. 360system via a DNS service.
346@item pt/ protocol translation daemon. This daemon enables 4-to-6, 361@item pt/ protocol translation daemon. This daemon enables 4-to-6,
3476-to-4, 4-over-6 or 6-over-4 transitions for the local system. It essentially 3626-to-4, 4-over-6 or 6-over-4 transitions for the local system. It
348uses "DNS" to intercept DNS replies and then maps results to those offered by 363essentially uses "DNS" to intercept DNS replies and then maps results to
349the VPN, which then sends them using mesh to some daemon offering an 364those offered by the VPN, which then sends them using mesh to some daemon
350appropriate exit service. 365offering an appropriate exit service.
351@item identity/ Management of egos (alter egos) of a 366@item identity/ Management of egos (alter egos) of a user; identities are
352user; identities are essentially named ECC private keys and used for zones in 367essentially named ECC private keys and used for zones in the GNU name
353the GNU name system and for namespaces in file-sharing, but might find other 368system and for namespaces in file-sharing, but might find other uses later
354uses later
355@item revocation/ Key revocation service, can be used to revoke the 369@item revocation/ Key revocation service, can be used to revoke the
356private key of an identity if it has been compromised 370private key of an identity if it has been compromised
357@item namecache/ Cache 371@item namecache/ Cache
358for resolution results for the GNU name system; data is encrypted and can be 372for resolution results for the GNU name system; data is encrypted and can
359shared among users, loss of the data should ideally only result in a 373be shared among users, loss of the data should ideally only result in a
360performance degradation (persistence not required) 374performance degradation (persistence not required)
361@item namestore/ Database 375@item namestore/ Database
362for the GNU name system with per-user private information, persistence required 376for the GNU name system with per-user private information, persistence
377required
363@item gns/ GNU name system, a GNU approach to DNS and PKI. 378@item gns/ GNU name system, a GNU approach to DNS and PKI.
364@item dv/ A plugin 379@item dv/ A plugin
365for distance-vector (DV)-based routing. DV consists of a service and a 380for distance-vector (DV)-based routing. DV consists of a service and a
366transport plugin to provide peers with the illusion of a direct P2P connection 381transport plugin to provide peers with the illusion of a direct P2P
367for connections that use multiple (typically up to 3) hops in the actual 382connection for connections that use multiple (typically up to 3) hops in
368underlay network. 383the actual underlay network.
369@item regex/ Service for the (distributed) evaluation of 384@item regex/ Service for the (distributed) evaluation of
370regular expressions. 385regular expressions.
371@item scalarproduct/ The scalar product service offers an 386@item scalarproduct/ The scalar product service offers an
372API to perform a secure multiparty computation which calculates a scalar 387API to perform a secure multiparty computation which calculates a scalar
373product between two peers without exposing the private input vectors of the 388product between two peers without exposing the private input vectors of
374peers to each other. 389the peers to each other.
375@item consensus/ The consensus service will allow a set 390@item consensus/ The consensus service will allow a set
376of peers to agree on a set of values via a distributed set union computation. 391of peers to agree on a set of values via a distributed set union
392computation.
377@item rest/ The rest API allows access to GNUnet services using RESTful 393@item rest/ The rest API allows access to GNUnet services using RESTful
378interaction. The services provide plugins that can exposed by the rest server. 394interaction. The services provide plugins that can exposed by the rest
395server.
379@item experimentation/ The experimentation daemon coordinates distributed 396@item experimentation/ The experimentation daemon coordinates distributed
380experimentation to evaluate transport and ats properties 397experimentation to evaluate transport and ats properties
381@end table 398@end table
382 399
383@c *************************************************************************** 400@c ***********************************************************************
384@node System Architecture 401@node System Architecture
385@section System Architecture 402@section System Architecture
386 403
387GNUnet developers like legos. The blocks are indestructible, can be stacked 404GNUnet developers like legos. The blocks are indestructible, can be
388together to construct complex buildings and it is generally easy to swap one 405stacked together to construct complex buildings and it is generally easy
389block for a different one that has the same shape. GNUnet's architecture is 406to swap one block for a different one that has the same shape. GNUnet's
390based on legos: 407architecture is based on legos:
391 408
409@c images here
392 410
393 411This chapter documents the GNUnet lego system, also known as GNUnet's
394This chapter documents the GNUnet lego system, also known as GNUnet's system 412system architecture.
395architecture.
396 413
397The most common GNUnet component is a service. Services offer an API (or 414The most common GNUnet component is a service. Services offer an API (or
398several, depending on what you count as "an API") which is implemented as a 415several, depending on what you count as "an API") which is implemented as
399library. The library communicates with the main process of the service using a 416a library. The library communicates with the main process of the service
400service-specific network protocol. The main process of the service typically 417using a service-specific network protocol. The main process of the service
401doesn't fully provide everything that is needed --- it has holes to be filled 418typically doesn't fully provide everything that is needed --- it has holes
402by APIs to other services. 419to be filled by APIs to other services.
403 420
404A special kind of component in GNUnet are user interfaces and daemons. Like 421A special kind of component in GNUnet are user interfaces and daemons.
405services, they have holes to be filled by APIs of other services. Unlike 422Like services, they have holes to be filled by APIs of other services.
406services, daemons do not implement their own network protocol and they have no 423Unlike services, daemons do not implement their own network protocol and
407API: 424they have no API:
408 425
409The GNUnet system provides a range of services, daemons and user interfaces, 426The GNUnet system provides a range of services, daemons and user
410which are then combined into a layered GNUnet instance (also known as a peer). 427interfaces, which are then combined into a layered GNUnet instance (also
428known as a peer).
411 429
412Note that while it is generally possible to swap one service for another 430Note that while it is generally possible to swap one service for another
413compatible service, there is often only one implementation. However, during 431compatible service, there is often only one implementation. However,
414development we often have a "new" version of a service in parallel with an 432during development we often have a "new" version of a service in parallel
415"old" version. While the "new" version is not working, developers working on 433with an "old" version. While the "new" version is not working, developers
416other parts of the service can continue their development by simply using the 434working on other parts of the service can continue their development by
417"old" service. Alternative design ideas can also be easily investigated by 435simply using the "old" service. Alternative design ideas can also be
418swapping out individual components. This is typically achieved by simply 436easily investigated by swapping out individual components. This is
419changing the name of the "BINARY" in the respective configuration section. 437typically achieved by simply changing the name of the "BINARY" in the
420 438respective configuration section.
421Key properties of GNUnet services are that they must be separate processes and 439
422that they must protect themselves by applying tight error checking against the 440Key properties of GNUnet services are that they must be separate
423network protocol they implement (thereby achieving a certain degree of 441processes and that they must protect themselves by applying tight error
424robustness). 442checking against the network protocol they implement (thereby achieving a
443certain degree of robustness).
425 444
426On the other hand, the APIs are implemented to tolerate failures of the 445On the other hand, the APIs are implemented to tolerate failures of the
427service, isolating their host process from errors by the service. If the 446service, isolating their host process from errors by the service. If the
428service process crashes, other services and daemons around it should not also 447service process crashes, other services and daemons around it should not
429fail, but instead wait for the service process to be restarted by ARM. 448also fail, but instead wait for the service process to be restarted by
449ARM.
430 450
431 451
432@c *************************************************************************** 452@c ***********************************************************************
433@node Subsystem stability 453@node Subsystem stability
434@section Subsystem stability 454@section Subsystem stability
435 455
436This page documents the current stability of the various GNUnet subsystems. 456This page documents the current stability of the various GNUnet
437Stability here describes the expected degree of compatibility with future 457subsystems. Stability here describes the expected degree of compatibility
438versions of GNUnet. For each subsystem we distinguish between compatibility on 458with future versions of GNUnet. For each subsystem we distinguish between
439the P2P network level (communication protocol between peers), the IPC level 459compatibility on the P2P network level (communication protocol between
440(communication between the service and the service library) and the API level 460peers), the IPC level (communication between the service and the service
441(stability of the API). P2P compatibility is relevant in terms of which 461library) and the API level (stability of the API). P2P compatibility is
442applications are likely going to be able to communicate with future versions of 462relevant in terms of which applications are likely going to be able to
443the network. IPC communication is relevant for the implementation of language 463communicate with future versions of the network. IPC communication is
444bindings that re-implement the IPC messages. Finally, API compatibility is 464relevant for the implementation of language bindings that re-implement the
445relevant to developers that hope to be able to avoid changes to applications 465IPC messages. Finally, API compatibility is relevant to developers that
446build on top of the APIs of the framework. 466hope to be able to avoid changes to applications build on top of the APIs
467of the framework.
447 468
448The following table summarizes our current view of the stability of the 469The following table summarizes our current view of the stability of the
449respective protocols or APIs: 470respective protocols or APIs:
@@ -495,20 +516,21 @@ Here is a rough explanation of the values:
495@item stable 516@item stable
496No incompatible changes are planned at this time; for IPC/APIs, if 517No incompatible changes are planned at this time; for IPC/APIs, if
497there are incompatible changes, they will be minor and might only require 518there are incompatible changes, they will be minor and might only require
498minimal changes to existing code; for P2P, changes will be avoided if at all 519minimal changes to existing code; for P2P, changes will be avoided if at
499possible for the 0.10.x-series 520all possible for the 0.10.x-series
500 521
501@item testing 522@item testing
502No incompatible changes are 523No incompatible changes are
503planned at this time, but the code is still known to be in flux; so while we 524planned at this time, but the code is still known to be in flux; so while
504have no concrete plans, our expectation is that there will still be minor 525we have no concrete plans, our expectation is that there will still be
505modifications; for P2P, changes will likely be extensions that should not break 526minor modifications; for P2P, changes will likely be extensions that
506existing code 527should not break existing code
507 528
508@item unstable 529@item unstable
509Changes are planned and will happen; however, they 530Changes are planned and will happen; however, they
510will not be totally radical and the result should still resemble what is there 531will not be totally radical and the result should still resemble what is
511now; nevertheless, anticipated changes will break protocol/API compatibility 532there now; nevertheless, anticipated changes will break protocol/API
533compatibility
512 534
513@item experimental 535@item experimental
514Changes are planned and the result may look nothing like 536Changes are planned and the result may look nothing like
@@ -521,7 +543,7 @@ Someone should think about where this subsystem headed
521This subsystem does not have an API/IPC-protocol/P2P-protocol 543This subsystem does not have an API/IPC-protocol/P2P-protocol
522@end table 544@end table
523 545
524@c *************************************************************************** 546@c ***********************************************************************
525@node Naming conventions and coding style guide 547@node Naming conventions and coding style guide
526@section Naming conventions and coding style guide 548@section Naming conventions and coding style guide
527 549
@@ -529,7 +551,7 @@ Here you can find some rules to help you write code for GNUnet.
529 551
530 552
531 553
532@c *************************************************************************** 554@c ***********************************************************************
533@menu 555@menu
534* Naming conventions:: 556* Naming conventions::
535* Coding style:: 557* Coding style::
@@ -539,7 +561,7 @@ Here you can find some rules to help you write code for GNUnet.
539@subsection Naming conventions 561@subsection Naming conventions
540 562
541 563
542@c *************************************************************************** 564@c ***********************************************************************
543@menu 565@menu
544* include files:: 566* include files::
545* binaries:: 567* binaries::
@@ -571,7 +593,7 @@ Here you can find some rules to help you write code for GNUnet.
571@end itemize 593@end itemize
572@end itemize 594@end itemize
573 595
574@c *************************************************************************** 596@c ***********************************************************************
575@node binaries 597@node binaries
576@subsubsection binaries 598@subsubsection binaries
577 599
@@ -584,20 +606,20 @@ Here you can find some rules to help you write code for GNUnet.
584@item libgnunetxxx.so: library for API xxx 606@item libgnunetxxx.so: library for API xxx
585@end itemize 607@end itemize
586 608
587@c *************************************************************************** 609@c ***********************************************************************
588@node logging 610@node logging
589@subsubsection logging 611@subsubsection logging
590 612
591@itemize @bullet 613@itemize @bullet
592@item services and daemons use their directory name in GNUNET_log_setup (i.e. 614@item services and daemons use their directory name in GNUNET_log_setup
593'core') and log using plain 'GNUNET_log'. 615(i.e. 'core') and log using plain 'GNUNET_log'.
594@item command-line tools use their full name in GNUNET_log_setup (i.e. 616@item command-line tools use their full name in GNUNET_log_setup (i.e.
595'gnunet-publish') and log using plain 'GNUNET_log'. 617'gnunet-publish') and log using plain 'GNUNET_log'.
596@item service access libraries log using 'GNUNET_log_from' and use 618@item service access libraries log using 'GNUNET_log_from' and use
597'DIRNAME-api' for the component (i.e. 'core-api') 619'DIRNAME-api' for the component (i.e. 'core-api')
598@item pure libraries (without associated service) use 'GNUNET_log_from' with 620@item pure libraries (without associated service) use 'GNUNET_log_from'
599the component set to their library name (without lib or '.so'), which should 621with the component set to their library name (without lib or '.so'),
600also be their directory name (i.e. 'nat') 622which should also be their directory name (i.e. 'nat')
601@item plugins should use 'GNUNET_log_from' with the directory name and the 623@item plugins should use 'GNUNET_log_from' with the directory name and the
602plugin name combined to produce the component name (i.e. 'transport-tcp'). 624plugin name combined to produce the component name (i.e. 'transport-tcp').
603@item logging should be unified per-file by defining a LOG macro with the 625@item logging should be unified per-file by defining a LOG macro with the
@@ -605,36 +627,38 @@ appropriate arguments, along these lines:@ #define LOG(kind,...)
605GNUNET_log_from (kind, "example-api",__VA_ARGS__) 627GNUNET_log_from (kind, "example-api",__VA_ARGS__)
606@end itemize 628@end itemize
607 629
608@c *************************************************************************** 630@c ***********************************************************************
609@node configuration 631@node configuration
610@subsubsection configuration 632@subsubsection configuration
611 633
612@itemize @bullet 634@itemize @bullet
613@item paths (that are substituted in all filenames) are in PATHS (have as few 635@item paths (that are substituted in all filenames) are in PATHS (have as
614as possible) 636few as possible)
615@item all options for a particular module (src/MODULE) are under [MODULE] 637@item all options for a particular module (src/MODULE) are under [MODULE]
616@item options for a plugin of a module are under [MODULE-PLUGINNAME] 638@item options for a plugin of a module are under [MODULE-PLUGINNAME]
617@end itemize 639@end itemize
618 640
619@c *************************************************************************** 641@c ***********************************************************************
620@node exported symbols 642@node exported symbols
621@subsubsection exported symbols 643@subsubsection exported symbols
622 644
623@itemize @bullet 645@itemize @bullet
624@item must start with "GNUNET_modulename_" and be defined in "modulename.c" 646@item must start with "GNUNET_modulename_" and be defined in
647"modulename.c"
625@item exceptions: those defined in gnunet_common.h 648@item exceptions: those defined in gnunet_common.h
626@end itemize 649@end itemize
627 650
628@c *************************************************************************** 651@c ***********************************************************************
629@node private (library-internal) symbols (including structs and macros) 652@node private (library-internal) symbols (including structs and macros)
630@subsubsection private (library-internal) symbols (including structs and macros) 653@subsubsection private (library-internal) symbols (including structs and macros)
631 654
632@itemize @bullet 655@itemize @bullet
633@item must NOT start with any prefix 656@item must NOT start with any prefix
634@item must not be exported in a way that linkers could use them or@ other 657@item must not be exported in a way that linkers could use them or@ other
635libraries might see them via headers; they must be either@ declared/defined in 658libraries might see them via headers; they must be either@
636C source files or in headers that are in@ the respective directory under 659declared/defined in C source files or in headers that are in@ the
637src/modulename/ and NEVER be@ declared in src/include/. 660respective directory under src/modulename/ and NEVER be@ declared
661in src/include/.
638@end itemize 662@end itemize
639 663
640@node testcases 664@node testcases
@@ -645,17 +669,18 @@ src/modulename/ and NEVER be@ declared in src/include/.
645@item "case-description" maybe omitted if there is only one test 669@item "case-description" maybe omitted if there is only one test
646@end itemize 670@end itemize
647 671
648@c *************************************************************************** 672@c ***********************************************************************
649@node performance tests 673@node performance tests
650@subsubsection performance tests 674@subsubsection performance tests
651 675
652@itemize @bullet 676@itemize @bullet
653@item must be called "perf_module-under-test_case-description.c" 677@item must be called "perf_module-under-test_case-description.c"
654@item "case-description" maybe omitted if there is only one performance test 678@item "case-description" maybe omitted if there is only one performance
679test
655@item Must only be run if HAVE_BENCHMARKS is satisfied 680@item Must only be run if HAVE_BENCHMARKS is satisfied
656@end itemize 681@end itemize
657 682
658@c *************************************************************************** 683@c ***********************************************************************
659@node src/ directories 684@node src/ directories
660@subsubsection src/ directories 685@subsubsection src/ directories
661 686
@@ -663,15 +688,15 @@ src/modulename/ and NEVER be@ declared in src/include/.
663@item gnunet-NAME: end-user applications (i.e., gnunet-search, gnunet-arm) 688@item gnunet-NAME: end-user applications (i.e., gnunet-search, gnunet-arm)
664@item gnunet-service-NAME: service processes with accessor library (i.e., 689@item gnunet-service-NAME: service processes with accessor library (i.e.,
665gnunet-service-arm) 690gnunet-service-arm)
666@item libgnunetNAME: accessor library (_service.h-header) or standalone library 691@item libgnunetNAME: accessor library (_service.h-header) or standalone
667(_lib.h-header) 692library (_lib.h-header)
668@item gnunet-daemon-NAME: daemon process without accessor library (i.e., 693@item gnunet-daemon-NAME: daemon process without accessor library (i.e.,
669gnunet-daemon-hostlist) and no GNUnet management port 694gnunet-daemon-hostlist) and no GNUnet management port
670@item libgnunet_plugin_DIR_NAME: loadable plugins (i.e., 695@item libgnunet_plugin_DIR_NAME: loadable plugins (i.e.,
671libgnunet_plugin_transport_tcp) 696libgnunet_plugin_transport_tcp)
672@end itemize 697@end itemize
673 698
674@c *************************************************************************** 699@c ***********************************************************************
675@node Coding style 700@node Coding style
676@subsection Coding style 701@subsection Coding style
677 702
@@ -691,18 +716,18 @@ instead of
691int i,j; 716int i,j;
692@end example 717@end example
693 718
694This helps keep diffs small and forces developers to think precisely about the 719This helps keep diffs small and forces developers to think precisely about
695type of every variable. Note that @code{char *} is different from @code{const 720the type of every variable. Note that @code{char *} is different from
696char*} and @code{int} is different from @code{unsigned int} or @code{uint32_t}. 721@code{const char*} and @code{int} is different from @code{unsigned int}
697Each variable type should be chosen with care. 722or @code{uint32_t}. Each variable type should be chosen with care.
698 723
699@item While @code{goto} should generally be avoided, having a @code{goto} to 724@item While @code{goto} should generally be avoided, having a @code{goto}
700the end of a function to a block of clean up statements (free, close, etc.) can 725to the end of a function to a block of clean up statements (free, close,
701be acceptable. 726etc.) can be acceptable.
702 727
703@item Conditions should be written with constants on the left (to avoid 728@item Conditions should be written with constants on the left (to avoid
704accidental assignment) and with the 'true' target being either the 'error' case 729accidental assignment) and with the 'true' target being either the
705or the significantly simpler continuation. For example:@ 730'error' case or the significantly simpler continuation. For example:
706 731
707@example 732@example
708if (0 != stat ("filename," &sbuf)) @{ error(); @} else @{ 733if (0 != stat ("filename," &sbuf)) @{ error(); @} else @{
@@ -710,38 +735,38 @@ if (0 != stat ("filename," &sbuf)) @{ error(); @} else @{
710@} 735@}
711@end example 736@end example
712 737
713
714instead of 738instead of
739
715@example 740@example
716if (stat ("filename," &sbuf) == 0) @{ 741if (stat ("filename," &sbuf) == 0) @{
717 /* handle normal case here */ 742 /* handle normal case here */
718@} else @{ error(); @} 743@} else @{ error(); @}
719@end example 744@end example
720 745
746If possible, the error clause should be terminated with a 'return' (or
747'goto' to some cleanup routine) and in this case, the 'else' clause
748should be omitted:
721 749
722If possible, the error clause should be terminated with a 'return' (or 'goto'
723to some cleanup routine) and in this case, the 'else' clause should be omitted:
724@example 750@example
725if (0 != stat ("filename," &sbuf)) @{ error(); return; @} 751if (0 != stat ("filename," &sbuf)) @{ error(); return; @}
726/* handle normal case here */ 752/* handle normal case here */
727@end example 753@end example
728 754
755This serves to avoid deep nesting. The 'constants on the left' rule
756applies to all constants (including. @code{GNUNET_SCHEDULER_NO_TASK}),
757NULL, and enums). With the two above rules (constants on left, errors in
758'true' branch), there is only one way to write most branches correctly.
729 759
730This serves to avoid deep nesting. The 'constants on the left' rule applies to 760@item Combined assignments and tests are allowed if they do not hinder
731all constants (including. @code{GNUNET_SCHEDULER_NO_TASK}), NULL, and enums). 761code clarity. For example, one can write:
732With the two above rules (constants on left, errors in 'true' branch), there is
733only one way to write most branches correctly.
734
735@item Combined assignments and tests are allowed if they do not hinder code
736clarity. For example, one can write:@
737 762
738@example 763@example
739if (NULL == (value = lookup_function())) @{ error(); return; @} 764if (NULL == (value = lookup_function())) @{ error(); return; @}
740@end example 765@end example
741 766
742 767
743@item Use @code{break} and @code{continue} wherever possible to avoid deep(er) 768@item Use @code{break} and @code{continue} wherever possible to avoid
744nesting. Thus, we would write:@ 769deep(er) nesting. Thus, we would write:
745 770
746@example 771@example
747next = head; while (NULL != (pos = next)) @{ next = pos->next; if (! 772next = head; while (NULL != (pos = next)) @{ next = pos->next; if (!
@@ -759,9 +784,9 @@ pos->next; if (should_free (pos)) @{
759@end example 784@end example
760 785
761 786
762@item We primarily use @code{for} and @code{while} loops. A @code{while} loop 787@item We primarily use @code{for} and @code{while} loops. A @code{while}
763is used if the method for advancing in the loop is not a straightforward 788loop is used if the method for advancing in the loop is not a
764increment operation. In particular, we use:@ 789straightforward increment operation. In particular, we use:
765 790
766@example 791@example
767next = head; 792next = head;
@@ -776,11 +801,12 @@ while (NULL != (pos = next))
776@end example 801@end example
777 802
778 803
779to free entries in a list (as the iteration changes the structure of the list 804to free entries in a list (as the iteration changes the structure of the
780due to the free; the equivalent @code{for} loop does no longer follow the 805list due to the free; the equivalent @code{for} loop does no longer
781simple @code{for} paradigm of @code{for(INIT;TEST;INC)}). However, for loops 806follow the simple @code{for} paradigm of @code{for(INIT;TEST;INC)}).
782that do follow the simple @code{for} paradigm we do use @code{for}, even if it 807However, for loops that do follow the simple @code{for} paradigm we do
783involves linked lists: 808use @code{for}, even if it involves linked lists:
809
784@example 810@example
785/* simple iteration over a linked list */ 811/* simple iteration over a linked list */
786for (pos = head; NULL != pos; pos = pos->next) 812for (pos = head; NULL != pos; pos = pos->next)
@@ -791,11 +817,13 @@ for (pos = head; NULL != pos; pos = pos->next)
791 817
792 818
793@item The first argument to all higher-order functions in GNUnet must be 819@item The first argument to all higher-order functions in GNUnet must be
794declared to be of type @code{void *} and is reserved for a closure. We do not 820declared to be of type @code{void *} and is reserved for a closure. We do
795use inner functions, as trampolines would conflict with setups that use 821not use inner functions, as trampolines would conflict with setups that
796non-executable stacks.@ The first statement in a higher-order function, which 822use non-executable stacks.@ The first statement in a higher-order
797unusually should be part of the variable declarations, should assign the 823function, which unusually should be part of the variable declarations,
798@code{cls} argument to the precise expected type. For example: 824should assign the @code{cls} argument to the precise expected type.
825For example:
826
799@example 827@example
800int callback (void *cls, char *args) @{ 828int callback (void *cls, char *args) @{
801 struct Foo *foo = cls; int other_variables; 829 struct Foo *foo = cls; int other_variables;
@@ -805,9 +833,9 @@ int callback (void *cls, char *args) @{
805@end example 833@end example
806 834
807 835
808@item It is good practice to write complex @code{if} expressions instead of 836@item It is good practice to write complex @code{if} expressions instead
809using deeply nested @code{if} statements. However, except for addition and 837of using deeply nested @code{if} statements. However, except for addition
810multiplication, all operators should use parens. This is fine:@ 838and multiplication, all operators should use parens. This is fine:
811 839
812@example 840@example
813if ( (1 == foo) || ((0 == bar) && (x != y)) ) 841if ( (1 == foo) || ((0 == bar) && (x != y)) )
@@ -825,42 +853,44 @@ if (0 == bar && x != y)
825 853
826 854
827Note that splitting the @code{if} statement above is debateable as the 855Note that splitting the @code{if} statement above is debateable as the
828@code{return x} is a very trivial statement. However, once the logic after the 856@code{return x} is a very trivial statement. However, once the logic after
829branch becomes more complicated (and is still identical), the "or" formulation 857the branch becomes more complicated (and is still identical), the "or"
830should be used for sure. 858formulation should be used for sure.
831 859
832@item There should be two empty lines between the end of the function and the 860@item There should be two empty lines between the end of the function and
833comments describing the following function. There should be a single empty line 861the comments describing the following function. There should be a single
834after the initial variable declarations of a function. If a function has no 862empty line after the initial variable declarations of a function. If a
835local variables, there should be no initial empty line. If a long function 863function has no local variables, there should be no initial empty line. If
836consists of several complex steps, those steps might be separated by an empty 864a long function consists of several complex steps, those steps might be
837line (possibly followed by a comment describing the following step). The code 865separated by an empty line (possibly followed by a comment describing the
838should not contain empty lines in arbitrary places; if in doubt, it is likely 866following step). The code should not contain empty lines in arbitrary
839better to NOT have an empty line (this way, more code will fit on the screen). 867places; if in doubt, it is likely better to NOT have an empty line (this
868way, more code will fit on the screen).
840@end itemize 869@end itemize
841 870
842@c *************************************************************************** 871@c ***********************************************************************
843@node Build-system 872@node Build-system
844@section Build-system 873@section Build-system
845 874
846If you have code that is likely not to compile or build rules you might want to 875If you have code that is likely not to compile or build rules you might
847not trigger for most developers, use "if HAVE_EXPERIMENTAL" in your 876want to not trigger for most developers, use "if HAVE_EXPERIMENTAL" in
848Makefile.am. Then it is OK to (temporarily) add non-compiling (or 877your Makefile.am. Then it is OK to (temporarily) add non-compiling (or
849known-to-not-port) code. 878known-to-not-port) code.
850 879
851If you want to compile all testcases but NOT run them, run configure with the@ 880If you want to compile all testcases but NOT run them, run configure with
852@code{--enable-test-suppression} option. 881the @code{--enable-test-suppression} option.
853 882
854If you want to run all testcases, including those that take a while, run 883If you want to run all testcases, including those that take a while, run
855configure with the@ @code{--enable-expensive-testcases} option. 884configure with the @code{--enable-expensive-testcases} option.
856 885
857If you want to compile and run benchmarks, run configure with the@ 886If you want to compile and run benchmarks, run configure with the
858@code{--enable-benchmarks} option. 887@code{--enable-benchmarks} option.
859 888
860If you want to obtain code coverage results, run configure with the@ 889If you want to obtain code coverage results, run configure with the
861@code{--enable-coverage} option and run the coverage.sh script in contrib/. 890@code{--enable-coverage} option and run the coverage.sh script in
891@file{contrib/}.
862 892
863@c *************************************************************************** 893@c ***********************************************************************
864@node Developing extensions for GNUnet using the gnunet-ext template 894@node Developing extensions for GNUnet using the gnunet-ext template
865@section Developing extensions for GNUnet using the gnunet-ext template 895@section Developing extensions for GNUnet using the gnunet-ext template
866 896
@@ -876,37 +906,44 @@ First of all you have to obtain gnunet-ext from git:
876@code{git clone https://gnunet.org/git/gnunet-ext.git} 906@code{git clone https://gnunet.org/git/gnunet-ext.git}
877 907
878The next step is to bootstrap and configure it. For configure you have to 908The next step is to bootstrap and configure it. For configure you have to
879provide the path containing GNUnet with @code{--with-gnunet=/path/to/gnunet} 909provide the path containing GNUnet with
880and the prefix where you want the install the extension using 910@code{--with-gnunet=/path/to/gnunet} and the prefix where you want the
881@code{--prefix=/path/to/install}@ @code{@ ./bootstrap@ ./configure 911install the extension using @code{--prefix=/path/to/install}:
882--prefix=/path/to/install --with-gnunet=/path/to/gnunet@ } 912
913@example
914./bootstrap
915./configure --prefix=/path/to/install --with-gnunet=/path/to/gnunet
916@end example
883 917
884When your GNUnet installation is not included in the default linker search 918When your GNUnet installation is not included in the default linker search
885path, you have to add @code{/path/to/gnunet} to the file @code{/etc/ld.so.conf} 919path, you have to add @code{/path/to/gnunet} to the file
886and run @code{ldconfig} or your add it to the environmental variable 920@file{/etc/ld.so.conf} and run @code{ldconfig} or your add it to the
887@code{LD_LIBRARY_PATH} by using 921environmental variable @code{LD_LIBRARY_PATH} by using
888 922
889@code{export LD_LIBRARY_PATH=/path/to/gnunet/lib} 923@code{export LD_LIBRARY_PATH=/path/to/gnunet/lib}
890 924
891@c *************************************************************************** 925@c ***********************************************************************
892@node Writing testcases 926@node Writing testcases
893@section Writing testcases 927@section Writing testcases
894 928
895Ideally, any non-trivial GNUnet code should be covered by automated testcases. 929Ideally, any non-trivial GNUnet code should be covered by automated
896Testcases should reside in the same place as the code that is being tested. The 930testcases. Testcases should reside in the same place as the code that is
897name of source files implementing tests should begin with "test_" followed by 931being tested. The name of source files implementing tests should begin
898the name of the file that contains the code that is being tested. 932with "test_" followed by the name of the file that contains the code that
899 933is being tested.
900Testcases in GNUnet should be integrated with the autotools build system. This 934
901way, developers and anyone building binary packages will be able to run all 935Testcases in GNUnet should be integrated with the autotools build system.
902testcases simply by running @code{make check}. The final testcases shipped with 936This way, developers and anyone building binary packages will be able to
903the distribution should output at most some brief progress information and not 937run all testcases simply by running @code{make check}. The final
904display debug messages by default. The success or failure of a testcase must be 938testcases shipped with the distribution should output at most some brief
905indicated by returning zero (success) or non-zero (failure) from the main 939progress information and not display debug messages by default. The
906method of the testcase. The integration with the autotools is relatively 940success or failure of a testcase must be indicated by returning zero
907straightforward and only requires modifications to the @code{Makefile.am} in 941(success) or non-zero (failure) from the main method of the testcase. The
908the directory containing the testcase. For a testcase testing the code in 942integration with the autotools is relatively straightforward and only
909@code{foo.c} the @code{Makefile.am} would contain the following lines: 943requires modifications to the @code{Makefile.am} in the directory
944containing the testcase. For a testcase testing the code in @code{foo.c}
945the @code{Makefile.am} would contain the following lines:
946
910@example 947@example
911check_PROGRAMS = test_foo TESTS = $(check_PROGRAMS) test_foo_SOURCES = 948check_PROGRAMS = test_foo TESTS = $(check_PROGRAMS) test_foo_SOURCES =
912test_foo.c test_foo_LDADD = $(top_builddir)/src/util/libgnunetutil.la 949test_foo.c test_foo_LDADD = $(top_builddir)/src/util/libgnunetutil.la
@@ -915,51 +952,53 @@ test_foo.c test_foo_LDADD = $(top_builddir)/src/util/libgnunetutil.la
915Naturally, other libraries used by the testcase may be specified in the 952Naturally, other libraries used by the testcase may be specified in the
916@code{LDADD} directive as necessary. 953@code{LDADD} directive as necessary.
917 954
918Often testcases depend on additional input files, such as a configuration file. 955Often testcases depend on additional input files, such as a configuration
919These support files have to be listed using the EXTRA_DIST directive in order 956file. These support files have to be listed using the EXTRA_DIST
920to ensure that they are included in the distribution. Example: 957directive in order to ensure that they are included in the distribution.
958Example:
959
921@example 960@example
922EXTRA_DIST = test_foo_data.conf 961EXTRA_DIST = test_foo_data.conf
923@end example 962@end example
924 963
964Executing @code{make check} will run all testcases in the current
965directory and all subdirectories. Testcases can be compiled individually
966by running @code{make test_foo} and then invoked directly using
967@code{./test_foo}. Note that due to the use of plugins in GNUnet, it is
968typically necessary to run @code{make install} before running any
969testcases. Thus the canonical command @code{make check install} has to be
970changed to @code{make install check} for GNUnet.
925 971
926Executing @code{make check} will run all testcases in the current directory and 972@c ***********************************************************************
927all subdirectories. Testcases can be compiled individually by running
928@code{make test_foo} and then invoked directly using @code{./test_foo}. Note
929that due to the use of plugins in GNUnet, it is typically necessary to run
930@code{make install} before running any testcases. Thus the canonical command
931@code{make check install} has to be changed to @code{make install check} for
932GNUnet.
933
934@c ***************************************************************************
935@node GNUnet's TESTING library 973@node GNUnet's TESTING library
936@section GNUnet's TESTING library 974@section GNUnet's TESTING library
937 975
938The TESTING library is used for writing testcases which involve starting a 976The TESTING library is used for writing testcases which involve starting a
939single or multiple peers. While peers can also be started by testcases using 977single or multiple peers. While peers can also be started by testcases
940the ARM subsystem, using TESTING library provides an elegant way to do this. 978using the ARM subsystem, using TESTING library provides an elegant way to
941The configurations of the peers are auto-generated from a given template to 979do this. The configurations of the peers are auto-generated from a given
942have non-conflicting port numbers ensuring that peers' services do not run into 980template to have non-conflicting port numbers ensuring that peers'
943bind errors. This is achieved by testing ports' availability by binding a 981services do not run into bind errors. This is achieved by testing ports'
944listening socket to them before allocating them to services in the generated 982availability by binding a listening socket to them before allocating them
945configurations. 983to services in the generated configurations.
946 984
947An another advantage while using TESTING is that it shortens the testcase 985An another advantage while using TESTING is that it shortens the testcase
948startup time as the hostkeys for peers are copied from a pre-computed set of 986startup time as the hostkeys for peers are copied from a pre-computed set
949hostkeys instead of generating them at peer startup which may take a 987of hostkeys instead of generating them at peer startup which may take a
950considerable amount of time when starting multiple peers or on an embedded 988considerable amount of time when starting multiple peers or on an embedded
951processor. 989processor.
952 990
953TESTING also allows for certain services to be shared among peers. This feature 991TESTING also allows for certain services to be shared among peers. This
954is invaluable when testing with multiple peers as it helps to reduce the number 992feature is invaluable when testing with multiple peers as it helps to
955of services run per each peer and hence the total number of processes run per 993reduce the number of services run per each peer and hence the total
956testcase. 994number of processes run per testcase.
957 995
958TESTING library only handles creating, starting and stopping peers. Features 996TESTING library only handles creating, starting and stopping peers.
959useful for testcases such as connecting peers in a topology are not available 997Features useful for testcases such as connecting peers in a topology are
960in TESTING but are available in the TESTBED subsystem. Furthermore, TESTING 998not available in TESTING but are available in the TESTBED subsystem.
961only creates peers on the localhost, however by using TESTBED testcases can 999Furthermore, TESTING only creates peers on the localhost, however by
962benefit from creating peers across multiple hosts. 1000using TESTBED testcases can benefit from creating peers across multiple
1001hosts.
963 1002
964@menu 1003@menu
965* API:: 1004* API::
@@ -968,113 +1007,118 @@ benefit from creating peers across multiple hosts.
968* Testing with multiple processes:: 1007* Testing with multiple processes::
969@end menu 1008@end menu
970 1009
971@c *************************************************************************** 1010@c ***********************************************************************
972@node API 1011@node API
973@subsection API 1012@subsection API
974 1013
975TESTING abstracts a group of peers as a TESTING system. All peers in a system 1014TESTING abstracts a group of peers as a TESTING system. All peers in a
976have common hostname and no two services of these peers have a same port or a 1015system have common hostname and no two services of these peers have a
977UNIX domain socket path. 1016same port or a UNIX domain socket path.
978 1017
979TESTING system can be created with the function 1018TESTING system can be created with the function
980@code{GNUNET_TESTING_system_create()} which returns a handle to the system. 1019@code{GNUNET_TESTING_system_create()} which returns a handle to the
981This function takes a directory path which is used for generating the 1020system. This function takes a directory path which is used for generating
982configurations of peers, an IP address from which connections to the peers' 1021the configurations of peers, an IP address from which connections to the
983services should be allowed, the hostname to be used in peers' configuration, 1022peers' services should be allowed, the hostname to be used in peers'
984and an array of shared service specifications of type @code{struct 1023configuration, and an array of shared service specifications of type
985GNUNET_TESTING_SharedService}. 1024@code{struct GNUNET_TESTING_SharedService}.
986 1025
987The shared service specification must specify the name of the service to share, 1026The shared service specification must specify the name of the service to
988the configuration pertaining to that shared service and the maximum number of 1027share, the configuration pertaining to that shared service and the
989peers that are allowed to share a single instance of the shared service. 1028maximum number of peers that are allowed to share a single instance of
990 1029the shared service.
991TESTING system created with @code{GNUNET_TESTING_system_create()} chooses ports 1030
992from the default range 12000 - 56000 while auto-generating configurations for 1031TESTING system created with @code{GNUNET_TESTING_system_create()} chooses
993peers. This range can be customised with the function 1032ports from the default range 12000 - 56000 while auto-generating
994@code{GNUNET_TESTING_system_create_with_portrange()}. This function is similar 1033configurations for peers. This range can be customised with the function
995to @code{GNUNET_TESTING_system_create()} except that it take 2 additional 1034@code{GNUNET_TESTING_system_create_with_portrange()}. This function is
996parameters --- the start and end of the port range to use. 1035similar to @code{GNUNET_TESTING_system_create()} except that it take 2
1036additional parameters --- the start and end of the port range to use.
997 1037
998A TESTING system is destroyed with the funciton 1038A TESTING system is destroyed with the funciton
999@code{GNUNET_TESTING_system_destory()}. This function takes the handle of the 1039@code{GNUNET_TESTING_system_destory()}. This function takes the handle of
1000system and a flag to remove the files created in the directory used to generate 1040the system and a flag to remove the files created in the directory used
1001configurations. 1041to generate configurations.
1002 1042
1003A peer is created with the function @code{GNUNET_TESTING_peer_configure()}. 1043A peer is created with the function
1004This functions takes the system handle, a configuration template from which the 1044@code{GNUNET_TESTING_peer_configure()}. This functions takes the system
1005configuration for the peer is auto-generated and the index from where the 1045handle, a configuration template from which the configuration for the peer
1006hostkey for the peer has to be copied from. When successfull, this function 1046is auto-generated and the index from where the hostkey for the peer has to
1007returs a handle to the peer which can be used to start and stop it and to 1047be copied from. When successfull, this function returs a handle to the
1008obtain the identity of the peer. If unsuccessful, a NULL pointer is returned 1048peer which can be used to start and stop it and to obtain the identity of
1009with an error message. This function handles the generated configuration to 1049the peer. If unsuccessful, a NULL pointer is returned with an error
1010have non-conflicting ports and paths. 1050message. This function handles the generated configuration to have
1051non-conflicting ports and paths.
1011 1052
1012Peers can be started and stopped by calling the functions 1053Peers can be started and stopped by calling the functions
1013@code{GNUNET_TESTING_peer_start()} and @code{GNUNET_TESTING_peer_stop()} 1054@code{GNUNET_TESTING_peer_start()} and @code{GNUNET_TESTING_peer_stop()}
1014respectively. A peer can be destroyed by calling the function 1055respectively. A peer can be destroyed by calling the function
1015@code{GNUNET_TESTING_peer_destroy}. When a peer is destroyed, the ports and 1056@code{GNUNET_TESTING_peer_destroy}. When a peer is destroyed, the ports
1016paths in allocated in its configuration are reclaimed for usage in new 1057and paths in allocated in its configuration are reclaimed for usage in new
1017peers. 1058peers.
1018 1059
1019@c *************************************************************************** 1060@c ***********************************************************************
1020@node Finer control over peer stop 1061@node Finer control over peer stop
1021@subsection Finer control over peer stop 1062@subsection Finer control over peer stop
1022 1063
1023Using @code{GNUNET_TESTING_peer_stop()} is normally fine for testcases. 1064Using @code{GNUNET_TESTING_peer_stop()} is normally fine for testcases.
1024However, calling this function for each peer is inefficient when trying to 1065However, calling this function for each peer is inefficient when trying to
1025shutdown multiple peers as this function sends the termination signal to the 1066shutdown multiple peers as this function sends the termination signal to
1026given peer process and waits for it to terminate. It would be faster in this 1067the given peer process and waits for it to terminate. It would be faster
1027case to send the termination signals to the peers first and then wait on them. 1068in this case to send the termination signals to the peers first and then
1028This is accomplished by the functions @code{GNUNET_TESTING_peer_kill()} which 1069wait on them. This is accomplished by the functions
1029sends a termination signal to the peer, and the function 1070@code{GNUNET_TESTING_peer_kill()} which sends a termination signal to the
1030@code{GNUNET_TESTING_peer_wait()} which waits on the peer. 1071peer, and the function @code{GNUNET_TESTING_peer_wait()} which waits on
1031 1072the peer.
1032Further finer control can be achieved by choosing to stop a peer asynchronously 1073
1033with the function @code{GNUNET_TESTING_peer_stop_async()}. This function takes 1074Further finer control can be achieved by choosing to stop a peer
1034a callback parameter and a closure for it in addition to the handle to the peer 1075asynchronously with the function @code{GNUNET_TESTING_peer_stop_async()}.
1035to stop. The callback function is called with the given closure when the peer 1076This function takes a callback parameter and a closure for it in addition
1036is stopped. Using this function eliminates blocking while waiting for the peer 1077to the handle to the peer to stop. The callback function is called with
1037to terminate. 1078the given closure when the peer is stopped. Using this function
1079eliminates blocking while waiting for the peer to terminate.
1038 1080
1039An asynchronous peer stop can be cancelled by calling the function 1081An asynchronous peer stop can be cancelled by calling the function
1040@code{GNUNET_TESTING_peer_stop_async_cancel()}. Note that calling this function 1082@code{GNUNET_TESTING_peer_stop_async_cancel()}. Note that calling this
1041does not prevent the peer from terminating if the termination signal has 1083function does not prevent the peer from terminating if the termination
1042already been sent to it. It does, however, cancels the callback to be called 1084signal has already been sent to it. It does, however, cancels the
1043when the peer is stopped. 1085callback to be called when the peer is stopped.
1044 1086
1045@c *************************************************************************** 1087@c ***********************************************************************
1046@node Helper functions 1088@node Helper functions
1047@subsection Helper functions 1089@subsection Helper functions
1048 1090
1049Most of the testcases can benefit from an abstraction which configures a peer 1091Most of the testcases can benefit from an abstraction which configures a
1050and starts it. This is provided by the function 1092peer and starts it. This is provided by the function
1051@code{GNUNET_TESTING_peer_run()}. This function takes the testing directory 1093@code{GNUNET_TESTING_peer_run()}. This function takes the testing
1052pathname, a configuration template, a callback and its closure. This function 1094directory pathname, a configuration template, a callback and its closure.
1053creates a peer in the given testing directory by using the configuration 1095This function creates a peer in the given testing directory by using the
1054template, starts the peer and calls the given callback with the given closure. 1096configuration template, starts the peer and calls the given callback with
1055 1097the given closure.
1056The function @code{GNUNET_TESTING_peer_run()} starts the ARM service of the 1098
1057peer which starts the rest of the configured services. A similar function 1099The function @code{GNUNET_TESTING_peer_run()} starts the ARM service of
1058@code{GNUNET_TESTING_service_run} can be used to just start a single service of 1100the peer which starts the rest of the configured services. A similar
1059a peer. In this case, the peer's ARM service is not started; instead, only the 1101function @code{GNUNET_TESTING_service_run} can be used to just start a
1060given service is run. 1102single service of a peer. In this case, the peer's ARM service is not
1061 1103started; instead, only the given service is run.
1062@c *************************************************************************** 1104
1105@c ***********************************************************************
1063@node Testing with multiple processes 1106@node Testing with multiple processes
1064@subsection Testing with multiple processes 1107@subsection Testing with multiple processes
1065 1108
1066When testing GNUnet, the splitting of the code into a services and clients 1109When testing GNUnet, the splitting of the code into a services and clients
1067often complicates testing. The solution to this is to have the testcase fork 1110often complicates testing. The solution to this is to have the testcase
1068@code{gnunet-service-arm}, ask it to start the required server and daemon 1111fork @code{gnunet-service-arm}, ask it to start the required server and
1069processes and then execute appropriate client actions (to test the client APIs 1112daemon processes and then execute appropriate client actions (to test the
1070or the core module or both). If necessary, multiple ARM services can be forked 1113client APIs or the core module or both). If necessary, multiple ARM
1071using different ports (!) to simulate a network. However, most of the time only 1114services can be forked using different ports (!) to simulate a network.
1072one ARM process is needed. Note that on exit, the testcase should shutdown ARM 1115However, most of the time only one ARM process is needed. Note that on
1073with a @code{TERM} signal (to give it the chance to cleanly stop its child 1116exit, the testcase should shutdown ARM with a @code{TERM} signal (to give
1074processes). 1117it the chance to cleanly stop its child processes).
1075 1118
1076The following code illustrates spawning and killing an ARM process from a 1119The following code illustrates spawning and killing an ARM process from a
1077testcase: 1120testcase:
1121
1078@example 1122@example
1079static void run (void *cls, char *const *args, const char 1123static void run (void *cls, char *const *args, const char
1080*cfgfile, const struct GNUNET_CONFIGURATION_Handle *cfg) @{ struct 1124*cfgfile, const struct GNUNET_CONFIGURATION_Handle *cfg) @{ struct
@@ -1090,43 +1134,47 @@ GNUNET_PROGRAM_run (argc, argv, "NAME-OF-TEST", "nohelp", options, &run, cls);
1090 1134
1091 1135
1092An alternative way that works well to test plugins is to implement a 1136An alternative way that works well to test plugins is to implement a
1093mock-version of the environment that the plugin expects and then to simply load 1137mock-version of the environment that the plugin expects and then to
1094the plugin directly. 1138simply load the plugin directly.
1095 1139
1096@c *************************************************************************** 1140@c ***********************************************************************
1097@node Performance regression analysis with Gauger 1141@node Performance regression analysis with Gauger
1098@section Performance regression analysis with Gauger 1142@section Performance regression analysis with Gauger
1099 1143
1100To help avoid performance regressions, GNUnet uses Gauger. Gauger is a simple 1144To help avoid performance regressions, GNUnet uses Gauger. Gauger is a
1101logging tool that allows remote hosts to send performance data to a central 1145simple logging tool that allows remote hosts to send performance data to
1102server, where this data can be analyzed and visualized. Gauger shows graphs of 1146a central server, where this data can be analyzed and visualized. Gauger
1103the repository revisions and the performace data recorded for each revision, so 1147shows graphs of the repository revisions and the performace data recorded
1104sudden performance peaks or drops can be identified and linked to a specific 1148for each revision, so sudden performance peaks or drops can be identified
1105revision number. 1149and linked to a specific revision number.
1106 1150
1107In the case of GNUnet, the buildbots log the performance data obtained during 1151In the case of GNUnet, the buildbots log the performance data obtained
1108the tests after each build. The data can be accesed on GNUnet's Gauger page. 1152during the tests after each build. The data can be accesed on GNUnet's
1109 1153Gauger page.
1110The menu on the left allows to select either the results of just one build bot 1154
1111(under "Hosts") or review the data from all hosts for a given test result 1155The menu on the left allows to select either the results of just one
1112(under "Metrics"). In case of very different absolute value of the results, for 1156build bot (under "Hosts") or review the data from all hosts for a given
1113instance arm vs. amd64 machines, the option "Normalize" on a metric view can 1157test result (under "Metrics"). In case of very different absolute value
1114help to get an idea about the performance evolution across all hosts. 1158of the results, for instance arm vs. amd64 machines, the option
1115 1159"Normalize" on a metric view can help to get an idea about the
1116Using Gauger in GNUnet and having the performance of a module tracked over time 1160performance evolution across all hosts.
1117is very easy. First of course, the testcase must generate some consistent 1161
1118metric, which makes sense to have logged. Highly volatile or random dependant 1162Using Gauger in GNUnet and having the performance of a module tracked over
1119metrics probably are not ideal candidates for meaningful regression detection. 1163time is very easy. First of course, the testcase must generate some
1120 1164consistent metric, which makes sense to have logged. Highly volatile or
1121To start logging any value, just include @code{gauger.h} in your testcase code. 1165random dependant metrics probably are not ideal candidates for meaningful
1122Then, use the macro @code{GAUGER()} to make the buildbots log whatever value is 1166regression detection.
1123of interest for you to @code{gnunet.org}'s Gauger server. No setup is necessary 1167
1124as most buildbots have already everything in place and new metrics are created 1168To start logging any value, just include @code{gauger.h} in your testcase
1125on demand. To delete a metric, you need to contact a member of the GNUnet 1169code. Then, use the macro @code{GAUGER()} to make the buildbots log
1126development team (a file will need to be removed manually from the respective 1170whatever value is of interest for you to @code{gnunet.org}'s Gauger
1127directory). 1171server. No setup is necessary as most buildbots have already everything
1172in place and new metrics are created on demand. To delete a metric, you
1173need to contact a member of the GNUnet development team (a file will need
1174to be removed manually from the respective directory).
1128 1175
1129The code in the test should look like this: 1176The code in the test should look like this:
1177
1130@example 1178@example
1131[other includes] 1179[other includes]
1132#include <gauger.h> 1180#include <gauger.h>
@@ -1139,13 +1187,14 @@ int main (int argc, char *argv[]) @{
1139 1187
1140 1188
1141Where: 1189Where:
1190
1142@table @asis 1191@table @asis
1143 1192
1144@item @strong{YOUR_MODULE} is a category in the gauger page and should be the 1193@item @strong{YOUR_MODULE} is a category in the gauger page and should be
1145name of the module or subsystem like "Core" or "DHT" 1194the name of the module or subsystem like "Core" or "DHT"
1146@item @strong{METRIC} is 1195@item @strong{METRIC} is
1147the name of the metric being collected and should be concise and descriptive, 1196the name of the metric being collected and should be concise and
1148like "PUT operations in sqlite-datastore". 1197descriptive, like "PUT operations in sqlite-datastore".
1149@item @strong{value} is the value 1198@item @strong{value} is the value
1150of the metric that is logged for this run. 1199of the metric that is logged for this run.
1151@item @strong{UNIT} is the unit in 1200@item @strong{UNIT} is the unit in
@@ -1155,7 +1204,7 @@ which the value is measured, for instance "kb/s" or "kb of RAM/node".
1155If you wish to use Gauger for your own project, you can grab a copy of the 1204If you wish to use Gauger for your own project, you can grab a copy of the
1156latest stable release or check out Gauger's Subversion repository. 1205latest stable release or check out Gauger's Subversion repository.
1157 1206
1158@c *************************************************************************** 1207@c ***********************************************************************
1159@node GNUnet's TESTBED Subsystem 1208@node GNUnet's TESTBED Subsystem
1160@section GNUnet's TESTBED Subsystem 1209@section GNUnet's TESTBED Subsystem
1161 1210
@@ -1166,39 +1215,46 @@ The architecture of the testbed module is divided into the following:
1166@itemize @bullet 1215@itemize @bullet
1167 1216
1168@item Testbed API: An API which is used by the testing driver programs. It 1217@item Testbed API: An API which is used by the testing driver programs. It
1169provides with functions for creating, destroying, starting, stopping peers, 1218provides with functions for creating, destroying, starting, stopping
1170etc. 1219peers, etc.
1171 1220
1172@item Testbed service (controller): A service which is started through the 1221@item Testbed service (controller): A service which is started through the
1173Testbed API. This service handles operations to create, destroy, start, stop 1222Testbed API. This service handles operations to create, destroy, start,
1174peers, connect them, modify their configurations. 1223stop peers, connect them, modify their configurations.
1175 1224
1176@item Testbed helper: When a controller has to be started on a host, the 1225@item Testbed helper: When a controller has to be started on a host, the
1177testbed API starts the testbed helper on that host which in turn starts the 1226testbed API starts the testbed helper on that host which in turn starts
1178controller. The testbed helper receives a configuration for the controller 1227the controller. The testbed helper receives a configuration for the
1179through its stdin and changes it to ensure the controller doesn't run into any 1228controller through its stdin and changes it to ensure the controller
1180port conflict on that host. 1229doesn't run into any port conflict on that host.
1181@end itemize 1230@end itemize
1182 1231
1183 1232
1184The testbed service (controller) is different from the other GNUnet services in 1233The testbed service (controller) is different from the other GNUnet
1185that it is not started by ARM and is not supposed to be run as a daemon. It is 1234services in that it is not started by ARM and is not supposed to be run
1186started by the testbed API through a testbed helper. In a typical scenario 1235as a daemon. It is started by the testbed API through a testbed helper.
1187involving multiple hosts, a controller is started on each host. Controllers 1236In a typical scenario involving multiple hosts, a controller is started
1188take up the actual task of creating peers, starting and stopping them on the 1237on each host. Controllers take up the actual task of creating peers,
1189hosts they run. 1238starting and stopping them on the hosts they run.
1190 1239
1191While running deployments on a single localhost the testbed API starts the 1240While running deployments on a single localhost the testbed API starts the
1192testbed helper directly as a child process. When running deployments on remote 1241testbed helper directly as a child process. When running deployments on
1193hosts the testbed API starts Testbed Helpers on each remote host through remote 1242remote hosts the testbed API starts Testbed Helpers on each remote host
1194shell. By default testbed API uses SSH as a remote shell. This can be changed 1243through remote shell. By default testbed API uses SSH as a remote shell.
1195by setting the environmental variable GNUNET_TESTBED_RSH_CMD to the required 1244This can be changed by setting the environmental variable
1196remote shell program. This variable can also contain parameters which are to be 1245GNUNET_TESTBED_RSH_CMD to the required remote shell program. This
1197passed to the remote shell program. For e.g:@ @code{@ export 1246variable can also contain parameters which are to be passed to the remote
1198GNUNET_TESTBED_RSH_CMD="ssh -o BatchMode=yes -o 1247shell program. For e.g:
1199NoHostAuthenticationForLocalhost=yes %h"@ }@ Substitutions are allowed int the 1248
1200above command string also allows for substitions. through placemarks which 1249@example
1201begin with a `%'. At present the following substitutions are supported 1250export GNUNET_TESTBED_RSH_CMD="ssh -o BatchMode=yes \
1251-o NoHostAuthenticationForLocalhost=yes %h"@
1252@end example
1253
1254Substitutions are allowed int the above command string also allows for
1255substitions. through placemarks which begin with a `%'. At present the
1256following substitutions are supported
1257
1202@itemize @bullet 1258@itemize @bullet
1203@item 1259@item
1204%h: hostname 1260%h: hostname
@@ -1208,41 +1264,56 @@ begin with a `%'. At present the following substitutions are supported
1208%p: port 1264%p: port
1209@end itemize 1265@end itemize
1210 1266
1211Note that the substitution placemark is replaced only when the corresponding 1267Note that the substitution placemark is replaced only when the
1212field is available and only once. Specifying @code{%u@@%h} doesn't work either. 1268corresponding field is available and only once. Specifying @code{%u@@%h}
1213If you want to user username substitutions for SSH use the argument @code{-l} 1269doesn't work either. If you want to user username substitutions for SSH
1214before the username substitution. Ex: @code{ssh -l %u -p %p %h} 1270use the argument @code{-l} before the username substitution.
1271Ex: @code{ssh -l %u -p %p %h}
1215 1272
1216The testbed API and the helper communicate through the helpers stdin and 1273The testbed API and the helper communicate through the helpers stdin and
1217stdout. As the helper is started through a remote shell on remote hosts any 1274stdout. As the helper is started through a remote shell on remote hosts
1218output messages from the remote shell interfere with the communication and 1275any output messages from the remote shell interfere with the communication
1219results in a failure while starting the helper. For this reason, it is 1276and results in a failure while starting the helper. For this reason, it is
1220suggested to use flags to make the remote shells produce no output messages and 1277suggested to use flags to make the remote shells produce no output
1221to have password-less logins. The default remote shell, SSH, the default 1278messages and to have password-less logins. The default remote shell, SSH,
1222options are "-o BatchMode=yes -o NoHostBasedAuthenticationForLocalhost=yes". 1279the default options are:
1280
1281@example
1282-o BatchMode=yes -o NoHostBasedAuthenticationForLocalhost=yes"
1283@end example
1284
1223Password-less logins should be ensured by using SSH keys. 1285Password-less logins should be ensured by using SSH keys.
1224 1286
1225Since the testbed API executes the remote shell as a non-interactive shell, 1287Since the testbed API executes the remote shell as a non-interactive
1226certain scripts like .bashrc, .profiler may not be executed. If this is the 1288shell, certain scripts like .bashrc, .profiler may not be executed. If
1227case testbed API can be forced to execute an interactive shell by setting up 1289this is the case testbed API can be forced to execute an interactive
1228the environmental variable `GNUNET_TESTBED_RSH_CMD_SUFFIX' to a shell program. 1290shell by setting up the environmental variable
1229An example could be:@ @code{@ export GNUNET_TESTBED_RSH_CMD_SUFFIX="sh -lc"@ }@ 1291`GNUNET_TESTBED_RSH_CMD_SUFFIX' to a shell program.
1230The testbed API will then execute the remote shell program as: @code{ 1292An example could be:
1231$GNUNET_TESTBED_RSH_CMD -p $port $dest $GNUNET_TESTBED_RSH_CMD_SUFFIX 1293
1232gnunet-helper-testbed } 1294@example
1233 1295export GNUNET_TESTBED_RSH_CMD_SUFFIX="sh -lc"
1234On some systems, problems may arise while starting testbed helpers if GNUnet is 1296@end example
1235installed into a custom location since the helper may not be found in the 1297
1236standard path. This can be addressed by setting the variable 1298The testbed API will then execute the remote shell program as:
1237`HELPER_BINARY_PATH' to the path of the testbed helper. Testbed API will then 1299
1238use this path to start helper binaries both locally and remotely. 1300@example
1301$GNUNET_TESTBED_RSH_CMD -p $port $dest $GNUNET_TESTBED_RSH_CMD_SUFFIX \
1302gnunet-helper-testbed
1303@end example
1304
1305On some systems, problems may arise while starting testbed helpers if
1306GNUnet is installed into a custom location since the helper may not be
1307found in the standard path. This can be addressed by setting the variable
1308`HELPER_BINARY_PATH' to the path of the testbed helper. Testbed API will
1309then use this path to start helper binaries both locally and remotely.
1239 1310
1240Testbed API can accessed by including "gnunet_testbed_service.h" file and 1311Testbed API can accessed by including "gnunet_testbed_service.h" file and
1241linking with -lgnunettestbed. 1312linking with -lgnunettestbed.
1242 1313
1243 1314
1244 1315
1245@c *************************************************************************** 1316@c ***********************************************************************
1246@menu 1317@menu
1247* Supported Topologies:: 1318* Supported Topologies::
1248* Hosts file format:: 1319* Hosts file format::
@@ -1255,56 +1326,60 @@ linking with -lgnunettestbed.
1255@node Supported Topologies 1326@node Supported Topologies
1256@subsection Supported Topologies 1327@subsection Supported Topologies
1257 1328
1258While testing multi-peer deployments, it is often needed that the peers are 1329While testing multi-peer deployments, it is often needed that the peers
1259connected in some topology. This requirement is addressed by the function 1330are connected in some topology. This requirement is addressed by the
1260@code{GNUNET_TESTBED_overlay_connect()} which connects any given two peers in 1331function @code{GNUNET_TESTBED_overlay_connect()} which connects any given
1261the testbed. 1332two peers in the testbed.
1262 1333
1263The API also provides a helper function 1334The API also provides a helper function
1264@code{GNUNET_TESTBED_overlay_configure_topology()} to connect a given set of 1335@code{GNUNET_TESTBED_overlay_configure_topology()} to connect a given set
1265peers in any of the following supported topologies: 1336of peers in any of the following supported topologies:
1337
1266@itemize @bullet 1338@itemize @bullet
1267 1339
1268@item @code{GNUNET_TESTBED_TOPOLOGY_CLIQUE}: All peers are connected with each 1340@item @code{GNUNET_TESTBED_TOPOLOGY_CLIQUE}: All peers are connected with
1269other 1341each other
1270 1342
1271@item @code{GNUNET_TESTBED_TOPOLOGY_LINE}: Peers are connected to form a line 1343@item @code{GNUNET_TESTBED_TOPOLOGY_LINE}: Peers are connected to form a
1344line
1272 1345
1273@item @code{GNUNET_TESTBED_TOPOLOGY_RING}: Peers are connected to form a ring 1346@item @code{GNUNET_TESTBED_TOPOLOGY_RING}: Peers are connected to form a
1274topology 1347ring topology
1275 1348
1276@item @code{GNUNET_TESTBED_TOPOLOGY_2D_TORUS}: Peers are connected to form a 2 1349@item @code{GNUNET_TESTBED_TOPOLOGY_2D_TORUS}: Peers are connected to
1277dimensional torus topology. The number of peers may not be a perfect square, in 1350form a 2 dimensional torus topology. The number of peers may not be a
1278that case the resulting torus may not have the uniform poloidal and toroidal 1351perfect square, in that case the resulting torus may not have the uniform
1279lengths 1352poloidal and toroidal lengths
1280 1353
1281@item @code{GNUNET_TESTBED_TOPOLOGY_ERDOS_RENYI}: Topology is generated to form 1354@item @code{GNUNET_TESTBED_TOPOLOGY_ERDOS_RENYI}: Topology is generated
1282a random graph. The number of links to be present should be given 1355to form a random graph. The number of links to be present should be given
1283 1356
1284@item @code{GNUNET_TESTBED_TOPOLOGY_SMALL_WORLD}: Peers are connected to form a 1357@item @code{GNUNET_TESTBED_TOPOLOGY_SMALL_WORLD}: Peers are connected to
12852D Torus with some random links among them. The number of random links are to 1358form a 2D Torus with some random links among them. The number of random
1286be given 1359links are to be given
1287 1360
1288@item @code{GNUNET_TESTBED_TOPOLOGY_SMALL_WORLD_RING}: Peers are connected to 1361@item @code{GNUNET_TESTBED_TOPOLOGY_SMALL_WORLD_RING}: Peers are
1289form a ring with some random links among them. The number of random links are 1362connected to form a ring with some random links among them. The number of
1290to be given 1363random links are to be given
1291 1364
1292@item @code{GNUNET_TESTBED_TOPOLOGY_SCALE_FREE}: Connects peers in a topology 1365@item @code{GNUNET_TESTBED_TOPOLOGY_SCALE_FREE}: Connects peers in a
1293where peer connectivity follows power law - new peers are connected with high 1366topology where peer connectivity follows power law - new peers are
1294probabililty to well connected peers. See Emergence of Scaling in Random 1367connected with high probabililty to well connected peers.
1295Networks. Science 286, 509-512, 1999. 1368@footnote{See Emergence of Scaling in Random Networks. Science 286,
1369509-512, 1999.}
1296 1370
1297@item @code{GNUNET_TESTBED_TOPOLOGY_FROM_FILE}: The topology information is 1371@item @code{GNUNET_TESTBED_TOPOLOGY_FROM_FILE}: The topology information
1298loaded from a file. The path to the file has to be given. See Topology file 1372is loaded from a file. The path to the file has to be given. See Topology
1299format for the format of this file. 1373file format for the format of this file.
1300 1374
1301@item @code{GNUNET_TESTBED_TOPOLOGY_NONE}: No topology 1375@item @code{GNUNET_TESTBED_TOPOLOGY_NONE}: No topology
1302@end itemize 1376@end itemize
1303 1377
1304 1378
1305The above supported topologies can be specified respectively by setting the 1379The above supported topologies can be specified respectively by setting
1306variable @code{OVERLAY_TOPOLOGY} to the following values in the configuration 1380the variable @code{OVERLAY_TOPOLOGY} to the following values in the
1307passed to Testbed API functions @code{GNUNET_TESTBED_test_run()} and 1381configuration passed to Testbed API functions
1382@code{GNUNET_TESTBED_test_run()} and
1308@code{GNUNET_TESTBED_run()}: 1383@code{GNUNET_TESTBED_run()}:
1309@itemize @bullet 1384@itemize @bullet
1310@item @code{CLIQUE} 1385@item @code{CLIQUE}
@@ -1322,114 +1397,115 @@ passed to Testbed API functions @code{GNUNET_TESTBED_test_run()} and
1322 1397
1323Topologies @code{RANDOM}, @code{SMALL_WORLD} and @code{SMALL_WORLD_RING} 1398Topologies @code{RANDOM}, @code{SMALL_WORLD} and @code{SMALL_WORLD_RING}
1324require the option @code{OVERLAY_RANDOM_LINKS} to be set to the number of 1399require the option @code{OVERLAY_RANDOM_LINKS} to be set to the number of
1325random links to be generated in the configuration. The option will be ignored 1400random links to be generated in the configuration. The option will be
1326for the rest of the topologies. 1401ignored for the rest of the topologies.
1327 1402
1328Topology @code{SCALE_FREE} requires the options @code{SCALE_FREE_TOPOLOGY_CAP} 1403Topology @code{SCALE_FREE} requires the options
1329to be set to the maximum number of peers which can connect to a peer and 1404@code{SCALE_FREE_TOPOLOGY_CAP} to be set to the maximum number of peers
1330@code{SCALE_FREE_TOPOLOGY_M} to be set to how many peers a peer should be 1405which can connect to a peer and @code{SCALE_FREE_TOPOLOGY_M} to be set to
1331atleast connected to. 1406how many peers a peer should be atleast connected to.
1332 1407
1333Similarly, the topology @code{FROM_FILE} requires the option 1408Similarly, the topology @code{FROM_FILE} requires the option
1334@code{OVERLAY_TOPOLOGY_FILE} to contain the path of the file containing the 1409@code{OVERLAY_TOPOLOGY_FILE} to contain the path of the file containing
1335topology information. This option is ignored for the rest of the topologies. 1410the topology information. This option is ignored for the rest of the
1336See Topology file format for the format of this file. 1411topologies. See Topology file format for the format of this file.
1337 1412
1338@c *************************************************************************** 1413@c ***********************************************************************
1339@node Hosts file format 1414@node Hosts file format
1340@subsection Hosts file format 1415@subsection Hosts file format
1341 1416
1342The testbed API offers the function GNUNET_TESTBED_hosts_load_from_file() to 1417The testbed API offers the function GNUNET_TESTBED_hosts_load_from_file()
1343load from a given file details about the hosts which testbed can use for 1418to load from a given file details about the hosts which testbed can use
1344deploying peers. This function is useful to keep the data about hosts separate 1419for deploying peers. This function is useful to keep the data about hosts
1345instead of hard coding them in code. 1420separate instead of hard coding them in code.
1346 1421
1347Another helper function from testbed API, GNUNET_TESTBED_run() also takes a 1422Another helper function from testbed API, GNUNET_TESTBED_run() also takes
1348hosts file name as its parameter. It uses the above function to populate the 1423a hosts file name as its parameter. It uses the above function to
1349hosts data structures and start controllers to deploy peers. 1424populate the hosts data structures and start controllers to deploy peers.
1350 1425
1351These functions require the hosts file to be of the following format: 1426These functions require the hosts file to be of the following format:
1352@itemize @bullet 1427@itemize @bullet
1353@item Each line is interpreted to have details about a host 1428@item Each line is interpreted to have details about a host
1354@item Host details should include the username to use for logging into the 1429@item Host details should include the username to use for logging into the
1355host, the hostname of the host and the port number to use for the remote shell 1430host, the hostname of the host and the port number to use for the remote
1356program. All thee values should be given. 1431shell program. All thee values should be given.
1357@item These details should be given in the following format: 1432@item These details should be given in the following format:
1358@code{<username>@@<hostname>:<port>} 1433@code{<username>@@<hostname>:<port>}
1359@end itemize 1434@end itemize
1360 1435
1361Note that having canonical hostnames may cause problems while resolving the IP 1436Note that having canonical hostnames may cause problems while resolving
1362addresses (See this bug). Hence it is advised to provide the hosts' IP 1437the IP addresses (See this bug). Hence it is advised to provide the hosts'
1363numerical addresses as hostnames whenever possible. 1438IP numerical addresses as hostnames whenever possible.
1364 1439
1365@c *************************************************************************** 1440@c ***********************************************************************
1366@node Topology file format 1441@node Topology file format
1367@subsection Topology file format 1442@subsection Topology file format
1368 1443
1369A topology file describes how peers are to be connected. It should adhere to 1444A topology file describes how peers are to be connected. It should adhere
1370the following format for testbed to parse it correctly. 1445to the following format for testbed to parse it correctly.
1371 1446
1372Each line should begin with the target peer id. This should be followed by a 1447Each line should begin with the target peer id. This should be followed by
1373colon(`:') and origin peer ids seperated by `|'. All spaces except for newline 1448a colon(`:') and origin peer ids seperated by `|'. All spaces except for
1374characters are ignored. The API will then try to connect each origin peer to 1449newline characters are ignored. The API will then try to connect each
1375the target peer. 1450origin peer to the target peer.
1376 1451
1377For example, the following file will result in 5 overlay connections: [2->1], 1452For example, the following file will result in 5 overlay connections:
1378[3->1],[4->3], [0->3], [2->0]@ @code{@ 1:2|3@ 3:4| 0@ 0: 2@ } 1453[2->1], [3->1],[4->3], [0->3], [2->0]@ @code{@ 1:2|3@ 3:4| 0@ 0: 2@ }
1379 1454
1380@c *************************************************************************** 1455@c ***********************************************************************
1381@node Testbed Barriers 1456@node Testbed Barriers
1382@subsection Testbed Barriers 1457@subsection Testbed Barriers
1383 1458
1384The testbed subsystem's barriers API facilitates coordination among the peers 1459The testbed subsystem's barriers API facilitates coordination among the
1385run by the testbed and the experiment driver. The concept is similar to the 1460peers run by the testbed and the experiment driver. The concept is
1386barrier synchronisation mechanism found in parallel programming or 1461similar to the barrier synchronisation mechanism found in parallel
1387multi-threading paradigms - a peer waits at a barrier upon reaching it until 1462programming or multi-threading paradigms - a peer waits at a barrier upon
1388the barrier is reached by a predefined number of peers. This predefined number 1463reaching it until the barrier is reached by a predefined number of peers.
1389of peers required to cross a barrier is also called quorum. We say a peer has 1464This predefined number of peers required to cross a barrier is also called
1390reached a barrier if the peer is waiting for the barrier to be crossed. 1465quorum. We say a peer has reached a barrier if the peer is waiting for the
1391Similarly a barrier is said to be reached if the required quorum of peers reach 1466barrier to be crossed. Similarly a barrier is said to be reached if the
1392the barrier. A barrier which is reached is deemed as crossed after all the 1467required quorum of peers reach the barrier. A barrier which is reached is
1393peers waiting on it are notified. 1468deemed as crossed after all the peers waiting on it are notified.
1394 1469
1395The barriers API provides the following functions: 1470The barriers API provides the following functions:
1396@itemize @bullet 1471@itemize @bullet
1397@item @strong{@code{GNUNET_TESTBED_barrier_init()}:} function to initialse a 1472@item @strong{@code{GNUNET_TESTBED_barrier_init()}:} function to
1398barrier in the experiment 1473initialse a barrier in the experiment
1399@item @strong{@code{GNUNET_TESTBED_barrier_cancel()}:} function to cancel a 1474@item @strong{@code{GNUNET_TESTBED_barrier_cancel()}:} function to cancel
1400barrier which has been initialised before 1475a barrier which has been initialised before
1401@item @strong{@code{GNUNET_TESTBED_barrier_wait()}:} function to signal barrier 1476@item @strong{@code{GNUNET_TESTBED_barrier_wait()}:} function to signal
1402service that the caller has reached a barrier and is waiting for it to be 1477barrier service that the caller has reached a barrier and is waiting for
1403crossed 1478it to be crossed
1404@item @strong{@code{GNUNET_TESTBED_barrier_wait_cancel()}:} function to stop 1479@item @strong{@code{GNUNET_TESTBED_barrier_wait_cancel()}:} function to
1405waiting for a barrier to be crossed 1480stop waiting for a barrier to be crossed
1406@end itemize 1481@end itemize
1407 1482
1408 1483
1409Among the above functions, the first two, namely 1484Among the above functions, the first two, namely
1410@code{GNUNET_TESTBED_barrier_init()} and @code{GNUNET_TESTBED_barrier_cancel()} 1485@code{GNUNET_TESTBED_barrier_init()} and
1411are used by experiment drivers. All barriers should be initialised by the 1486@code{GNUNET_TESTBED_barrier_cancel()} are used by experiment drivers. All
1412experiment driver by calling @code{GNUNET_TESTBED_barrier_init()}. This 1487barriers should be initialised by the experiment driver by calling
1413function takes a name to identify the barrier, the quorum required for the 1488@code{GNUNET_TESTBED_barrier_init()}. This function takes a name to
1414barrier to be crossed and a notification callback for notifying the experiment 1489identify the barrier, the quorum required for the barrier to be crossed
1415driver when the barrier is crossed. @code{GNUNET_TESTBED_barrier_cancel()} 1490and a notification callback for notifying the experiment driver when the
1416cancels an initialised barrier and frees the resources allocated for it. This 1491barrier is crossed. @code{GNUNET_TESTBED_barrier_cancel()} cancels an
1492initialised barrier and frees the resources allocated for it. This
1417function can be called upon a initialised barrier before it is crossed. 1493function can be called upon a initialised barrier before it is crossed.
1418 1494
1419The remaining two functions @code{GNUNET_TESTBED_barrier_wait()} and 1495The remaining two functions @code{GNUNET_TESTBED_barrier_wait()} and
1420@code{GNUNET_TESTBED_barrier_wait_cancel()} are used in the peer's processes. 1496@code{GNUNET_TESTBED_barrier_wait_cancel()} are used in the peer's
1421@code{GNUNET_TESTBED_barrier_wait()} connects to the local barrier service 1497processes. @code{GNUNET_TESTBED_barrier_wait()} connects to the local
1422running on the same host the peer is running on and registers that the caller 1498barrier service running on the same host the peer is running on and
1423has reached the barrier and is waiting for the barrier to be crossed. Note that 1499registers that the caller has reached the barrier and is waiting for the
1424this function can only be used by peers which are started by testbed as this 1500barrier to be crossed. Note that this function can only be used by peers
1425function tries to access the local barrier service which is part of the testbed 1501which are started by testbed as this function tries to access the local
1426controller service. Calling @code{GNUNET_TESTBED_barrier_wait()} on an 1502barrier service which is part of the testbed controller service. Calling
1427uninitialised barrier results in failure. 1503@code{GNUNET_TESTBED_barrier_wait()} on an uninitialised barrier results
1428@code{GNUNET_TESTBED_barrier_wait_cancel()} cancels the notification registered 1504in failure. @code{GNUNET_TESTBED_barrier_wait_cancel()} cancels the
1429by @code{GNUNET_TESTBED_barrier_wait()}. 1505notification registered by @code{GNUNET_TESTBED_barrier_wait()}.
1430 1506
1431 1507
1432@c *************************************************************************** 1508@c ***********************************************************************
1433@menu 1509@menu
1434* Implementation:: 1510* Implementation::
1435@end menu 1511@end menu
@@ -1437,10 +1513,11 @@ by @code{GNUNET_TESTBED_barrier_wait()}.
1437@node Implementation 1513@node Implementation
1438@subsubsection Implementation 1514@subsubsection Implementation
1439 1515
1440Since barriers involve coordination between experiment driver and peers, the 1516Since barriers involve coordination between experiment driver and peers,
1441barrier service in the testbed controller is split into two components. The 1517the barrier service in the testbed controller is split into two
1442first component responds to the message generated by the barrier API used by 1518components. The first component responds to the message generated by the
1443the experiment driver (functions @code{GNUNET_TESTBED_barrier_init()} and 1519barrier API used by the experiment driver (functions
1520@code{GNUNET_TESTBED_barrier_init()} and
1444@code{GNUNET_TESTBED_barrier_cancel()}) and the second component to the 1521@code{GNUNET_TESTBED_barrier_cancel()}) and the second component to the
1445messages generated by barrier API used by peers (functions 1522messages generated by barrier API used by peers (functions
1446@code{GNUNET_TESTBED_barrier_wait()} and 1523@code{GNUNET_TESTBED_barrier_wait()} and
@@ -1449,11 +1526,11 @@ messages generated by barrier API used by peers (functions
1449Calling @code{GNUNET_TESTBED_barrier_init()} sends a 1526Calling @code{GNUNET_TESTBED_barrier_init()} sends a
1450@code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_INIT} message to the master 1527@code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_INIT} message to the master
1451controller. The master controller then registers a barrier and calls 1528controller. The master controller then registers a barrier and calls
1452@code{GNUNET_TESTBED_barrier_init()} for each its subcontrollers. In this way 1529@code{GNUNET_TESTBED_barrier_init()} for each its subcontrollers. In this
1453barrier initialisation is propagated to the controller hierarchy. While 1530way barrier initialisation is propagated to the controller hierarchy.
1454propagating initialisation, any errors at a subcontroller such as timeout 1531While propagating initialisation, any errors at a subcontroller such as
1455during further propagation are reported up the hierarchy back to the experiment 1532timeout during further propagation are reported up the hierarchy back to
1456driver. 1533the experiment driver.
1457 1534
1458Similar to @code{GNUNET_TESTBED_barrier_init()}, 1535Similar to @code{GNUNET_TESTBED_barrier_init()},
1459@code{GNUNET_TESTBED_barrier_cancel()} propagates 1536@code{GNUNET_TESTBED_barrier_cancel()} propagates
@@ -1498,7 +1575,7 @@ propagation --- the upward propagation is needed for ensuring that the barrier
1498is reached by all the controllers and the downward propagation is for 1575is reached by all the controllers and the downward propagation is for
1499triggering that the barrier is crossed. 1576triggering that the barrier is crossed.
1500 1577
1501@c *************************************************************************** 1578@c ***********************************************************************
1502@node Automatic large-scale deployment of GNUnet in the PlanetLab testbed 1579@node Automatic large-scale deployment of GNUnet in the PlanetLab testbed
1503@subsection Automatic large-scale deployment of GNUnet in the PlanetLab testbed 1580@subsection Automatic large-scale deployment of GNUnet in the PlanetLab testbed
1504 1581
@@ -1515,7 +1592,7 @@ Please also check @uref{https://gnunet.org/installation-fedora8-svn} and@
1515instructions how to install GNUnet on a PlanetLab node. 1592instructions how to install GNUnet on a PlanetLab node.
1516 1593
1517 1594
1518@c *************************************************************************** 1595@c ***********************************************************************
1519@menu 1596@menu
1520* PlanetLab Automation for Fedora8 nodes:: 1597* PlanetLab Automation for Fedora8 nodes::
1521* Install buildslave on PlanetLab nodes running fedora core 8:: 1598* Install buildslave on PlanetLab nodes running fedora core 8::
@@ -1526,7 +1603,7 @@ instructions how to install GNUnet on a PlanetLab node.
1526@node PlanetLab Automation for Fedora8 nodes 1603@node PlanetLab Automation for Fedora8 nodes
1527@subsubsection PlanetLab Automation for Fedora8 nodes 1604@subsubsection PlanetLab Automation for Fedora8 nodes
1528 1605
1529@c *************************************************************************** 1606@c ***********************************************************************
1530@node Install buildslave on PlanetLab nodes running fedora core 8 1607@node Install buildslave on PlanetLab nodes running fedora core 8
1531@subsubsection Install buildslave on PlanetLab nodes running fedora core 8 1608@subsubsection Install buildslave on PlanetLab nodes running fedora core 8
1532@c ** Actually this is a subsubsubsection, but must be fixed differently 1609@c ** Actually this is a subsubsubsection, but must be fixed differently
@@ -1554,7 +1631,7 @@ The setup will download the matching twisted package and install it.@ It will
1554also try to install the latest version of zope.interface which will fail to 1631also try to install the latest version of zope.interface which will fail to
1555install. Buildslave will work anyway since version 3.8.0 was installed before! 1632install. Buildslave will work anyway since version 3.8.0 was installed before!
1556 1633
1557@c *************************************************************************** 1634@c ***********************************************************************
1558@node Setup a new PlanetLab testbed using GPLMT 1635@node Setup a new PlanetLab testbed using GPLMT
1559@subsubsection Setup a new PlanetLab testbed using GPLMT 1636@subsubsection Setup a new PlanetLab testbed using GPLMT
1560 1637
@@ -1600,7 +1677,7 @@ Use @code{buildbot start <basedir>} to start the server
1600@item Setup the buildslaves 1677@item Setup the buildslaves
1601@end itemize 1678@end itemize
1602 1679
1603@c *************************************************************************** 1680@c ***********************************************************************
1604@node Why do i get an ssh error when using the regex profiler? 1681@node Why do i get an ssh error when using the regex profiler?
1605@subsubsection Why do i get an ssh error when using the regex profiler? 1682@subsubsection Why do i get an ssh error when using the regex profiler?
1606 1683
@@ -1623,7 +1700,7 @@ You can test your setup by running `ssh 127.0.0.1` in a terminal and then in
1623the opened session run it again. If you were not asked for a password on either 1700the opened session run it again. If you were not asked for a password on either
1624login, then you should be good to go. 1701login, then you should be good to go.
1625 1702
1626@c *************************************************************************** 1703@c ***********************************************************************
1627@node TESTBED Caveats 1704@node TESTBED Caveats
1628@subsection TESTBED Caveats 1705@subsection TESTBED Caveats
1629 1706
@@ -1631,7 +1708,7 @@ This section documents a few caveats when using the GNUnet testbed
1631subsystem. 1708subsystem.
1632 1709
1633 1710
1634@c *************************************************************************** 1711@c ***********************************************************************
1635@menu 1712@menu
1636* CORE must be started:: 1713* CORE must be started::
1637* ATS must want the connections:: 1714* ATS must want the connections::
@@ -1650,7 +1727,7 @@ indirectly depends on CORE being started with FORCESTART will also do. This
1650issue largely arises if users try to over-optimize by not starting any services 1727issue largely arises if users try to over-optimize by not starting any services
1651with FORCESTART. 1728with FORCESTART.
1652 1729
1653@c *************************************************************************** 1730@c ***********************************************************************
1654@node ATS must want the connections 1731@node ATS must want the connections
1655@subsubsection ATS must want the connections 1732@subsubsection ATS must want the connections
1656 1733
@@ -1666,7 +1743,7 @@ of connection failures (see '-e' option of gnunet-testbed-profiler). This issue
1666largely arises for dense overlay topologies, especially if you try to create 1743largely arises for dense overlay topologies, especially if you try to create
1667cliques with more than 20 peers. 1744cliques with more than 20 peers.
1668 1745
1669@c *************************************************************************** 1746@c ***********************************************************************
1670@node libgnunetutil 1747@node libgnunetutil
1671@section libgnunetutil 1748@section libgnunetutil
1672 1749
@@ -1717,7 +1794,7 @@ porting of GNUnet easier.
1717* The CONTAINER_MDLL API:: 1794* The CONTAINER_MDLL API::
1718@end menu 1795@end menu
1719 1796
1720@c *************************************************************************** 1797@c ***********************************************************************
1721@node Logging 1798@node Logging
1722@subsection Logging 1799@subsection Logging
1723 1800
@@ -1877,7 +1954,7 @@ an error in definition formatting or an error in regular expression syntax, and
1877will not report the failure in any way. 1954will not report the failure in any way.
1878 1955
1879 1956
1880@c *************************************************************************** 1957@c ***********************************************************************
1881@menu 1958@menu
1882* Examples:: 1959* Examples::
1883* Log files:: 1960* Log files::
@@ -1924,7 +2001,7 @@ limitation, on Windows, GNUNET_FORCE_LOGFILE @strong{MUST} be set in order to
1924GNUNET_FORCE_LOG to work. 2001GNUNET_FORCE_LOG to work.
1925 2002
1926 2003
1927@c *************************************************************************** 2004@c ***********************************************************************
1928@node Log files 2005@node Log files
1929@subsubsection Log files 2006@subsubsection Log files
1930 2007
@@ -1962,7 +2039,7 @@ begins. If you do not use any date-related string format codes, logs would
1962never be automatically deleted by GNUnet. 2039never be automatically deleted by GNUnet.
1963 2040
1964 2041
1965@c *************************************************************************** 2042@c ***********************************************************************
1966 2043
1967@node Updated behavior of GNUNET_log 2044@node Updated behavior of GNUNET_log
1968@subsubsection Updated behavior of GNUNET_log 2045@subsubsection Updated behavior of GNUNET_log
@@ -2032,7 +2109,7 @@ GNUNET_FORCE_LOG="^YOUR_SUBSYSTEM$;;;;DEBUG/;;;;WARNING"}@ Which will behave
2032almost like enabling DEBUG in that subsytem before the change. Of course you 2109almost like enabling DEBUG in that subsytem before the change. Of course you
2033can adapt it to your particular needs, this is only a quick example. 2110can adapt it to your particular needs, this is only a quick example.
2034 2111
2035@c *************************************************************************** 2112@c ***********************************************************************
2036@node Interprocess communication API (IPC) 2113@node Interprocess communication API (IPC)
2037@subsection Interprocess communication API (IPC) 2114@subsection Interprocess communication API (IPC)
2038 2115
@@ -2045,7 +2122,7 @@ AddressLookupMessage} as a request to ask the server to return the address of
2045any other peer connecting to the service.) 2122any other peer connecting to the service.)
2046 2123
2047 2124
2048@c *************************************************************************** 2125@c ***********************************************************************
2049@menu 2126@menu
2050* Define new message types:: 2127* Define new message types::
2051* Define message struct:: 2128* Define message struct::
@@ -2072,7 +2149,7 @@ First of all, you should define the new message type in
2072#define GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_REPLY 30 2149#define GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_REPLY 30
2073@end example 2150@end example
2074 2151
2075@c *************************************************************************** 2152@c ***********************************************************************
2076@node Define message struct 2153@node Define message struct
2077@subsubsection Define message struct 2154@subsubsection Define message struct
2078 2155
@@ -2096,7 +2173,7 @@ both ensure correct alignment when sending structs over the network
2096@menu 2173@menu
2097@end menu 2174@end menu
2098 2175
2099@c *************************************************************************** 2176@c ***********************************************************************
2100@node Client - Establish connection 2177@node Client - Establish connection
2101@subsubsection Client - Establish connection 2178@subsubsection Client - Establish connection
2102@c %**end of header 2179@c %**end of header
@@ -2110,7 +2187,7 @@ struct GNUNET_CLIENT_Connection *client; client =
2110GNUNET_CLIENT_connect ("transport", cfg); 2187GNUNET_CLIENT_connect ("transport", cfg);
2111@end example 2188@end example
2112 2189
2113@c *************************************************************************** 2190@c ***********************************************************************
2114@node Client - Initialize request message 2191@node Client - Initialize request message
2115@subsubsection Client - Initialize request message 2192@subsubsection Client - Initialize request message
2116@c %**end of header 2193@c %**end of header
@@ -2135,7 +2212,7 @@ endian, about the usage of the big/small edian order and the corresponding
2135conversion function please refer to Introduction of Big Endian and Little 2212conversion function please refer to Introduction of Big Endian and Little
2136Endian. 2213Endian.
2137 2214
2138@c *************************************************************************** 2215@c ***********************************************************************
2139@node Client - Send request and receive response 2216@node Client - Send request and receive response
2140@subsubsection Client - Send request and receive response 2217@subsubsection Client - Send request and receive response
2141@c %**end of header 2218@c %**end of header
@@ -2169,7 +2246,7 @@ argc, char**argv) @{ GNUNET_SERVICE_run(argc, argv, "transport"
2169GNUNET_SERVICE_OPTION_NONE, &run, NULL)); @} 2246GNUNET_SERVICE_OPTION_NONE, &run, NULL)); @}
2170@end example 2247@end example
2171 2248
2172@c *************************************************************************** 2249@c ***********************************************************************
2173@node Server - Add new handles for specified messages 2250@node Server - Add new handles for specified messages
2174@subsubsection Server - Add new handles for specified messages 2251@subsubsection Server - Add new handles for specified messages
2175@c %**end of header 2252@c %**end of header
@@ -2207,7 +2284,7 @@ variable size, for special cases the exact size of the specified message also
2207can be set. In addition, the terminator sign depicted as @code{@{NULL, NULL, 0, 2284can be set. In addition, the terminator sign depicted as @code{@{NULL, NULL, 0,
22080@}} is set in the last aera. 22850@}} is set in the last aera.
2209 2286
2210@c *************************************************************************** 2287@c ***********************************************************************
2211@node Server - Process request message 2288@node Server - Process request message
2212@subsubsection Server - Process request message 2289@subsubsection Server - Process request message
2213@c %**end of header 2290@c %**end of header
@@ -2248,7 +2325,7 @@ request message, and the processing of this request would be terminated.
2248In comparison to the aforementioned situation, when the argument is equal to 2325In comparison to the aforementioned situation, when the argument is equal to
2249@code{GNUNET_OK}, the service would continue to process the requst message. 2326@code{GNUNET_OK}, the service would continue to process the requst message.
2250 2327
2251@c *************************************************************************** 2328@c ***********************************************************************
2252@node Server - Response to client 2329@node Server - Response to client
2253@subsubsection Server - Response to client 2330@subsubsection Server - Response to client
2254@c %**end of header 2331@c %**end of header
@@ -2278,7 +2355,7 @@ GNUNET_SERVER_transmit_context_run (tc, rtimeout);
2278Note that, there are also a number of other APIs provided to the service to 2355Note that, there are also a number of other APIs provided to the service to
2279send the message. 2356send the message.
2280 2357
2281@c *************************************************************************** 2358@c ***********************************************************************
2282@node Server - Notification of clients 2359@node Server - Notification of clients
2283@subsubsection Server - Notification of clients 2360@subsubsection Server - Notification of clients
2284@c %**end of header 2361@c %**end of header
@@ -2302,7 +2379,7 @@ the server code still typically needs to call@
2302@code{GNUNET_SERVER_receive_done} so that the client can transmit further 2379@code{GNUNET_SERVER_receive_done} so that the client can transmit further
2303messages to the server. 2380messages to the server.
2304 2381
2305@c *************************************************************************** 2382@c ***********************************************************************
2306@node Conversion between Network Byte Order (Big Endian) and Host Byte Order 2383@node Conversion between Network Byte Order (Big Endian) and Host Byte Order
2307@subsubsection Conversion between Network Byte Order (Big Endian) and Host Byte Order 2384@subsubsection Conversion between Network Byte Order (Big Endian) and Host Byte Order
2308@c %** subsub? it's a referenced page on the ipc document. 2385@c %** subsub? it's a referenced page on the ipc document.
@@ -2357,7 +2434,7 @@ GNUNET_TIME_absolute_ntoh (struct GNUNET_TIME_AbsoluteNBO a) Convert relative
2357time from network byte order. 2434time from network byte order.
2358@end table 2435@end table
2359 2436
2360@c *************************************************************************** 2437@c ***********************************************************************
2361 2438
2362@node Cryptography API 2439@node Cryptography API
2363@subsection Cryptography API 2440@subsection Cryptography API
@@ -2394,7 +2471,7 @@ used by most applications; most importantly,@
2394GNUNET_CRYPTO_rsa_key_create_from_hash does not create an RSA-key that should 2471GNUNET_CRYPTO_rsa_key_create_from_hash does not create an RSA-key that should
2395be considered secure for traditional applications of RSA. 2472be considered secure for traditional applications of RSA.
2396 2473
2397@c *************************************************************************** 2474@c ***********************************************************************
2398@node Message Queue API 2475@node Message Queue API
2399@subsection Message Queue API 2476@subsection Message Queue API
2400@c %**end of header 2477@c %**end of header
@@ -2518,7 +2595,7 @@ callback. When canceling an envelope, it is not necessary@ to call
2518 2595
2519@strong{ Implementing Queues }@ @code{TODO} 2596@strong{ Implementing Queues }@ @code{TODO}
2520 2597
2521@c *************************************************************************** 2598@c ***********************************************************************
2522@node Service API 2599@node Service API
2523@subsection Service API 2600@subsection Service API
2524@c %**end of header 2601@c %**end of header
@@ -2584,7 +2661,7 @@ allowing existing 'normal' clients to set (possibly persistent) statistic
2584values before terminating. 2661values before terminating.
2585@end table 2662@end table
2586 2663
2587@c *************************************************************************** 2664@c ***********************************************************************
2588@node Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps 2665@node Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps
2589@subsection Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps 2666@subsection Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps
2590@c %**end of header 2667@c %**end of header
@@ -2599,7 +2676,7 @@ API quirks (and their implications for applications) that were recently
2599introduced to minimize the footprint of the hash map. 2676introduced to minimize the footprint of the hash map.
2600 2677
2601 2678
2602@c *************************************************************************** 2679@c ***********************************************************************
2603@menu 2680@menu
2604* Analysis:: 2681* Analysis::
2605* Solution:: 2682* Solution::
@@ -2652,7 +2729,7 @@ just an extreme example: overheads in practice are actually sometimes close to
2652those highlighted in this example. This is especially true for maps with a 2729those highlighted in this example. This is especially true for maps with a
2653significant number of entries, as there we tend to really try to keep the 2730significant number of entries, as there we tend to really try to keep the
2654entries small. 2731entries small.
2655@c *************************************************************************** 2732@c ***********************************************************************
2656@node Solution 2733@node Solution
2657@subsubsection Solution 2734@subsubsection Solution
2658@c %**end of header 2735@c %**end of header
@@ -2668,7 +2745,7 @@ stores an entry in the hash map, it @strong{must} provide a pointer to the key
2668within the entry, not just a pointer to a transient location of the key. If 2745within the entry, not just a pointer to a transient location of the key. If
2669the client code does not meet these requirements, the result is a dangling 2746the client code does not meet these requirements, the result is a dangling
2670pointer and undefined behavior of the (multi-)hash map API. 2747pointer and undefined behavior of the (multi-)hash map API.
2671@c *************************************************************************** 2748@c ***********************************************************************
2672@node Migration 2749@node Migration
2673@subsubsection Migration 2750@subsubsection Migration
2674@c %**end of header 2751@c %**end of header
@@ -2711,7 +2788,7 @@ If everything was done correctly, you now use about 60 bytes less memory per
2711entry in @code{map}. However, if now (or in the future) any call to @code{put} 2788entry in @code{map}. However, if now (or in the future) any call to @code{put}
2712does not ensure that the given key is valid until the entry is removed from the 2789does not ensure that the given key is valid until the entry is removed from the
2713map, undefined behavior is likely to be observed. 2790map, undefined behavior is likely to be observed.
2714@c *************************************************************************** 2791@c ***********************************************************************
2715@node Conclusion 2792@node Conclusion
2716@subsubsection Conclusion 2793@subsubsection Conclusion
2717@c %**end of header 2794@c %**end of header
@@ -2722,7 +2799,7 @@ robust as additional invariants are imposed on the multi hash map client. Thus
2722applications should refrain from enabling the new mode unless the resulting 2799applications should refrain from enabling the new mode unless the resulting
2723performance increase is deemed significant enough. In particular, it should 2800performance increase is deemed significant enough. In particular, it should
2724generally not be used in new code (wait at least until benchmarks exist). 2801generally not be used in new code (wait at least until benchmarks exist).
2725@c *************************************************************************** 2802@c ***********************************************************************
2726@node Availability 2803@node Availability
2727@subsubsection Availability 2804@subsubsection Availability
2728@c %**end of header 2805@c %**end of header
@@ -2733,7 +2810,7 @@ previously audited and modified to take advantage of the new capability. In
2733particular, memory consumption of the file-sharing service is expected to drop 2810particular, memory consumption of the file-sharing service is expected to drop
2734by 20-30% due to this change. 2811by 20-30% due to this change.
2735 2812
2736@c *************************************************************************** 2813@c ***********************************************************************
2737@node The CONTAINER_MDLL API 2814@node The CONTAINER_MDLL API
2738@subsection The CONTAINER_MDLL API 2815@subsection The CONTAINER_MDLL API
2739@c %**end of header 2816@c %**end of header
@@ -2789,7 +2866,7 @@ functions for inserting (at head, at tail, after a given element) and removing
2789elements from the list. Iterating over the list should be done by directly 2866elements from the list. Iterating over the list should be done by directly
2790accessing the "next_XX" and/or "prev_XX" members. 2867accessing the "next_XX" and/or "prev_XX" members.
2791 2868
2792@c *************************************************************************** 2869@c ***********************************************************************
2793@node The Automatic Restart Manager (ARM) 2870@node The Automatic Restart Manager (ARM)
2794@section The Automatic Restart Manager (ARM) 2871@section The Automatic Restart Manager (ARM)
2795@c %**end of header 2872@c %**end of header
@@ -2810,7 +2887,7 @@ with it.
2810* Reliability:: 2887* Reliability::
2811@end menu 2888@end menu
2812 2889
2813@c *************************************************************************** 2890@c ***********************************************************************
2814@node Basic functionality 2891@node Basic functionality
2815@subsection Basic functionality 2892@subsection Basic functionality
2816@c %**end of header 2893@c %**end of header
@@ -2833,7 +2910,7 @@ test_arm_api.c. The test case connects to ARM, starts it, then uses it to start
2833a service "resolver", stops the "resolver" then stops "ARM". 2910a service "resolver", stops the "resolver" then stops "ARM".
2834@end itemize 2911@end itemize
2835 2912
2836@c *************************************************************************** 2913@c ***********************************************************************
2837@node Key configuration options 2914@node Key configuration options
2838@subsection Key configuration options 2915@subsection Key configuration options
2839@c %**end of header 2916@c %**end of header
@@ -2895,7 +2972,7 @@ that are going to run.@
2895 2972
2896@end table 2973@end table
2897 2974
2898@c *************************************************************************** 2975@c ***********************************************************************
2899@node Availability2 2976@node Availability2
2900@subsection Availability2 2977@subsection Availability2
2901@c %**end of header 2978@c %**end of header
@@ -2934,7 +3011,7 @@ work).
2934@item Other client services now can directly connect directly to the "server". 3011@item Other client services now can directly connect directly to the "server".
2935@end itemize 3012@end itemize
2936 3013
2937@c *************************************************************************** 3014@c ***********************************************************************
2938@node Reliability 3015@node Reliability
2939@subsection Reliability 3016@subsection Reliability
2940 3017
@@ -2975,7 +3052,7 @@ backoff(S) will remain half an hour, hence ARM won't be busy for a lot of time
2975trying to restart a problematic service. 3052trying to restart a problematic service.
2976@end itemize 3053@end itemize
2977 3054
2978@c *************************************************************************** 3055@c ***********************************************************************
2979@node GNUnet's TRANSPORT Subsystem 3056@node GNUnet's TRANSPORT Subsystem
2980@section GNUnet's TRANSPORT Subsystem 3057@section GNUnet's TRANSPORT Subsystem
2981@c %**end of header 3058@c %**end of header