diff options
Diffstat (limited to 'doc/tutorial/gnunet-tutorial.texi')
-rw-r--r-- | doc/tutorial/gnunet-tutorial.texi | 1568 |
1 files changed, 1568 insertions, 0 deletions
diff --git a/doc/tutorial/gnunet-tutorial.texi b/doc/tutorial/gnunet-tutorial.texi new file mode 100644 index 000000000..4f685668d --- /dev/null +++ b/doc/tutorial/gnunet-tutorial.texi | |||
@@ -0,0 +1,1568 @@ | |||
1 | \input texinfo | ||
2 | @c %**start of header | ||
3 | @setfilename gnunet-tutorial.info | ||
4 | @documentencoding UTF-8 | ||
5 | @settitle GNUnet Tutorial | ||
6 | @c @exampleindent 2 | ||
7 | @c %**end of header | ||
8 | |||
9 | @include version.texi | ||
10 | |||
11 | @copying | ||
12 | Copyright @copyright{} 2001-2018 GNUnet e.V. | ||
13 | |||
14 | Permission is granted to copy, distribute and/or modify this document | ||
15 | under the terms of the GNU Free Documentation License, Version 1.3 or | ||
16 | any later version published by the Free Software Foundation; with no | ||
17 | Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A | ||
18 | copy of the license is included in the section entitled ``GNU Free | ||
19 | Documentation License''. | ||
20 | |||
21 | A copy of the license is also available from the Free Software | ||
22 | Foundation Web site at @url{http://www.gnu.org/licenses/fdl.html}. | ||
23 | |||
24 | Alternately, this document is also available under the General | ||
25 | Public License, version 3 or later, as published by the Free Software | ||
26 | Foundation. A copy of the license is included in the section entitled | ||
27 | ``GNU General Public License''. | ||
28 | |||
29 | A copy of the license is also available from the Free Software | ||
30 | Foundation Web site at @url{http://www.gnu.org/licenses/gpl.html}. | ||
31 | @end copying | ||
32 | |||
33 | @dircategory Tutorial | ||
34 | @direntry | ||
35 | * GNUnet-C-Tutorial: (gnunet-c-tutorial). C Tutorial for GNunet | ||
36 | @end direntry | ||
37 | |||
38 | |||
39 | @titlepage | ||
40 | @title GNUnet C Tutorial | ||
41 | @subtitle A Tutorial for GNUnet @value{VERSION} (C version) | ||
42 | @author The GNUnet Developers | ||
43 | |||
44 | @page | ||
45 | @vskip 0pt plus 1filll | ||
46 | |||
47 | @insertcopying | ||
48 | @end titlepage | ||
49 | |||
50 | @contents | ||
51 | |||
52 | @c **** TODO | ||
53 | @c 1. Update content? | ||
54 | @c 2. Either reference main documentation or | ||
55 | @c 3. Merge this into main documentation | ||
56 | |||
57 | @node Top | ||
58 | @top Introduction | ||
59 | |||
60 | This tutorials explains how to install GNUnet on a | ||
61 | GNU/Linux system and gives an introduction on how | ||
62 | GNUnet can be used to develop a Peer-to-Peer application. | ||
63 | Detailed installation instructions for | ||
64 | various operating systems and a detailed list of all | ||
65 | dependencies can be found on our website at | ||
66 | @uref{https://gnunet.org/installation} and in our | ||
67 | Reference Documentation (GNUnet Handbook). | ||
68 | |||
69 | Please read this tutorial carefully since every single step is | ||
70 | important, and do not hesitate to contact the GNUnet team if you have | ||
71 | any questions or problems! Visit this link in your webbrowser to learn | ||
72 | how to contact the GNUnet team: | ||
73 | @uref{https://gnunet.org/contact_information} | ||
74 | |||
75 | @menu | ||
76 | |||
77 | * Installing GNUnet:: Installing GNUnet | ||
78 | * Introduction to GNUnet Architecture:: Introduction to GNUnet Architecture | ||
79 | * First Steps with GNUnet:: First Steps with GNUnet | ||
80 | * Developing Applications:: Developing Applications | ||
81 | * GNU Free Documentation License:: The license of this manual | ||
82 | |||
83 | @detailmenu | ||
84 | --- The Detailed Node Listing --- | ||
85 | |||
86 | Installing GNUnet | ||
87 | |||
88 | * Obtaining a stable version:: | ||
89 | * Installing Build Tool Chain and Dependencies:: | ||
90 | * Obtaining the latest version from Git:: | ||
91 | * Compiling and Installing GNUnet:: | ||
92 | * Common Issues - Check your GNUnet installation:: | ||
93 | |||
94 | Introduction to GNUnet Architecture | ||
95 | |||
96 | First Steps with GNUnet | ||
97 | |||
98 | * Configure your peer:: | ||
99 | * Start a peer:: | ||
100 | * Monitor a peer:: | ||
101 | * Starting Two Peers by Hand:: | ||
102 | * Starting Peers Using the Testbed Service:: | ||
103 | |||
104 | Developing Applications | ||
105 | |||
106 | * gnunet-ext:: | ||
107 | * Adapting the Template:: | ||
108 | * Writing a Client Application:: | ||
109 | * Writing a Service:: | ||
110 | * Interacting directly with other Peers using the CORE Service:: | ||
111 | * Storing peer-specific data using the PEERSTORE service:: | ||
112 | * Using the DHT:: | ||
113 | * Debugging with gnunet-arm:: | ||
114 | |||
115 | @end detailmenu | ||
116 | @end menu | ||
117 | |||
118 | @node Installing GNUnet | ||
119 | @chapter Installing GNUnet | ||
120 | |||
121 | First of all you have to install a current version of GNUnet. | ||
122 | You can download a tarball of a stable version from GNU FTP mirrors | ||
123 | or obtain the latest development version from our Git repository. | ||
124 | |||
125 | Most of the time you should prefer to download the stable version | ||
126 | since with the latest development version things can be broken, | ||
127 | functionality can be changed or tests can fail. You should only use | ||
128 | the development version if you know that you require a certain | ||
129 | feature or a certain issue has been fixed since the last release. | ||
130 | |||
131 | @menu | ||
132 | * Obtaining a stable version:: | ||
133 | * Installing Build Tool Chain and Dependencies:: | ||
134 | * Obtaining the latest version from Git:: | ||
135 | * Compiling and Installing GNUnet:: | ||
136 | * Common Issues - Check your GNUnet installation:: | ||
137 | @end menu | ||
138 | |||
139 | @node Obtaining a stable version | ||
140 | @section Obtaining a stable version | ||
141 | |||
142 | Download the tarball from | ||
143 | @indicateurl{https://ftp.gnu.org/gnu/gnunet/gnunet-@value{VERSION}.tar.gz}. | ||
144 | |||
145 | Make sure to download the associated @file{.sig} file and to verify the | ||
146 | authenticity of the tarball against it, like this: | ||
147 | |||
148 | @example | ||
149 | $ wget https://ftp.gnu.org/gnu/gnunet/gnunet-@value{VERSION}.tar.gz.sig | ||
150 | $ gpg --verify-files gnunet-@value{VERSION}.tar.gz.sig | ||
151 | @end example | ||
152 | |||
153 | @noindent | ||
154 | If this command fails because you do not have the required public key, | ||
155 | then you need to run the following command to import it: | ||
156 | |||
157 | @example | ||
158 | $ gpg --keyserver keys.gnupg.net --recv-keys 48426C7E | ||
159 | @end example | ||
160 | |||
161 | @noindent | ||
162 | and rerun the @code{gpg --verify-files} command. | ||
163 | |||
164 | @b{Note:}@ | ||
165 | @b{The pub key to sign the 0.10.1 release has been | ||
166 | revoked}. You will get an error message stating that | ||
167 | @b{there is no known public key or that it has been revoked}. | ||
168 | The next release of GNUnet will have a valid signature | ||
169 | again. We are sorry for the inconvenience this causes. | ||
170 | Another possible source you could use is our | ||
171 | "gnunet" git repository which, since the change from SVN to git in 2016, | ||
172 | has mandatory signed commits by every developer. | ||
173 | |||
174 | After verifying the signature you can extract the tarball. | ||
175 | The resulting directory will be renamed to @file{gnunet}, which we will | ||
176 | be using in the remainder of this document to refer to the | ||
177 | root of the source directory. | ||
178 | |||
179 | @example | ||
180 | $ tar xvzf gnunet-@value{VERSION}.tar.gz | ||
181 | $ mv gnunet-@value{VERSION} gnunet | ||
182 | @end example | ||
183 | |||
184 | @c FIXME: This can be irritating for the reader - First we say git should | ||
185 | @c be avoid unless it is really required, and then we write this | ||
186 | @c paragraph: | ||
187 | @noindent | ||
188 | However, please note that stable versions can be very outdated. | ||
189 | As a developer you are @b{strongly} encouraged to use the version | ||
190 | from @uref{https://gnunet.org/git/, git}. | ||
191 | |||
192 | @node Installing Build Tool Chain and Dependencies | ||
193 | @section Installing Build Tool Chain and Dependencies | ||
194 | |||
195 | To successfully compile GNUnet, you need the tools to build GNUnet and | ||
196 | the required dependencies. Please take a look at the | ||
197 | GNUnet Reference Documentation | ||
198 | (@pxref{Dependencies, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation}) | ||
199 | for a list of required dependencies and | ||
200 | (@pxref{Generic installation instructions, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation}) | ||
201 | read its Installation chapter for specific instructions for | ||
202 | your Operating System. | ||
203 | Please check the notes at the end of the configure process about | ||
204 | required dependencies. | ||
205 | |||
206 | For GNUnet bootstrapping support and the HTTP(S) plugin you should | ||
207 | install @uref{https://gnunet.org/gnurl, libgnurl}. | ||
208 | For the filesharing service you should install at least one of the | ||
209 | datastore backends (MySQL, SQlite and PostgreSQL are supported). | ||
210 | |||
211 | @node Obtaining the latest version from Git | ||
212 | @section Obtaining the latest version from Git | ||
213 | |||
214 | The latest development version can be obtained from our Git repository. | ||
215 | To get the code you need to have @code{Git} installed. Usually your | ||
216 | Operating System package manager should provide a suitable distribution | ||
217 | of git (otherwise check out Guix or Nix). If you are using an Operating | ||
218 | System based on Debian's apt: | ||
219 | |||
220 | @example | ||
221 | $ sudo apt-get install git | ||
222 | @end example | ||
223 | |||
224 | This is required for obtaining the repository, which is achieved with | ||
225 | the following command: | ||
226 | |||
227 | @example | ||
228 | $ git clone https://gnunet.org/git/gnunet | ||
229 | @end example | ||
230 | |||
231 | @noindent | ||
232 | After cloning the repository, you have to execute the @file{bootstrap} | ||
233 | script in the new directory: | ||
234 | |||
235 | @example | ||
236 | $ cd gnunet | ||
237 | $ ./bootstrap | ||
238 | @end example | ||
239 | |||
240 | @noindent | ||
241 | The remainder of this tutorial will assume that you have the | ||
242 | Git branch ``master'' checked out. | ||
243 | |||
244 | @node Compiling and Installing GNUnet | ||
245 | @section Compiling and Installing GNUnet | ||
246 | |||
247 | Note: This section is a duplication of the more in depth | ||
248 | @pxref{GNUnet Installation Handbook, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation}. | ||
249 | |||
250 | First, you need to install libgnupgerror @geq{} 1.27 and | ||
251 | libgcrypt @geq{} 1.7.6: | ||
252 | |||
253 | @example | ||
254 | $ export GNUPGFTP="https://www.gnupg.org/ftp/gcrypt" | ||
255 | $ wget $GNUPGFTP/libgpg-error/libgpg-error-1.27.tar.bz2 | ||
256 | $ tar xf libgpg-error-1.27.tar.bz2 | ||
257 | $ cd libgpg-error-1.27 | ||
258 | $ ./configure | ||
259 | $ make | ||
260 | $ sudo make install | ||
261 | $ cd .. | ||
262 | @end example | ||
263 | |||
264 | @example | ||
265 | $ export GNUPGFTP="https://www.gnupg.org/ftp/gcrypt" | ||
266 | $ wget $GNUPGFTP/libgcrypt/libgcrypt-1.7.6.tar.bz2 | ||
267 | $ tar xf libgcrypt-1.7.6.tar.bz2 | ||
268 | $ cd libgcrypt-1.7.6 | ||
269 | $ ./configure | ||
270 | $ make | ||
271 | $ sudo make install | ||
272 | $ cd .. | ||
273 | @end example | ||
274 | |||
275 | @menu | ||
276 | * Installation:: | ||
277 | @end menu | ||
278 | |||
279 | @node Installation | ||
280 | @subsection Installation | ||
281 | Assuming all dependencies are installed, the following commands will | ||
282 | compile and install GNUnet in your home directory. You can specify the | ||
283 | directory where GNUnet will be installed by changing the | ||
284 | @code{--prefix} value when calling @command{./configure}. If | ||
285 | you do not specifiy a prefix, GNUnet is installed in the directory | ||
286 | @file{/usr/local}. When developing new applications you may want | ||
287 | to enable verbose logging by adding @code{--enable-logging=verbose}: | ||
288 | |||
289 | @example | ||
290 | $ export PREFIX=$HOME | ||
291 | $ ./configure --prefix=$PREFIX --enable-logging | ||
292 | $ make | ||
293 | $ make install | ||
294 | @end example | ||
295 | |||
296 | @noindent | ||
297 | After installing GNUnet you have to add your GNUnet installation | ||
298 | to your path environmental variable. In addition you have to | ||
299 | create the @file{.config} directory in your home directory | ||
300 | (unless it already exists) where GNUnet stores its data and an | ||
301 | empty GNUnet configuration file: | ||
302 | |||
303 | @example | ||
304 | $ export PATH=$PATH:$PREFIX/bin | ||
305 | $ echo export PATH=$PREFIX/bin:\\$PATH >> ~/.bashrc | ||
306 | $ mkdir ~/.config/ | ||
307 | $ touch ~/.config/gnunet.conf | ||
308 | @end example | ||
309 | |||
310 | @node Common Issues - Check your GNUnet installation | ||
311 | @section Common Issues - Check your GNUnet installation | ||
312 | |||
313 | You should check your installation to ensure that installing GNUnet | ||
314 | was successful up to this point. You should be able to access GNUnet's | ||
315 | binaries and run GNUnet's self check. | ||
316 | |||
317 | @example | ||
318 | $ which gnunet-arm | ||
319 | $PREFIX/bin/gnunet-arm | ||
320 | @end example | ||
321 | |||
322 | @noindent | ||
323 | should return $PREFIX/bin/gnunet-arm (where $PREFIX is the location | ||
324 | you have set earlier). It should be located in your | ||
325 | GNUnet installation and the output should not be empty. | ||
326 | |||
327 | If you see an output like: | ||
328 | |||
329 | @example | ||
330 | $ which gnunet-arm | ||
331 | @end example | ||
332 | |||
333 | @noindent | ||
334 | check your PATH variable to ensure GNUnet's @file{bin} directory is | ||
335 | included. | ||
336 | |||
337 | GNUnet provides tests for all of its subcomponents. Assuming you have | ||
338 | successfully built GNUnet, run | ||
339 | |||
340 | @example | ||
341 | $ cd gnunet | ||
342 | $ make check | ||
343 | @end example | ||
344 | |||
345 | @noindent | ||
346 | to execute tests for all components. @command{make check} traverses all | ||
347 | subdirectories in @file{src}. For every subdirectory you should | ||
348 | get a message like this: | ||
349 | |||
350 | @example | ||
351 | make[2]: Entering directory `/home/$USER/gnunet/contrib' | ||
352 | PASS: test_gnunet_prefix | ||
353 | ============= | ||
354 | 1 test passed | ||
355 | ============= | ||
356 | @end example | ||
357 | |||
358 | @node Introduction to GNUnet Architecture | ||
359 | @chapter Introduction to GNUnet Architecture | ||
360 | |||
361 | GNUnet is organized in layers and services. Each service is composed of a | ||
362 | main service implementation and a client library for other programs to use | ||
363 | the service's functionality, described by an API. | ||
364 | @c This approach is shown in | ||
365 | @c FIXME: enable this once the commented block below works: | ||
366 | @c figure~\ref fig:service. | ||
367 | Some services provide an additional command line tool to enable the user | ||
368 | to interact with the service. | ||
369 | |||
370 | Very often it is other GNUnet services that will use these APIs to build | ||
371 | the higher layers of GNUnet on top of the lower ones. Each layer expands | ||
372 | or extends the functionality of the service below (for instance, to build | ||
373 | a mesh on top of a DHT). | ||
374 | @c FXIME: See comment above. | ||
375 | @c See figure ~\ref fig:interaction for an illustration of this approach. | ||
376 | |||
377 | @c ** @image filename[, width[, height[, alttext[, extension]]]] | ||
378 | @c FIXME: Texlive (?) 20112 makes the assumption that this means | ||
379 | @c 'images/OBJECTNAME.txt' but later versions of it (2017) use this | ||
380 | @c syntax as described below. | ||
381 | @c TODO: Checkout the makedoc script Guile uses. | ||
382 | |||
383 | @c FIXME!!! | ||
384 | @c @image{images/gnunet-tutorial-service,,5in,Service with API and network protocol,.png} | ||
385 | @c @image{images/gnunet-tutorial-system,,5in,The layered system architecture of GNUnet,.png} | ||
386 | |||
387 | @c \begin{figure}[!h] | ||
388 | @c \begin{center} | ||
389 | @c % \begin{subfigure} | ||
390 | @c \begin{subfigure}[b]{0.3\textwidth} | ||
391 | @c \centering | ||
392 | @c \includegraphics[width=\textwidth]{figs/Service.pdf} | ||
393 | @c \caption{Service with API and network protocol} | ||
394 | @c \label{fig:service} | ||
395 | @c \end{subfigure} | ||
396 | @c ~~~~~~~~~~ | ||
397 | @c \begin{subfigure}[b]{0.3\textwidth} | ||
398 | @c \centering | ||
399 | @c \includegraphics[width=\textwidth]{figs/System.pdf} | ||
400 | @c \caption{Service interaction} | ||
401 | @c \label{fig:interaction} | ||
402 | @c \end{subfigure} | ||
403 | @c \end{center} | ||
404 | @c \caption{GNUnet's layered system architecture} | ||
405 | @c \end{figure} | ||
406 | |||
407 | The main service implementation runs as a standalone process in the | ||
408 | Operating System and the client code runs as part of the client program, | ||
409 | so crashes of a client do not affect the service process or other clients. | ||
410 | The service and the clients communicate via a message protocol to be | ||
411 | defined and implemented by the programmer. | ||
412 | |||
413 | @node First Steps with GNUnet | ||
414 | @chapter First Steps with GNUnet | ||
415 | |||
416 | @menu | ||
417 | * Configure your peer:: | ||
418 | * Start a peer:: | ||
419 | * Monitor a peer:: | ||
420 | * Starting Two Peers by Hand:: | ||
421 | * Starting Peers Using the Testbed Service:: | ||
422 | @end menu | ||
423 | |||
424 | @node Configure your peer | ||
425 | @section Configure your peer | ||
426 | |||
427 | First of all we need to configure your peer. Each peer is started with | ||
428 | a configuration containing settings for GNUnet itself and its services. | ||
429 | This configuration is based on the default configuration shipped with | ||
430 | GNUnet and can be modified. The default configuration is located in the | ||
431 | @file{$PREFIX/share/gnunet/config.d} directory. When starting a peer, you | ||
432 | can specify a customized configuration using the the @command{-c} command | ||
433 | line switch when starting the ARM service and all other services. When | ||
434 | using a modified configuration the default values are loaded and only | ||
435 | values specified in the configuration file will replace the default | ||
436 | values. | ||
437 | |||
438 | Since we want to start additional peers later, we need some modifications | ||
439 | from the default configuration. We need to create a separate service | ||
440 | home and a file containing our modifications for this peer: | ||
441 | |||
442 | @example | ||
443 | $ mkdir ~/gnunet1/ | ||
444 | $ touch peer1.conf | ||
445 | @end example | ||
446 | |||
447 | @noindent | ||
448 | Now add the following lines to @file{peer1.conf} to use this directory. | ||
449 | For simplified usage we want to prevent the peer to connect to the GNUnet | ||
450 | network since this could lead to confusing output. This modifications | ||
451 | will replace the default settings: | ||
452 | |||
453 | @example | ||
454 | [PATHS] | ||
455 | # Use this directory to store GNUnet data | ||
456 | GNUNET_HOME = ~/gnunet1/ | ||
457 | [hostlist] | ||
458 | # prevent bootstrapping | ||
459 | SERVERS = | ||
460 | @end example | ||
461 | |||
462 | @node Start a peer | ||
463 | @section Start a peer | ||
464 | Each GNUnet instance (called peer) has an identity (peer ID) based on a | ||
465 | cryptographic public private key pair. The peer ID is the printable hash | ||
466 | of the public key. | ||
467 | |||
468 | GNUnet services are controlled by a master service, the so called | ||
469 | @dfn{Automatic Restart Manager} (ARM). ARM starts, stops and even | ||
470 | restarts services automatically or on demand when a client connects. | ||
471 | You interact with the ARM service using the @command{gnunet-arm} tool. | ||
472 | GNUnet can then be started with @command{gnunet-arm -s} and stopped with | ||
473 | @command{gnunet-arm -e}. An additional service not automatically started | ||
474 | can be started using @command{gnunet-arm -i <service name>} and stopped | ||
475 | using @command{gnunet-arm -k <servicename>}. | ||
476 | |||
477 | Once you have started your peer, you can use many other GNUnet commands | ||
478 | to interact with it. For example, you can run: | ||
479 | |||
480 | @example | ||
481 | $ gnunet-peerinfo -s | ||
482 | @end example | ||
483 | |||
484 | @noindent | ||
485 | to obtain the public key of your peer. | ||
486 | |||
487 | You should see an output containing the peer ID similar to: | ||
488 | |||
489 | @example | ||
490 | I am peer `0PA02UVRKQTS2C .. JL5Q78F6H0B1ACPV1CJI59MEQUMQCC5G'. | ||
491 | @end example | ||
492 | |||
493 | @node Monitor a peer | ||
494 | @section Monitor a peer | ||
495 | |||
496 | In this section, we will monitor the behaviour of our peer's DHT | ||
497 | service with respect to a specific key. First we will start | ||
498 | GNUnet and then start the DHT service and use the DHT monitor tool | ||
499 | to monitor the PUT and GET commands we issue ussing the | ||
500 | @command{gnunet-dht-put} and @command{gnunet-dht-get} commands. | ||
501 | Using the ``monitor'' line given below, you can observe the behavior | ||
502 | of your own peer's DHT with respect to the specified KEY: | ||
503 | |||
504 | @example | ||
505 | # start gnunet with all default services: | ||
506 | $ gnunet-arm -c ~/peer1.conf -s | ||
507 | # start DHT service: | ||
508 | $ gnunet-arm -c ~/peer1.conf -i dht | ||
509 | $ cd ~/gnunet/src/dht; | ||
510 | $ ./gnunet-dht-monitor -c ~/peer1.conf -k KEY | ||
511 | @end example | ||
512 | |||
513 | @noindent | ||
514 | Now open a separate terminal and change again to | ||
515 | the @file{gnunet/src/dht} directory: | ||
516 | |||
517 | @example | ||
518 | $ cd ~/gnunet/src/dht | ||
519 | # put VALUE under KEY in the DHT: | ||
520 | $ ./gnunet-dht-put -c ~/peer1.conf -k KEY -d VALUE | ||
521 | # get key KEY from the DHT: | ||
522 | $ ./gnunet/src/dht/gnunet-dht-get -c ~/peer1.conf -k KEY | ||
523 | # print statistics about current GNUnet state: | ||
524 | $ gnunet-statistics -c ~/peer1.conf | ||
525 | # print statistics about DHT service: | ||
526 | $ gnunet-statistics -c ~/peer1.conf -s dht | ||
527 | @end example | ||
528 | |||
529 | @node Starting Two Peers by Hand | ||
530 | @section Starting Two Peers by Hand | ||
531 | |||
532 | This section describes how to start two peers on the same machine by hand. | ||
533 | The process is rather painful, but the description is somewhat | ||
534 | instructive. In practice, you might prefer the automated method | ||
535 | (@pxref{Starting Peers Using the Testbed Service}). | ||
536 | |||
537 | @menu | ||
538 | * Setup a second peer:: | ||
539 | * Start the second peer and connect the peers:: | ||
540 | * How to connect manually:: | ||
541 | @end menu | ||
542 | |||
543 | @node Setup a second peer | ||
544 | @subsection Setup a second peer | ||
545 | We will now start a second peer on your machine. | ||
546 | For the second peer, you will need to manually create a modified | ||
547 | configuration file to avoid conflicts with ports and directories. | ||
548 | A peers configuration file is by default located | ||
549 | in @file{~/.gnunet/gnunet.conf}. This file is typically very short | ||
550 | or even empty as only the differences to the defaults need to be | ||
551 | specified. The defaults are located in many files in the | ||
552 | @file{$PREFIX/share/gnunet/config.d} directory. | ||
553 | |||
554 | To configure the second peer, use the files | ||
555 | @file{$PREFIX/share/gnunet/config.d} as a template for your main | ||
556 | configuration file: | ||
557 | |||
558 | @example | ||
559 | $ cat $PREFIX/share/gnunet/config.d/*.conf > peer2.conf | ||
560 | @end example | ||
561 | |||
562 | @noindent | ||
563 | Now you have to edit @file{peer2.conf} and change: | ||
564 | |||
565 | @itemize | ||
566 | @item @code{GNUNET\_TEST\_HOME} under @code{PATHS} | ||
567 | @item Every (uncommented) value for ``@code{PORT}'' (add 10000) in any | ||
568 | section (the option may be commented out if @code{PORT} is | ||
569 | prefixed by "\#", in this case, UNIX domain sockets are used | ||
570 | and the PORT option does not need to be touched) | ||
571 | @item Every value for ``@code{UNIXPATH}'' in any section | ||
572 | (e.g. by adding a "-p2" suffix) | ||
573 | @end itemize | ||
574 | |||
575 | to a fresh, unique value. Make sure that the PORT numbers stay | ||
576 | below 65536. From now on, whenever you interact with the second peer, | ||
577 | you need to specify @command{-c peer2.conf} as an additional | ||
578 | command line argument. | ||
579 | |||
580 | Now, generate the 2nd peer's private key: | ||
581 | |||
582 | @example | ||
583 | $ gnunet-peerinfo -s -c peer2.conf | ||
584 | @end example | ||
585 | |||
586 | @noindent | ||
587 | This may take a while, generate entropy using your keyboard or mouse | ||
588 | as needed. Also, make sure the output is different from the | ||
589 | gnunet-peerinfo output for the first peer (otherwise you made an | ||
590 | error in the configuration). | ||
591 | |||
592 | @node Start the second peer and connect the peers | ||
593 | @subsection Start the second peer and connect the peers | ||
594 | |||
595 | Then, you can start a second peer using: | ||
596 | |||
597 | @example | ||
598 | $ gnunet-arm -c peer2.conf -s | ||
599 | $ gnunet-arm -c peer2.conf -i dht | ||
600 | $ ~/gnunet/src/dht/gnunet-dht-put -c peer2.conf -k KEY -d VALUE | ||
601 | $ ~/gnunet/src/dht/gnunet-dht-get -c peer2.conf -k KEY | ||
602 | @end example | ||
603 | |||
604 | If you want the two peers to connect, you have multiple options: | ||
605 | |||
606 | @itemize | ||
607 | @item UDP neighbour discovery (automatic) | ||
608 | @item Setup a bootstrap server | ||
609 | @item Connect manually | ||
610 | @end itemize | ||
611 | |||
612 | To setup peer 1 as bootstrapping server change the configuration of | ||
613 | the first one to be a hostlist server by adding the following lines to | ||
614 | @file{peer1.conf} to enable bootstrapping server: | ||
615 | |||
616 | @example | ||
617 | [hostlist] | ||
618 | OPTIONS = -p | ||
619 | @end example | ||
620 | |||
621 | @noindent | ||
622 | Then change @file{peer2.conf} and replace the ``@code{SERVERS}'' | ||
623 | line in the ``@code{[hostlist]}'' section with | ||
624 | ``@code{http://localhost:8080/}''. Restart both peers using: | ||
625 | |||
626 | @example | ||
627 | # stop first peer | ||
628 | $ gnunet-arm -c peer1.conf -e | ||
629 | # start first peer | ||
630 | $ gnunet-arm -c peer1.conf -s | ||
631 | # start second peer | ||
632 | $ gnunet-arm -c peer2.conf -s | ||
633 | @end example | ||
634 | |||
635 | @noindent | ||
636 | Note that if you start your peers without changing these settings, they | ||
637 | will use the ``global'' hostlist servers of the GNUnet P2P network and | ||
638 | likely connect to those peers. At that point, debugging might become | ||
639 | tricky as you're going to be connected to many more peers and would | ||
640 | likely observe traffic and behaviors that are not explicitly controlled | ||
641 | by you. | ||
642 | |||
643 | @node How to connect manually | ||
644 | @subsection How to connect manually | ||
645 | |||
646 | If you want to use the @code{peerinfo} tool to connect your | ||
647 | peers, you should: | ||
648 | |||
649 | @itemize | ||
650 | @item Set @code{IMMEDIATE_START = NO} in section @code{hostlist} | ||
651 | (to not connect to the global GNUnet) | ||
652 | @item Start both peers running @command{gnunet-arm -c peer1.conf -s} | ||
653 | and @command{gnunet-arm -c peer2.conf -s} | ||
654 | @item Get @code{HELLO} message of the first peer running | ||
655 | @command{gnunet-peerinfo -c peer1.conf -g} | ||
656 | @item Give the output to the second peer by running | ||
657 | @command{gnunet-peerinfo -c peer2.conf -p '<output>'} | ||
658 | @end itemize | ||
659 | |||
660 | Check that they are connected using @command{gnunet-core -c peer1.conf}, | ||
661 | which should give you the other peer's peer identity: | ||
662 | |||
663 | @example | ||
664 | $ gnunet-core -c peer1.conf | ||
665 | Peer `9TVUCS8P5A7ILLBGO6 [...shortened...] 1KNBJ4NGCHP3JPVULDG' | ||
666 | @end example | ||
667 | |||
668 | @node Starting Peers Using the Testbed Service | ||
669 | @section Starting Peers Using the Testbed Service | ||
670 | @c \label{sec:testbed} | ||
671 | |||
672 | GNUnet's testbed service is used for testing scenarios where | ||
673 | a number of peers are to be started. The testbed can manage peers | ||
674 | on a single host or on multiple hosts in a distributed fashion. | ||
675 | On a single affordable computer, it should be possible to run | ||
676 | around tens of peers without drastically increasing the load on the | ||
677 | system. | ||
678 | |||
679 | The testbed service can be access through its API | ||
680 | @file{include/gnunet\_testbed\_service.h}. The API provides many | ||
681 | routines for managing a group of peers. It also provides a helper | ||
682 | function @code{GNUNET\_TESTBED\_test\_run()} to quickly setup a | ||
683 | minimalistic testing environment on a single host. | ||
684 | |||
685 | This function takes a configuration file which will be used as a | ||
686 | template configuration for the peers. The testbed takes care of | ||
687 | modifying relevant options in the peers' configuration such as | ||
688 | @code{SERVICEHOME}, @code{PORT}, @code{UNIXPATH} to unique values | ||
689 | so that peers run without running into conflicts. It also checks | ||
690 | and assigns the ports in configurations only if they are free. | ||
691 | |||
692 | Additionally, the testbed service also reads its options from the | ||
693 | same configuration file. Various available options and details | ||
694 | about them can be found in the testbed default configuration file | ||
695 | @file{src/testbed/testbed.conf}. | ||
696 | |||
697 | With the testbed API, a sample test case can be structured as follows: | ||
698 | |||
699 | @example | ||
700 | @verbatiminclude examples/testbed_test.c | ||
701 | @end example | ||
702 | |||
703 | @noindent | ||
704 | The source code for the above listing can be found at | ||
705 | @uref{https://gnunet.org/git/gnunet.git/tree/doc/ | ||
706 | documentation/testbed_test.c} | ||
707 | or in the @file{doc/documentation/} folder of your repository check-out. | ||
708 | After installing GNUnet, the above source code can be compiled as: | ||
709 | |||
710 | @example | ||
711 | $ export CPPFLAGS="-I/path/to/gnunet/headers" | ||
712 | $ export LDFLAGS="-L/path/to/gnunet/libraries" | ||
713 | $ gcc $CPPFLAGS $LDFLAGS -o testbed-test testbed_test.c \ | ||
714 | -lgnunettestbed -lgnunetdht -lgnunetutil | ||
715 | # Generate (empty) configuration | ||
716 | $ touch template.conf | ||
717 | # run it (press CTRL-C to stop) | ||
718 | $ ./testbed-test | ||
719 | @end example | ||
720 | |||
721 | @noindent | ||
722 | The @code{CPPFLAGS} and @code{LDFLAGS} are necessary if GNUnet | ||
723 | is installed into a different directory other than @file{/usr/local}. | ||
724 | |||
725 | All of testbed API's peer management functions treat management | ||
726 | actions as operations and return operation handles. It is expected | ||
727 | that the operations begin immediately, but they may get delayed (to | ||
728 | balance out load on the system). The program using the API then has | ||
729 | to take care of marking the operation as ``done'' so that its | ||
730 | associated resources can be freed immediately and other waiting | ||
731 | operations can be executed. Operations will be canceled if they are | ||
732 | marked as ``done'' before their completion. | ||
733 | |||
734 | An operation is treated as completed when it succeeds or fails. | ||
735 | Completion of an operation is either conveyed as events through | ||
736 | @dfn{controller event callback} or through respective | ||
737 | @dfn{operation completion callbacks}. | ||
738 | In functions which support completion notification | ||
739 | through both controller event callback and operation | ||
740 | completion callback, first the controller event callback will be | ||
741 | called. If the operation is not marked as done in that callback | ||
742 | or if the callback is given as NULL when creating the operation, | ||
743 | the operation completion callback will be called. The API | ||
744 | documentation shows which event are to be expected in the | ||
745 | controller event notifications. It also documents any exceptional | ||
746 | behaviour. | ||
747 | |||
748 | Once the peers are started, test cases often need to connect | ||
749 | some of the peers' services. Normally, opening a connect to | ||
750 | a peer's service requires the peer's configuration. While using | ||
751 | testbed, the testbed automatically generates per-peer configuration. | ||
752 | Accessing those configurations directly through file system is | ||
753 | discouraged as their locations are dynamically created and will be | ||
754 | different among various runs of testbed. To make access to these | ||
755 | configurations easy, testbed API provides the function | ||
756 | @code{GNUNET\_TESTBED\_service\_connect()}. This function fetches | ||
757 | the configuration of a given peer and calls the @dfn{Connect Adapter}. | ||
758 | In the example code, it is the @code{dht\_ca}. A connect adapter is | ||
759 | expected to open the connection to the needed service by using the | ||
760 | provided configuration and return the created service connection handle. | ||
761 | Successful connection to the needed service is signaled through | ||
762 | @code{service\_connect\_comp\_cb}. | ||
763 | |||
764 | A dual to connect adapter is the @dfn{Disconnect Adapter}. This callback | ||
765 | is called after the connect adapter has been called when the operation | ||
766 | from @code{GNUNET\_TESTBED\_service\_connect()} is marked as ``done''. | ||
767 | It has to disconnect from the service with the provided service | ||
768 | handle (@code{op\_result}). | ||
769 | |||
770 | Exercise: Find out how many peers you can run on your system. | ||
771 | |||
772 | Exercise: Find out how to create a 2D torus topology by changing the | ||
773 | options in the configuration file. | ||
774 | @xref{Supported Topologies, The GNUnet Reference Documentation ,, gnunet, The GNUnet Reference Documentation}, | ||
775 | then use the DHT API to store and retrieve values in the network. | ||
776 | |||
777 | @node Developing Applications | ||
778 | @chapter Developing Applications | ||
779 | |||
780 | @menu | ||
781 | * gnunet-ext:: | ||
782 | * Adapting the Template:: | ||
783 | * Writing a Client Application:: | ||
784 | * Writing a Service:: | ||
785 | * Interacting directly with other Peers using the CORE Service:: | ||
786 | * Storing peer-specific data using the PEERSTORE service:: | ||
787 | * Using the DHT:: | ||
788 | * Debugging with gnunet-arm:: | ||
789 | @end menu | ||
790 | |||
791 | @node gnunet-ext | ||
792 | @section gnunet-ext | ||
793 | To develop a new peer-to-peer application or to extend GNUnet we provide | ||
794 | a template build system for writing GNUnet extensions in C. It can be | ||
795 | obtained as follows: | ||
796 | |||
797 | @example | ||
798 | $ git clone https://gnunet.org/git/gnunet-ext | ||
799 | $ cd gnunet-ext/ | ||
800 | $ ./bootstrap | ||
801 | $ ./configure --prefix=$PREFIX --with-gnunet=$PREFIX | ||
802 | $ make | ||
803 | $ make install | ||
804 | $ make check | ||
805 | @end example | ||
806 | |||
807 | @noindent | ||
808 | The GNUnet ext template includes examples and a working buildsystem | ||
809 | for a new GNUnet service. A common GNUnet service consists of the | ||
810 | following parts which will be discussed in detail in the remainder | ||
811 | of this document. The functionality of a GNUnet service is implemented in: | ||
812 | |||
813 | @itemize | ||
814 | @item the GNUnet service (gnunet-ext/src/ext/gnunet-service-ext.c) | ||
815 | @item the client API (gnunet-ext/src/ext/ext_api.c) | ||
816 | @item the client application using the service API | ||
817 | (gnunet-ext/src/ext/gnunet-ext.c) | ||
818 | @end itemize | ||
819 | |||
820 | The interfaces for these entities are defined in: | ||
821 | |||
822 | @itemize | ||
823 | @item client API interface (gnunet-ext/src/ext/ext.h) | ||
824 | @item the service interface (gnunet-ext/src/include/gnunet_service_SERVICE.h) | ||
825 | @item the P2P protocol (gnunet-ext/src/include/gnunet_protocols_ext.h) | ||
826 | @end itemize | ||
827 | |||
828 | |||
829 | In addition the ext systems provides: | ||
830 | |||
831 | @itemize | ||
832 | @item a test testing the API (gnunet-ext/src/ext/test_ext_api.c) | ||
833 | @item a configuration template for the service | ||
834 | (gnunet-ext/src/ext/ext.conf.in) | ||
835 | @end itemize | ||
836 | |||
837 | @node Adapting the Template | ||
838 | @section Adapting the Template | ||
839 | |||
840 | The first step for writing any extension with a new service is to | ||
841 | ensure that the @file{ext.conf.in} file contains entries for the | ||
842 | @code{UNIXPATH}, @code{PORT} and @code{BINARY} for the service in a | ||
843 | section named after the service. | ||
844 | |||
845 | If you want to adapt the template rename the @file{ext.conf.in} to | ||
846 | match your services name, you have to modify the @code{AC\_OUTPUT} | ||
847 | section in @file{configure.ac} in the @file{gnunet-ext} root. | ||
848 | |||
849 | @node Writing a Client Application | ||
850 | @section Writing a Client Application | ||
851 | |||
852 | When writing any client application (for example, a command-line | ||
853 | tool), the basic structure is to start with the | ||
854 | @code{GNUNET\_PROGRAM\_run} function. This function will parse | ||
855 | command-line options, setup the scheduler and then invoke the | ||
856 | @code{run} function (with the remaining non-option arguments) | ||
857 | and a handle to the parsed configuration (and the configuration | ||
858 | file name that was used, which is typically not needed): | ||
859 | |||
860 | @example | ||
861 | @verbatiminclude examples/001.c | ||
862 | @end example | ||
863 | |||
864 | @menu | ||
865 | * Handling command-line options:: | ||
866 | * Writing a Client Library:: | ||
867 | * Writing a user interface:: | ||
868 | @end menu | ||
869 | |||
870 | @node Handling command-line options | ||
871 | @subsection Handling command-line options | ||
872 | |||
873 | Options can then be added easily by adding global variables and | ||
874 | expanding the @code{options} array. For example, the following would | ||
875 | add a string-option and a binary flag (defaulting to @code{NULL} and | ||
876 | @code{GNUNET\_NO} respectively): | ||
877 | |||
878 | @example | ||
879 | @verbatiminclude examples/002.c | ||
880 | @end example | ||
881 | |||
882 | Issues such as displaying some helpful text describing options using | ||
883 | the @code{--help} argument and error handling are taken care of when | ||
884 | using this approach. Other @code{GNUNET\_GETOPT\_}-functions can be used | ||
885 | to obtain integer value options, increment counters, etc. You can | ||
886 | even write custom option parsers for special circumstances not covered | ||
887 | by the available handlers. To check if an argument was specified by the | ||
888 | user you initialize the variable with a specific value (e.g. NULL for | ||
889 | a string and GNUNET\_SYSERR for a integer) and check after parsing | ||
890 | happened if the values were modified. | ||
891 | |||
892 | Inside the @code{run} method, the program would perform the | ||
893 | application-specific logic, which typically involves initializing and | ||
894 | using some client library to interact with the service. The client | ||
895 | library is supposed to implement the IPC whereas the service provides | ||
896 | more persistent P2P functions. | ||
897 | |||
898 | Exercise: Add a few command-line options and print them inside | ||
899 | of @code{run}. What happens if the user gives invalid arguments? | ||
900 | |||
901 | @node Writing a Client Library | ||
902 | @subsection Writing a Client Library | ||
903 | |||
904 | The first and most important step in writing a client library is to | ||
905 | decide on an API for the library. Typical API calls include | ||
906 | connecting to the service, performing application-specific requests | ||
907 | and cleaning up. Many examples for such service APIs can be found | ||
908 | in the @file{gnunet/src/include/gnunet\_*\_service.h} files. | ||
909 | |||
910 | Then, a client-service protocol needs to be designed. This typically | ||
911 | involves defining various message formats in a header that will be | ||
912 | included by both the service and the client library (but is otherwise | ||
913 | not shared and hence located within the service's directory and not | ||
914 | installed by @command{make install}). Each message must start with a | ||
915 | @code{struct GNUNET\_MessageHeader} and must be shorter than 64k. By | ||
916 | convention, all fields in IPC (and P2P) messages must be in big-endian | ||
917 | format (and thus should be read using @code{ntohl} and similar | ||
918 | functions and written using @code{htonl} and similar functions). | ||
919 | Unique message types must be defined for each message struct in the | ||
920 | @file{gnunet\_protocols.h} header (or an extension-specific include | ||
921 | file). | ||
922 | |||
923 | @menu | ||
924 | * Connecting to the Service:: | ||
925 | * Sending messages:: | ||
926 | * Receiving Replies from the Service:: | ||
927 | @end menu | ||
928 | |||
929 | @node Connecting to the Service | ||
930 | @subsubsection Connecting to the Service | ||
931 | |||
932 | Before a client library can implement the application-specific protocol | ||
933 | with the service, a connection must be created: | ||
934 | |||
935 | @example | ||
936 | @verbatiminclude examples/003.c | ||
937 | @end example | ||
938 | |||
939 | @noindent | ||
940 | As a result a @code{GNUNET\_MQ\_Handle} is returned | ||
941 | which can to used henceforth to transmit messages to the service. | ||
942 | The complete MQ API can be found in @file{gnunet\_mq\_lib.h}. | ||
943 | The @code{hanlders} array in the example above is incomplete. | ||
944 | Here is where you will define which messages you expect to | ||
945 | receive from the service, and which functions handle them. | ||
946 | The @code{error\_cb} is a function that is to be called whenever | ||
947 | there are errors communicating with the service. | ||
948 | |||
949 | @node Sending messages | ||
950 | @subsubsection Sending messages | ||
951 | |||
952 | In GNUnet, messages are always sent beginning with a | ||
953 | @code{struct GNUNET\_MessageHeader} in big endian format. | ||
954 | This header defines the size and the type of the | ||
955 | message, the payload follows after this header. | ||
956 | |||
957 | @example | ||
958 | @verbatiminclude examples/004.c | ||
959 | @end example | ||
960 | |||
961 | @noindent | ||
962 | Existing message types are defined in @file{gnunet\_protocols.h}. | ||
963 | A common way to create a message is with an envelope: | ||
964 | |||
965 | @example | ||
966 | @verbatiminclude examples/005.c | ||
967 | @end example | ||
968 | |||
969 | @noindent | ||
970 | Exercise: Define a message struct that includes a 32-bit | ||
971 | unsigned integer in addition to the standard GNUnet MessageHeader. | ||
972 | Add a C struct and define a fresh protocol number for your message. | ||
973 | Protocol numbers in gnunet-ext are defined | ||
974 | in @file{gnunet-ext/src/include/gnunet_protocols_ext.h} | ||
975 | |||
976 | Exercise: Find out how you can determine the number of messages | ||
977 | in a message queue. | ||
978 | |||
979 | Exercise: Find out how you can determine when a message you | ||
980 | have queued was actually transmitted. | ||
981 | |||
982 | Exercise: Define a helper function to transmit a 32-bit | ||
983 | unsigned integer (as payload) to a service using some given client | ||
984 | handle. | ||
985 | |||
986 | @node Receiving Replies from the Service | ||
987 | @subsubsection Receiving Replies from the Service | ||
988 | |||
989 | Clients can receive messages from the service using the handlers | ||
990 | specified in the @code{handlers} array we specified when connecting | ||
991 | to the service. Entries in the the array are usually created using | ||
992 | one of two macros, depending on whether the message is fixed size | ||
993 | or variable size. Variable size messages are managed using two | ||
994 | callbacks, one to check that the message is well-formed, the other | ||
995 | to actually process the message. Fixed size messages are fully | ||
996 | checked by the MQ-logic, and thus only need to provide the handler | ||
997 | to process the message. Note that the prefixes @code{check\_} | ||
998 | and @code{handle\_} are mandatory. | ||
999 | |||
1000 | @example | ||
1001 | @verbatiminclude examples/006.c | ||
1002 | @end example | ||
1003 | |||
1004 | @noindent | ||
1005 | Exercise: Expand your helper function to receive a response message | ||
1006 | (for example, containing just the @code{struct GNUnet MessageHeader} | ||
1007 | without any payload). Upon receiving the service's response, you | ||
1008 | should call a callback provided to your helper function's API. | ||
1009 | |||
1010 | Exercise: Figure out where you can pass values to the | ||
1011 | closures (@code{cls}). | ||
1012 | |||
1013 | @node Writing a user interface | ||
1014 | @subsection Writing a user interface | ||
1015 | |||
1016 | Given a client library, all it takes to access a service now is to | ||
1017 | combine calls to the client library with parsing command-line | ||
1018 | options. | ||
1019 | |||
1020 | Exercise: Call your client API from your @code{run()} method in your | ||
1021 | client application to send a request to the service. For example, | ||
1022 | send a 32-bit integer value based on a number given at the | ||
1023 | command-line to the service. | ||
1024 | |||
1025 | @node Writing a Service | ||
1026 | @section Writing a Service | ||
1027 | |||
1028 | Before you can test the client you've written so far, you'll | ||
1029 | need to also implement the corresponding service. | ||
1030 | |||
1031 | @menu | ||
1032 | * Code Placement:: | ||
1033 | * Starting a Service:: | ||
1034 | @end menu | ||
1035 | |||
1036 | @node Code Placement | ||
1037 | @subsection Code Placement | ||
1038 | |||
1039 | New services are placed in their own subdirectory under | ||
1040 | @file{gnunet/src}. This subdirectory should contain the API | ||
1041 | implementation file @file{SERVICE\_api.c}, the description of | ||
1042 | the client-service protocol @file{SERVICE.h} and P2P protocol | ||
1043 | @file{SERVICE\_protocol.h}, the implementation of the service itself | ||
1044 | @file{gnunet-service-SERVICE.h} and several files for tests, | ||
1045 | including test code and configuration files. | ||
1046 | |||
1047 | @node Starting a Service | ||
1048 | @subsection Starting a Service | ||
1049 | |||
1050 | The key API definition for creating a service is the | ||
1051 | @code{GNUNET\_SERVICE\_MAIN} macro: | ||
1052 | |||
1053 | @example | ||
1054 | @verbatiminclude examples/007.c | ||
1055 | @end example | ||
1056 | |||
1057 | @noindent | ||
1058 | In addition to the service name and flags, the macro takes three | ||
1059 | functions, typically called @code{run}, @code{client\_connect\_cb} and | ||
1060 | @code{client\_disconnect\_cb} as well as an array of message handlers | ||
1061 | that will be called for incoming messages from clients. | ||
1062 | |||
1063 | A minimal version of the three central service funtions would look | ||
1064 | like this: | ||
1065 | |||
1066 | @example | ||
1067 | @verbatiminclude examples/008.c | ||
1068 | @end example | ||
1069 | |||
1070 | @noindent | ||
1071 | Exercise: Write a stub service that processes no messages at all | ||
1072 | in your code. Create a default configuration for it, integrate it | ||
1073 | with the build system and start the service from | ||
1074 | @command{gnunet-service-arm} using @command{gnunet-arm -i NAME}. | ||
1075 | |||
1076 | Exercise: Figure out how to set the closure (@code{cls}) for handlers | ||
1077 | of a service. | ||
1078 | |||
1079 | Exercise: Figure out how to send messages from the service back to the | ||
1080 | client. | ||
1081 | |||
1082 | Each handler function in the service @b{must} eventually (possibly in some | ||
1083 | asynchronous continuation) call | ||
1084 | @code{GNUNET\_SERVICE\_client\_continue()}. Only after this call | ||
1085 | additional messages from the same client may | ||
1086 | be processed. This way, the service can throttle processing messages | ||
1087 | from the same client. | ||
1088 | |||
1089 | Exercise: Change the service to ``handle'' the message from your | ||
1090 | client (for now, by printing a message). What happens if you | ||
1091 | forget to call @code{GNUNET\_SERVICE\_client\_continue()}? | ||
1092 | |||
1093 | @node Interacting directly with other Peers using the CORE Service | ||
1094 | @section Interacting directly with other Peers using the CORE Service | ||
1095 | |||
1096 | FIXME: This section still needs to be updated to the lastest API! | ||
1097 | |||
1098 | One of the most important services in GNUnet is the @code{CORE} service | ||
1099 | managing connections between peers and handling encryption between peers. | ||
1100 | |||
1101 | One of the first things any service that extends the P2P protocol | ||
1102 | typically does is connect to the @code{CORE} service using: | ||
1103 | |||
1104 | @example | ||
1105 | @verbatiminclude examples/009.c | ||
1106 | @end example | ||
1107 | |||
1108 | @menu | ||
1109 | * New P2P connections:: | ||
1110 | * Receiving P2P Messages:: | ||
1111 | * Sending P2P Messages:: | ||
1112 | * End of P2P connections:: | ||
1113 | @end menu | ||
1114 | |||
1115 | @node New P2P connections | ||
1116 | @subsection New P2P connections | ||
1117 | |||
1118 | Before any traffic with a different peer can be exchanged, the peer must | ||
1119 | be known to the service. This is notified by the @code{CORE} | ||
1120 | @code{connects} callback, which communicates the identity of the new | ||
1121 | peer to the service: | ||
1122 | |||
1123 | @example | ||
1124 | @verbatiminclude examples/010.c | ||
1125 | @end example | ||
1126 | |||
1127 | @noindent | ||
1128 | Note that whatever you return from @code{connects} is given as the | ||
1129 | @code{cls} argument to the message handlers for messages from | ||
1130 | the respective peer. | ||
1131 | |||
1132 | Exercise: Create a service that connects to the @code{CORE}. Then | ||
1133 | start (and connect) two peers and print a message once your connect | ||
1134 | callback is invoked. | ||
1135 | |||
1136 | @node Receiving P2P Messages | ||
1137 | @subsection Receiving P2P Messages | ||
1138 | |||
1139 | To receive messages from @code{CORE}, you pass the desired | ||
1140 | @code{handlers} to the @code{GNUNET\_CORE\_connect()} function, | ||
1141 | just as we showed for services. | ||
1142 | |||
1143 | It is your responsibility to process messages fast enough or | ||
1144 | to implement flow control. If an application does not process | ||
1145 | CORE messages fast enough, CORE will randomly drop messages | ||
1146 | to not keep a very long queue in memory. | ||
1147 | |||
1148 | Exercise: Start one peer with a new service that has a message | ||
1149 | handler and start a second peer that only has your ``old'' service | ||
1150 | without message handlers. Which ``connect'' handlers are invoked when | ||
1151 | the two peers are connected? Why? | ||
1152 | |||
1153 | @node Sending P2P Messages | ||
1154 | @subsection Sending P2P Messages | ||
1155 | |||
1156 | You can transmit messages to other peers using the @code{mq} you were | ||
1157 | given during the @code{connect} callback. Note that the @code{mq} | ||
1158 | automatically is released upon @code{disconnect} and that you must | ||
1159 | not use it afterwards. | ||
1160 | |||
1161 | It is your responsibility to not over-fill the message queue, GNUnet | ||
1162 | will send the messages roughly in the order given as soon as possible. | ||
1163 | |||
1164 | Exercise: Write a service that upon connect sends messages as | ||
1165 | fast as possible to the other peer (the other peer should run a | ||
1166 | service that ``processes'' those messages). How fast is the | ||
1167 | transmission? Count using the STATISTICS service on both ends. Are | ||
1168 | messages lost? How can you transmit messages faster? What happens if | ||
1169 | you stop the peer that is receiving your messages? | ||
1170 | |||
1171 | @node End of P2P connections | ||
1172 | @subsection End of P2P connections | ||
1173 | |||
1174 | If a message handler returns @code{GNUNET\_SYSERR}, the remote | ||
1175 | peer shuts down or there is an unrecoverable network | ||
1176 | disconnection, CORE notifies the service that the peer disconnected. | ||
1177 | After this notification no more messages will be received from the | ||
1178 | peer and the service is no longer allowed to send messages to the peer. | ||
1179 | The disconnect callback looks like the following: | ||
1180 | |||
1181 | @example | ||
1182 | @verbatiminclude examples/011.c | ||
1183 | @end example | ||
1184 | |||
1185 | @noindent | ||
1186 | Exercise: Fix your service to handle peer disconnects. | ||
1187 | |||
1188 | @node Storing peer-specific data using the PEERSTORE service | ||
1189 | @section Storing peer-specific data using the PEERSTORE service | ||
1190 | |||
1191 | GNUnet's PEERSTORE service offers a persistorage for arbitrary | ||
1192 | peer-specific data. Other GNUnet services can use the PEERSTORE | ||
1193 | to store, retrieve and monitor data records. Each data record | ||
1194 | stored with PEERSTORE contains the following fields: | ||
1195 | |||
1196 | @itemize | ||
1197 | @item subsystem: Name of the subsystem responsible for the record. | ||
1198 | @item peerid: Identity of the peer this record is related to. | ||
1199 | @item key: a key string identifying the record. | ||
1200 | @item value: binary record value. | ||
1201 | @item expiry: record expiry date. | ||
1202 | @end itemize | ||
1203 | |||
1204 | The first step is to start a connection to the PEERSTORE service: | ||
1205 | @example | ||
1206 | @verbatiminclude examples/012.c | ||
1207 | @end example | ||
1208 | |||
1209 | The service handle @code{peerstore_handle} will be needed for | ||
1210 | all subsequent PEERSTORE operations. | ||
1211 | |||
1212 | @menu | ||
1213 | * Storing records:: | ||
1214 | * Retrieving records:: | ||
1215 | * Monitoring records:: | ||
1216 | * Disconnecting from PEERSTORE:: | ||
1217 | @end menu | ||
1218 | |||
1219 | @node Storing records | ||
1220 | @subsection Storing records | ||
1221 | |||
1222 | To store a new record, use the following function: | ||
1223 | |||
1224 | @example | ||
1225 | @verbatiminclude examples/013.c | ||
1226 | @end example | ||
1227 | |||
1228 | @noindent | ||
1229 | The @code{options} parameter can either be | ||
1230 | @code{GNUNET_PEERSTORE_STOREOPTION_MULTIPLE} which means that multiple | ||
1231 | values can be stored under the same key combination | ||
1232 | (subsystem, peerid, key), or @code{GNUNET_PEERSTORE_STOREOPTION_REPLACE} | ||
1233 | which means that PEERSTORE will replace any existing values under the | ||
1234 | given key combination (subsystem, peerid, key) with the new given value. | ||
1235 | |||
1236 | The continuation function @code{cont} will be called after the store | ||
1237 | request is successfully sent to the PEERSTORE service. This does not | ||
1238 | guarantee that the record is successfully stored, only that it was | ||
1239 | received by the service. | ||
1240 | |||
1241 | The @code{GNUNET_PEERSTORE_store} function returns a handle to the store | ||
1242 | operation. This handle can be used to cancel the store operation only | ||
1243 | before the continuation function is called: | ||
1244 | |||
1245 | @example | ||
1246 | @verbatiminclude examples/013.1.c | ||
1247 | @end example | ||
1248 | |||
1249 | @node Retrieving records | ||
1250 | @subsection Retrieving records | ||
1251 | |||
1252 | To retrieve stored records, use the following function: | ||
1253 | |||
1254 | @example | ||
1255 | @verbatiminclude examples/014.c | ||
1256 | @end example | ||
1257 | |||
1258 | @noindent | ||
1259 | The values of @code{peer} and @code{key} can be @code{NULL}. This | ||
1260 | allows the iteration over values stored under any of the following | ||
1261 | key combinations: | ||
1262 | |||
1263 | @itemize | ||
1264 | @item (subsystem) | ||
1265 | @item (subsystem, peerid) | ||
1266 | @item (subsystem, key) | ||
1267 | @item (subsystem, peerid, key) | ||
1268 | @end itemize | ||
1269 | |||
1270 | The @code{callback} function will be called once with each retrieved | ||
1271 | record and once more with a @code{NULL} record to signal the end of | ||
1272 | results. | ||
1273 | |||
1274 | The @code{GNUNET_PEERSTORE_iterate} function returns a handle to the | ||
1275 | iterate operation. This handle can be used to cancel the iterate | ||
1276 | operation only before the callback function is called with a | ||
1277 | @code{NULL} record. | ||
1278 | |||
1279 | @node Monitoring records | ||
1280 | @subsection Monitoring records | ||
1281 | |||
1282 | PEERSTORE offers the functionality of monitoring for new records | ||
1283 | stored under a specific key combination (subsystem, peerid, key). | ||
1284 | To start the monitoring, use the following function: | ||
1285 | |||
1286 | @example | ||
1287 | @verbatiminclude examples/015.c | ||
1288 | @end example | ||
1289 | |||
1290 | @noindent | ||
1291 | Whenever a new record is stored under the given key combination, | ||
1292 | the @code{callback} function will be called with this new | ||
1293 | record. This will continue until the connection to the PEERSTORE | ||
1294 | service is broken or the watch operation is canceled: | ||
1295 | |||
1296 | @example | ||
1297 | @verbatiminclude examples/016.c | ||
1298 | @end example | ||
1299 | |||
1300 | @node Disconnecting from PEERSTORE | ||
1301 | @subsection Disconnecting from PEERSTORE | ||
1302 | |||
1303 | When the connection to the PEERSTORE service is no longer needed, | ||
1304 | disconnect using the following function: | ||
1305 | |||
1306 | @example | ||
1307 | @verbatiminclude examples/017.c | ||
1308 | @end example | ||
1309 | |||
1310 | @noindent | ||
1311 | If the @code{sync_first} flag is set to @code{GNUNET_YES}, | ||
1312 | the API will delay the disconnection until all store requests | ||
1313 | are received by the PEERSTORE service. Otherwise, it will | ||
1314 | disconnect immediately. | ||
1315 | |||
1316 | @node Using the DHT | ||
1317 | @section Using the DHT | ||
1318 | |||
1319 | The DHT allows to store data so other peers in the P2P network can | ||
1320 | access it and retrieve data stored by any peers in the network. | ||
1321 | This section will explain how to use the DHT. Of course, the first | ||
1322 | thing to do is to connect to the DHT service: | ||
1323 | |||
1324 | @example | ||
1325 | @verbatiminclude examples/018.c | ||
1326 | @end example | ||
1327 | |||
1328 | @noindent | ||
1329 | The second parameter indicates how many requests in parallel to expect. | ||
1330 | It is not a hard limit, but a good approximation will make the DHT more | ||
1331 | efficient. | ||
1332 | |||
1333 | @menu | ||
1334 | * Storing data in the DHT:: | ||
1335 | * Obtaining data from the DHT:: | ||
1336 | * Implementing a block plugin:: | ||
1337 | * Monitoring the DHT:: | ||
1338 | @end menu | ||
1339 | |||
1340 | @node Storing data in the DHT | ||
1341 | @subsection Storing data in the DHT | ||
1342 | Since the DHT is a dynamic environment (peers join and leave frequently) | ||
1343 | the data that we put in the DHT does not stay there indefinitely. It is | ||
1344 | important to ``refresh'' the data periodically by simply storing it | ||
1345 | again, in order to make sure other peers can access it. | ||
1346 | |||
1347 | The put API call offers a callback to signal that the PUT request has been | ||
1348 | sent. This does not guarantee that the data is accessible to others peers, | ||
1349 | or even that is has been stored, only that the service has requested to | ||
1350 | a neighboring peer the retransmission of the PUT request towards its final | ||
1351 | destination. Currently there is no feedback about whether or not the data | ||
1352 | has been sucessfully stored or where it has been stored. In order to | ||
1353 | improve the availablilty of the data and to compensate for possible | ||
1354 | errors, peers leaving and other unfavorable events, just make several | ||
1355 | PUT requests! | ||
1356 | |||
1357 | @example | ||
1358 | @verbatiminclude examples/019.c | ||
1359 | @end example | ||
1360 | |||
1361 | @noindent | ||
1362 | Exercise: Store a value in the DHT periodically to make sure it | ||
1363 | is available over time. You might consider using the function | ||
1364 | @code{GNUNET\_SCHEDULER\_add\_delayed} and call | ||
1365 | @code{GNUNET\_DHT\_put} from inside a helper function. | ||
1366 | |||
1367 | @node Obtaining data from the DHT | ||
1368 | @subsection Obtaining data from the DHT | ||
1369 | |||
1370 | As we saw in the previous example, the DHT works in an asynchronous mode. | ||
1371 | Each request to the DHT is executed ``in the background'' and the API | ||
1372 | calls return immediately. In order to receive results from the DHT, the | ||
1373 | API provides a callback. Once started, the request runs in the service, | ||
1374 | the service will try to get as many results as possible (filtering out | ||
1375 | duplicates) until the timeout expires or we explicitly stop the request. | ||
1376 | It is possible to give a ``forever'' timeout with | ||
1377 | @code{GNUNET\_TIME\_UNIT\_FOREVER\_REL}. | ||
1378 | |||
1379 | If we give a route option @code{GNUNET\_DHT\_RO\_RECORD\_ROUTE} | ||
1380 | the callback will get a list of all the peers the data has travelled, | ||
1381 | both on the PUT path and on the GET path. | ||
1382 | |||
1383 | @example | ||
1384 | @verbatiminclude examples/020.c | ||
1385 | @end example | ||
1386 | |||
1387 | @noindent | ||
1388 | Exercise: Store a value in the DHT and after a while retrieve it. | ||
1389 | Show the IDs of all the peers the requests have gone through. | ||
1390 | In order to convert a peer ID to a string, use the function | ||
1391 | @code{GNUNET\_i2s}. Pay attention to the route option parameters | ||
1392 | in both calls! | ||
1393 | |||
1394 | @node Implementing a block plugin | ||
1395 | @subsection Implementing a block plugin | ||
1396 | |||
1397 | In order to store data in the DHT, it is necessary to provide a block | ||
1398 | plugin. The DHT uses the block plugin to ensure that only well-formed | ||
1399 | requests and replies are transmitted over the network. | ||
1400 | |||
1401 | The block plugin should be put in a file @file{plugin\_block\_SERVICE.c} | ||
1402 | in the service's respective directory. The | ||
1403 | mandatory functions that need to be implemented for a block plugin are | ||
1404 | described in the following sections. | ||
1405 | |||
1406 | @menu | ||
1407 | * Validating requests and replies:: | ||
1408 | * Deriving a key from a reply:: | ||
1409 | * Initialization of the plugin:: | ||
1410 | * Shutdown of the plugin:: | ||
1411 | * Integration of the plugin with the build system:: | ||
1412 | @end menu | ||
1413 | |||
1414 | @node Validating requests and replies | ||
1415 | @subsubsection Validating requests and replies | ||
1416 | |||
1417 | The evaluate function should validate a reply or a request. It returns | ||
1418 | a @code{GNUNET\_BLOCK\_EvaluationResult}, which is an enumeration. All | ||
1419 | possible answers are in @file{gnunet\_block\_lib.h}. The function will | ||
1420 | be called with a @code{reply\_block} argument of @code{NULL} for | ||
1421 | requests. Note that depending on how @code{evaluate} is called, only | ||
1422 | some of the possible return values are valid. The specific meaning of | ||
1423 | the @code{xquery} argument is application-specific. Applications that | ||
1424 | do not use an extended query should check that the @code{xquery\_size} | ||
1425 | is zero. The block group is typically used to filter duplicate | ||
1426 | replies. | ||
1427 | |||
1428 | @example | ||
1429 | @verbatiminclude examples/021.c | ||
1430 | @end example | ||
1431 | |||
1432 | @noindent | ||
1433 | Note that it is mandatory to detect duplicate replies in this function | ||
1434 | and return the respective status code. Duplicate detection is | ||
1435 | typically done using the Bloom filter block group provided by | ||
1436 | @file{libgnunetblockgroup.so}. Failure to do so may cause replies to | ||
1437 | circle in the network. | ||
1438 | |||
1439 | @node Deriving a key from a reply | ||
1440 | @subsubsection Deriving a key from a reply | ||
1441 | |||
1442 | The DHT can operate more efficiently if it is possible to derive a key | ||
1443 | from the value of the corresponding block. The @code{get\_key} | ||
1444 | function is used to obtain the key of a block --- for example, by | ||
1445 | means of hashing. If deriving the key is not possible, the function | ||
1446 | should simply return @code{GNUNET\_SYSERR} (the DHT will still work | ||
1447 | just fine with such blocks). | ||
1448 | |||
1449 | @example | ||
1450 | @verbatiminclude examples/022.c | ||
1451 | @end example | ||
1452 | |||
1453 | @node Initialization of the plugin | ||
1454 | @subsubsection Initialization of the plugin | ||
1455 | |||
1456 | The plugin is realized as a shared C library. The library must export | ||
1457 | an initialization function which should initialize the plugin. The | ||
1458 | initialization function specifies what block types the plugin cares | ||
1459 | about and returns a struct with the functions that are to be used for | ||
1460 | validation and obtaining keys (the ones just defined above). | ||
1461 | |||
1462 | @example | ||
1463 | @verbatiminclude examples/023.c | ||
1464 | @end example | ||
1465 | |||
1466 | @node Shutdown of the plugin | ||
1467 | @subsubsection Shutdown of the plugin | ||
1468 | |||
1469 | Following GNUnet's general plugin API concept, the plugin must | ||
1470 | export a second function for cleaning up. It usually does very | ||
1471 | little. | ||
1472 | |||
1473 | @example | ||
1474 | @verbatiminclude examples/024.c | ||
1475 | @end example | ||
1476 | |||
1477 | @node Integration of the plugin with the build system | ||
1478 | @subsubsection Integration of the plugin with the build system | ||
1479 | |||
1480 | In order to compile the plugin, the @file{Makefile.am} file for the | ||
1481 | service SERVICE should contain a rule similar to this: | ||
1482 | @c Actually this is a Makefile not C. But the whole structure of examples | ||
1483 | @c must be improved. | ||
1484 | |||
1485 | @example | ||
1486 | @verbatiminclude examples/025.Makefile.am | ||
1487 | @end example | ||
1488 | |||
1489 | @noindent | ||
1490 | Exercise: Write a block plugin that accepts all queries | ||
1491 | and all replies but prints information about queries and replies | ||
1492 | when the respective validation hooks are called. | ||
1493 | |||
1494 | @node Monitoring the DHT | ||
1495 | @subsection Monitoring the DHT | ||
1496 | |||
1497 | It is possible to monitor the functioning of the local | ||
1498 | DHT service. When monitoring the DHT, the service will | ||
1499 | alert the monitoring program of any events, both started | ||
1500 | locally or received for routing from another peer. | ||
1501 | The are three different types of events possible: a | ||
1502 | GET request, a PUT request or a response (a reply to a GET). | ||
1503 | |||
1504 | Since the different events have different associated data, | ||
1505 | the API gets 3 different callbacks (one for each message type) | ||
1506 | and optional type and key parameters, to allow for filtering of | ||
1507 | messages. When an event happens, the appropiate callback is | ||
1508 | called with all the information about the event. | ||
1509 | |||
1510 | @example | ||
1511 | @verbatiminclude examples/026.c | ||
1512 | @end example | ||
1513 | |||
1514 | @node Debugging with gnunet-arm | ||
1515 | @section Debugging with gnunet-arm | ||
1516 | |||
1517 | Even if services are managed by @command{gnunet-arm}, you can | ||
1518 | start them with @command{gdb} or @command{valgrind}. For | ||
1519 | example, you could add the following lines to your | ||
1520 | configuration file to start the DHT service in a @command{gdb} | ||
1521 | session in a fresh @command{xterm}: | ||
1522 | |||
1523 | @example | ||
1524 | [dht] | ||
1525 | PREFIX=xterm -e gdb --args | ||
1526 | @end example | ||
1527 | |||
1528 | @noindent | ||
1529 | Alternatively, you can stop a service that was started via | ||
1530 | ARM and run it manually: | ||
1531 | |||
1532 | @example | ||
1533 | $ gnunet-arm -k dht | ||
1534 | $ gdb --args gnunet-service-dht -L DEBUG | ||
1535 | $ valgrind gnunet-service-dht -L DEBUG | ||
1536 | @end example | ||
1537 | |||
1538 | @noindent | ||
1539 | Assuming other services are well-written, they will automatically | ||
1540 | re-integrate the restarted service with the peer. | ||
1541 | |||
1542 | GNUnet provides a powerful logging mechanism providing log | ||
1543 | levels @code{ERROR}, @code{WARNING}, @code{INFO} and @code{DEBUG}. | ||
1544 | The current log level is configured using the @code{$GNUNET_FORCE_LOG} | ||
1545 | environmental variable. The @code{DEBUG} level is only available if | ||
1546 | @command{--enable-logging=verbose} was used when running | ||
1547 | @command{configure}. More details about logging can be found under | ||
1548 | @uref{https://gnunet.org/logging}. | ||
1549 | |||
1550 | You should also probably enable the creation of core files, by setting | ||
1551 | @code{ulimit}, and echo'ing @code{1} into | ||
1552 | @file{/proc/sys/kernel/core\_uses\_pid}. Then you can investigate the | ||
1553 | core dumps with @command{gdb}, which is often the fastest method to | ||
1554 | find simple errors. | ||
1555 | |||
1556 | Exercise: Add a memory leak to your service and obtain a trace | ||
1557 | pointing to the leak using @command{valgrind} while running the service | ||
1558 | from @command{gnunet-service-arm}. | ||
1559 | |||
1560 | |||
1561 | @c ********************************************************************* | ||
1562 | @node GNU Free Documentation License | ||
1563 | @appendix GNU Free Documentation License | ||
1564 | @cindex license, GNU Free Documentation License | ||
1565 | @include fdl-1.3.texi | ||
1566 | |||
1567 | |||
1568 | @bye | ||