diff options
Diffstat (limited to 'doc/documentation/gnunet-c-tutorial.texi')
-rw-r--r-- | doc/documentation/gnunet-c-tutorial.texi | 1519 |
1 files changed, 1519 insertions, 0 deletions
diff --git a/doc/documentation/gnunet-c-tutorial.texi b/doc/documentation/gnunet-c-tutorial.texi new file mode 100644 index 000000000..69a4c6478 --- /dev/null +++ b/doc/documentation/gnunet-c-tutorial.texi | |||
@@ -0,0 +1,1519 @@ | |||
1 | \input texinfo | ||
2 | @c %**start of header | ||
3 | @setfilename gnunet-c-tutorial.info | ||
4 | @documentencoding UTF-8 | ||
5 | @settitle GNUnet C Tutorial | ||
6 | @exampleindent 2 | ||
7 | @c %**end of header | ||
8 | |||
9 | @c including 'version.texi' makes makeinfo throw errors. | ||
10 | @include version2.texi | ||
11 | |||
12 | @copying | ||
13 | Copyright @copyright{} 2001-2017 GNUnet e.V. | ||
14 | |||
15 | Permission is granted to copy, distribute and/or modify this document | ||
16 | under the terms of the GNU Free Documentation License, Version 1.3 or | ||
17 | any later version published by the Free Software Foundation; with no | ||
18 | Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A | ||
19 | copy of the license is included in the section entitled ``GNU Free | ||
20 | Documentation License''. | ||
21 | |||
22 | A copy of the license is also available from the Free Software | ||
23 | Foundation Web site at @url{http://www.gnu.org/licenses/fdl.html}. | ||
24 | |||
25 | Alternately, this document is also available under the General | ||
26 | Public License, version 3 or later, as published by the Free Software | ||
27 | Foundation. A copy of the license is included in the section entitled | ||
28 | ``GNU General Public License''. | ||
29 | |||
30 | A copy of the license is also available from the Free Software | ||
31 | Foundation Web site at @url{http://www.gnu.org/licenses/gpl.html}. | ||
32 | @end copying | ||
33 | |||
34 | @dircategory Tutorial | ||
35 | @direntry | ||
36 | * GNUnet-C-Tutorial: (gnunet-c-tutorial). C Tutorial for GNunet | ||
37 | @end direntry | ||
38 | |||
39 | |||
40 | @titlepage | ||
41 | @title GNUnet C Tutorial | ||
42 | @subtitle A Tutorial for GNUnet @value{VERSION} (C version) | ||
43 | @author The GNUnet Developers | ||
44 | |||
45 | @page | ||
46 | @vskip 0pt plus 1filll | ||
47 | |||
48 | @insertcopying | ||
49 | @end titlepage | ||
50 | |||
51 | @contents | ||
52 | |||
53 | @c **** TODO | ||
54 | @c 1. Update content? | ||
55 | @c 2. Either reference main documentation or | ||
56 | @c 3. Merge this into main documentation | ||
57 | |||
58 | @node Top | ||
59 | @top Introduction | ||
60 | |||
61 | This tutorials explains how to install GNUnet on a | ||
62 | GNU/Linux system and gives an introduction on how | ||
63 | GNUnet can be used to develop a Peer-to-Peer application. | ||
64 | Detailed installation instructions for | ||
65 | various operating systems and a detailed list of all | ||
66 | dependencies can be found on our website at | ||
67 | @uref{https://gnunet.org/installation} and in our | ||
68 | Reference Documentation (GNUnet Handbook). | ||
69 | |||
70 | Please read this tutorial carefully since every single step is | ||
71 | important and do not hesitate to contact the GNUnet team if you have | ||
72 | any questions or problems! Check here how to contact the GNUnet | ||
73 | team: @uref{https://gnunet.org/contact_information} | ||
74 | |||
75 | @menu | ||
76 | |||
77 | * Installing GNUnet:: Installing GNUnet | ||
78 | * Introduction to GNUnet Architecture:: Introduction to GNUnet Architecture | ||
79 | * First Steps with GNUnet:: First Steps with GNUnet | ||
80 | * Developing Applications:: Developing Applications | ||
81 | |||
82 | @detailmenu | ||
83 | --- The Detailed Node Listing --- | ||
84 | |||
85 | Installing GNUnet | ||
86 | |||
87 | * Obtaining a stable version:: | ||
88 | * Installing Build Tool Chain and Dependencies:: | ||
89 | * Obtaining the latest version from Git:: | ||
90 | * Compiling and Installing GNUnet:: | ||
91 | * Common Issues - Check your GNUnet installation:: | ||
92 | |||
93 | Introduction to GNUnet Architecture | ||
94 | |||
95 | First Steps with GNUnet | ||
96 | |||
97 | * Configure your peer:: | ||
98 | * Start a peer:: | ||
99 | * Monitor a peer:: | ||
100 | * Starting Two Peers by Hand:: | ||
101 | * Starting Peers Using the Testbed Service:: | ||
102 | |||
103 | Developing Applications | ||
104 | |||
105 | * gnunet-ext:: | ||
106 | * Adapting the Template:: | ||
107 | * Writing a Client Application:: | ||
108 | * Writing a Service:: | ||
109 | * Interacting directly with other Peers using the CORE Service:: | ||
110 | * Storing peer-specific data using the PEERSTORE service:: | ||
111 | * Using the DHT:: | ||
112 | * Debugging with gnunet-arm:: | ||
113 | |||
114 | @end detailmenu | ||
115 | @end menu | ||
116 | |||
117 | @node Installing GNUnet | ||
118 | @chapter Installing GNUnet | ||
119 | |||
120 | First of all you have to install a current version of GNUnet. | ||
121 | You can download a tarball of a stable version from GNU FTP mirrors | ||
122 | or obtain the latest development version from our Git repository. | ||
123 | |||
124 | Most of the time you should prefer to download the stable version | ||
125 | since with the latest development version things can be broken, | ||
126 | functionality can be changed or tests can fail. You should only use | ||
127 | the development version if you know that you require a certain | ||
128 | feature or a certain issue has been fixed since the last release. | ||
129 | |||
130 | @menu | ||
131 | * Obtaining a stable version:: | ||
132 | * Installing Build Tool Chain and Dependencies:: | ||
133 | * Obtaining the latest version from Git:: | ||
134 | * Compiling and Installing GNUnet:: | ||
135 | * Common Issues - Check your GNUnet installation:: | ||
136 | @end menu | ||
137 | |||
138 | @node Obtaining a stable version | ||
139 | @section Obtaining a stable version | ||
140 | |||
141 | Download the tarball from | ||
142 | @indicateurl{https://ftp.gnu.org/gnu/gnunet/gnunet-@value{VERSION}.tar.gz}. | ||
143 | |||
144 | Make sure to download the associated @file{.sig} file and to verify the | ||
145 | authenticity of the tarball against it, like this: | ||
146 | |||
147 | @example | ||
148 | $ wget https://ftp.gnu.org/gnu/gnunet/gnunet-@value{VERSION}.tar.gz.sig | ||
149 | $ gpg --verify-files gnunet-@value{VERSION}.tar.gz.sig | ||
150 | @end example | ||
151 | |||
152 | @noindent | ||
153 | If this command fails because you do not have the required public key, | ||
154 | then you need to run this command to import it: | ||
155 | |||
156 | @example | ||
157 | $ gpg --keyserver keys.gnupg.net --recv-keys 48426C7E | ||
158 | @end example | ||
159 | |||
160 | @noindent | ||
161 | and rerun the @code{gpg --verify-files} command. | ||
162 | |||
163 | Now you can extract the tarball and rename the resulting | ||
164 | directory to @i{gnunet} which we will be using in the | ||
165 | remainder of this document. | ||
166 | |||
167 | @example | ||
168 | $ tar xvzf gnunet-@value{VERSION}.tar.gz | ||
169 | $ mv gnunet-@value{VERSION} gnunet | ||
170 | $ cd gnunet | ||
171 | @end example | ||
172 | |||
173 | @noindent | ||
174 | However, please note that stable versions can be very outdated. | ||
175 | As a developer you are @b{strongly} encouraged to use the version | ||
176 | from @uref{https://gnunet.org/git/, git}. | ||
177 | |||
178 | @node Installing Build Tool Chain and Dependencies | ||
179 | @section Installing Build Tool Chain and Dependencies | ||
180 | |||
181 | To successfully compile GNUnet you need the tools to build GNUnet and | ||
182 | the required dependencies. Please have a look at | ||
183 | @uref{https://gnunet.org/dependencies} for a list of required dependencies | ||
184 | and @uref{https://gnunet.org/generic_installation} for specific | ||
185 | instructions for your operating system. Please check the notes at | ||
186 | the end of the configure process about required dependencies. | ||
187 | |||
188 | For GNUnet bootstrapping support and the http(s) plugin you should | ||
189 | install @uref{https://gnunet.org/gnurl, libgnurl}. | ||
190 | For the filesharing service you should install at least one of the | ||
191 | datastore backends. MySQL, SQlite and PostgreSQL are supported. | ||
192 | |||
193 | @node Obtaining the latest version from Git | ||
194 | @section Obtaining the latest version from Git | ||
195 | |||
196 | The latest development version can obtained from our Git repository. | ||
197 | To obtain the code you need Git installed and checkout the repository | ||
198 | using: | ||
199 | |||
200 | @example | ||
201 | $ git clone https://gnunet.org/git/gnunet | ||
202 | @end example | ||
203 | |||
204 | @noindent | ||
205 | After cloning the repository you have to execute the @file{bootstrap} | ||
206 | script in the directory: | ||
207 | |||
208 | @example | ||
209 | $ cd gnunet ; ./bootstrap | ||
210 | @end example | ||
211 | |||
212 | @noindent | ||
213 | The remainder of this tutorial assumes that you have the Git branch | ||
214 | ``master'' checked out. | ||
215 | |||
216 | @node Compiling and Installing GNUnet | ||
217 | @section Compiling and Installing GNUnet | ||
218 | |||
219 | First, you need to install at least libgnupgerror 1.27 and | ||
220 | libgcrypt 1.7.6. | ||
221 | |||
222 | @example | ||
223 | $ export GNUPGFTP="https://www.gnupg.org/ftp/gcrypt" | ||
224 | $ wget $GNUPGFTP/libgpg-error/libgpg-error-1.27.tar.bz2 | ||
225 | $ tar xf libgpg-error-1.27.tar.bz2 | ||
226 | $ cd libgpg-error-1.27 | ||
227 | $ ./configure | ||
228 | $ sudo make install | ||
229 | $ cd .. | ||
230 | @end example | ||
231 | |||
232 | @example | ||
233 | $ export GNUPGFTP="https://www.gnupg.org/ftp/gcrypt" | ||
234 | $ wget $GNUPGFTP/libgcrypt/libgcrypt-1.7.6.tar.bz2 | ||
235 | $ tar xf libgcrypt-1.7.6.tar.bz2 | ||
236 | $ cd libgcrypt-1.7.6 | ||
237 | $ ./configure | ||
238 | $ sudo make install | ||
239 | $ cd .. | ||
240 | @end example | ||
241 | |||
242 | @menu | ||
243 | * Installation:: | ||
244 | @end menu | ||
245 | |||
246 | @node Installation | ||
247 | @subsection Installation | ||
248 | Assuming all dependencies are installed, the following commands will | ||
249 | compile and install GNUnet in your home directory. You can specify the | ||
250 | directory where GNUnet will be installed by changing the | ||
251 | @code{--prefix} value when calling @command{./configure}. If | ||
252 | you do not specifiy a prefix, GNUnet is installed in the directory | ||
253 | @file{/usr/local}. When developing new applications you may want | ||
254 | to enable verbose logging by adding @code{--enable-logging=verbose}: | ||
255 | |||
256 | @example | ||
257 | $ ./configure --prefix=$PREFIX --enable-logging | ||
258 | $ make | ||
259 | $ make install | ||
260 | @end example | ||
261 | |||
262 | @noindent | ||
263 | After installing GNUnet you have to add your GNUnet installation | ||
264 | to your path environmental variable. In addition you have to | ||
265 | create the @file{.config} directory in your home directory | ||
266 | (unless it already exists) where GNUnet stores its data and an | ||
267 | empty GNUnet configuration file: | ||
268 | |||
269 | @example | ||
270 | $ export PATH=$PATH:$PREFIX/bin | ||
271 | $ echo export PATH=$PREFIX/bin:\\$PATH >> ~/.bashrc | ||
272 | $ mkdir ~/.config/ | ||
273 | $ touch ~/.config/gnunet.conf | ||
274 | @end example | ||
275 | |||
276 | @node Common Issues - Check your GNUnet installation | ||
277 | @section Common Issues - Check your GNUnet installation | ||
278 | |||
279 | You should check your installation to ensure that installing GNUnet | ||
280 | was successful up to this point. You should be able to access GNUnet's | ||
281 | binaries and run GNUnet's self check. | ||
282 | |||
283 | @example | ||
284 | $ which gnunet-arm | ||
285 | @end example | ||
286 | |||
287 | @noindent | ||
288 | should return $PREFIX/bin/gnunet-arm. It should be located in your | ||
289 | GNUnet installation and the output should not be empty. | ||
290 | If you see an output like: | ||
291 | |||
292 | @example | ||
293 | $ which gnunet-arm | ||
294 | @end example | ||
295 | |||
296 | @noindent | ||
297 | check your PATH variable to ensure GNUnet's @file{bin} directory is | ||
298 | included. | ||
299 | |||
300 | GNUnet provides tests for all of its subcomponents. Run | ||
301 | |||
302 | @example | ||
303 | $ make check | ||
304 | @end example | ||
305 | |||
306 | @noindent | ||
307 | to execute tests for all components. @command{make check} traverses all | ||
308 | subdirectories in @file{src}. For every subdirectory you should | ||
309 | get a message like this: | ||
310 | |||
311 | @example | ||
312 | make[2]: Entering directory `/home/$USER/gnunet/contrib' | ||
313 | PASS: test_gnunet_prefix | ||
314 | ============= | ||
315 | 1 test passed | ||
316 | ============= | ||
317 | @end example | ||
318 | |||
319 | @node Introduction to GNUnet Architecture | ||
320 | @chapter Introduction to GNUnet Architecture | ||
321 | |||
322 | GNUnet is organized in layers and services. Each service is composed of a | ||
323 | main service implementation and a client library for other programs to use | ||
324 | the service's functionality, described by an API. | ||
325 | @c This approach is shown in | ||
326 | @c FIXME: enable this once the commented block below works: | ||
327 | @c figure~\ref fig:service. | ||
328 | Some services provide an additional command line tool to enable the user | ||
329 | to interact with the service. | ||
330 | |||
331 | Very often it is other GNUnet services that will use these APIs to build | ||
332 | the higher layers of GNUnet on top of the lower ones. Each layer expands | ||
333 | or extends the functionality of the service below (for instance, to build | ||
334 | a mesh on top of a DHT). | ||
335 | @c FXIME: See comment above. | ||
336 | @c See figure ~\ref fig:interaction for an illustration of this approach. | ||
337 | |||
338 | @c ** @image filename[, width[, height[, alttext[, extension]]]] | ||
339 | @c FIXME: Texlive (?) 20112 makes the assumption that this means | ||
340 | @c 'images/OBJECTNAME.txt' but later versions of it (2017) use this | ||
341 | @c syntax as described below. | ||
342 | @c TODO: Checkout the makedoc script Guile uses. | ||
343 | |||
344 | @image{images/gnunet-tutorial-service,,5in,Service with API and network protocol,.png} | ||
345 | |||
346 | @image{images/gnunet-tutorial-system,,5in,The layered system architecture of GNUnet,.png} | ||
347 | |||
348 | @c \begin{figure}[!h] | ||
349 | @c \begin{center} | ||
350 | @c % \begin{subfigure} | ||
351 | @c \begin{subfigure}[b]{0.3\textwidth} | ||
352 | @c \centering | ||
353 | @c \includegraphics[width=\textwidth]{figs/Service.pdf} | ||
354 | @c \caption{Service with API and network protocol} | ||
355 | @c \label{fig:service} | ||
356 | @c \end{subfigure} | ||
357 | @c ~~~~~~~~~~ | ||
358 | @c \begin{subfigure}[b]{0.3\textwidth} | ||
359 | @c \centering | ||
360 | @c \includegraphics[width=\textwidth]{figs/System.pdf} | ||
361 | @c \caption{Service interaction} | ||
362 | @c \label{fig:interaction} | ||
363 | @c \end{subfigure} | ||
364 | @c \end{center} | ||
365 | @c \caption{GNUnet's layered system architecture} | ||
366 | @c \end{figure} | ||
367 | |||
368 | The main service implementation runs as a standalone process in the | ||
369 | operating system and the client code runs as part of the client program, | ||
370 | so crashes of a client do not affect the service process or other clients. | ||
371 | The service and the clients communicate via a message protocol to be | ||
372 | defined and implemented by the programmer. | ||
373 | |||
374 | @node First Steps with GNUnet | ||
375 | @chapter First Steps with GNUnet | ||
376 | |||
377 | @menu | ||
378 | * Configure your peer:: | ||
379 | * Start a peer:: | ||
380 | * Monitor a peer:: | ||
381 | * Starting Two Peers by Hand:: | ||
382 | * Starting Peers Using the Testbed Service:: | ||
383 | @end menu | ||
384 | |||
385 | @node Configure your peer | ||
386 | @section Configure your peer | ||
387 | |||
388 | First of all we need to configure your peer. Each peer is started with | ||
389 | a configuration containing settings for GNUnet itself and its services. | ||
390 | This configuration is based on the default configuration shipped with | ||
391 | GNUnet and can be modified. The default configuration is located in the | ||
392 | @file{$PREFIX/share/gnunet/config.d} directory. When starting a peer, you | ||
393 | can specify a customized configuration using the the @command{-c} command | ||
394 | line switch when starting the ARM service and all other services. When | ||
395 | using a modified configuration the default values are loaded and only | ||
396 | values specified in the configuration file will replace the default | ||
397 | values. | ||
398 | |||
399 | Since we want to start additional peers later, we need some modifications | ||
400 | from the default configuration. We need to create a separate service | ||
401 | home and a file containing our modifications for this peer: | ||
402 | |||
403 | @example | ||
404 | $ mkdir ~/gnunet1/ | ||
405 | $ touch peer1.conf | ||
406 | @end example | ||
407 | |||
408 | @noindent | ||
409 | Now add the following lines to @file{peer1.conf} to use this directory. | ||
410 | For simplified usage we want to prevent the peer to connect to the GNUnet | ||
411 | network since this could lead to confusing output. This modifications | ||
412 | will replace the default settings: | ||
413 | |||
414 | @example | ||
415 | [PATHS] | ||
416 | # Use this directory to store GNUnet data | ||
417 | GNUNET_HOME = ~/gnunet1/ | ||
418 | [hostlist] | ||
419 | # prevent bootstrapping | ||
420 | SERVERS = | ||
421 | @end example | ||
422 | |||
423 | @node Start a peer | ||
424 | @section Start a peer | ||
425 | Each GNUnet instance (called peer) has an identity (peer ID) based on a | ||
426 | cryptographic public private key pair. The peer ID is the printable hash | ||
427 | of the public key. | ||
428 | |||
429 | GNUnet services are controlled by a master service, the so called | ||
430 | @dfn{Automatic Restart Manager} (ARM). ARM starts, stops and even | ||
431 | restarts services automatically or on demand when a client connects. | ||
432 | You interact with the ARM service using the @command{gnunet-arm} tool. | ||
433 | GNUnet can then be started with @command{gnunet-arm -s} and stopped with | ||
434 | @command{gnunet-arm -e}. An additional service not automatically started | ||
435 | can be started using @command{gnunet-arm -i <service name>} and stopped | ||
436 | using @command{gnunet-arm -k <servicename>}. | ||
437 | |||
438 | Once you have started your peer, you can use many other GNUnet commands | ||
439 | to interact with it. For example, you can run: | ||
440 | |||
441 | @example | ||
442 | $ gnunet-peerinfo -s | ||
443 | @end example | ||
444 | |||
445 | @noindent | ||
446 | to obtain the public key of your peer. | ||
447 | |||
448 | You should see an output containing the peer ID similar to: | ||
449 | |||
450 | @example | ||
451 | I am peer `0PA02UVRKQTS2C .. JL5Q78F6H0B1ACPV1CJI59MEQUMQCC5G'. | ||
452 | @end example | ||
453 | |||
454 | @node Monitor a peer | ||
455 | @section Monitor a peer | ||
456 | |||
457 | In this section, we will monitor the behaviour of our peer's DHT | ||
458 | service with respect to a specific key. First we will start | ||
459 | GNUnet and then start the DHT service and use the DHT monitor tool | ||
460 | to monitor the PUT and GET commands we issue ussing the | ||
461 | @command{gnunet-dht-put} and @command{gnunet-dht-get} commands. | ||
462 | Using the ``monitor'' line given below, you can observe the behavior | ||
463 | of your own peer's DHT with respect to the specified KEY: | ||
464 | |||
465 | @example | ||
466 | # start gnunet with all default services: | ||
467 | $ gnunet-arm -c ~/peer1.conf -s | ||
468 | # start DHT service: | ||
469 | $ gnunet-arm -c ~/peer1.conf -i dht | ||
470 | $ cd ~/gnunet/src/dht; | ||
471 | $ ./gnunet-dht-monitor -c ~/peer1.conf -k KEY | ||
472 | @end example | ||
473 | |||
474 | @noindent | ||
475 | Now open a separate terminal and change again to | ||
476 | the @file{gnunet/src/dht} directory: | ||
477 | |||
478 | @example | ||
479 | $ cd ~/gnunet/src/dht | ||
480 | # put VALUE under KEY in the DHT: | ||
481 | $ ./gnunet-dht-put -c ~/peer1.conf -k KEY -d VALUE | ||
482 | # get key KEY from the DHT: | ||
483 | $ ./gnunet/src/dht/gnunet-dht-get -c ~/peer1.conf -k KEY | ||
484 | # print statistics about current GNUnet state: | ||
485 | $ gnunet-statistics -c ~/peer1.conf | ||
486 | # print statistics about DHT service: | ||
487 | $ gnunet-statistics -c ~/peer1.conf -s dht | ||
488 | @end example | ||
489 | |||
490 | @node Starting Two Peers by Hand | ||
491 | @section Starting Two Peers by Hand | ||
492 | |||
493 | This section describes how to start two peers on the same machine by hand. | ||
494 | The process is rather painful, but the description is somewhat | ||
495 | instructive. In practice, you might prefer the automated method | ||
496 | (@pxref{Starting Peers Using the Testbed Service}). | ||
497 | |||
498 | @menu | ||
499 | * Setup a second peer:: | ||
500 | * Start the second peer and connect the peers:: | ||
501 | * How to connect manually:: | ||
502 | @end menu | ||
503 | |||
504 | @node Setup a second peer | ||
505 | @subsection Setup a second peer | ||
506 | We will now start a second peer on your machine. | ||
507 | For the second peer, you will need to manually create a modified | ||
508 | configuration file to avoid conflicts with ports and directories. | ||
509 | A peers configuration file is by default located | ||
510 | in @file{~/.gnunet/gnunet.conf}. This file is typically very short | ||
511 | or even empty as only the differences to the defaults need to be | ||
512 | specified. The defaults are located in many files in the | ||
513 | @file{$PREFIX/share/gnunet/config.d} directory. | ||
514 | |||
515 | To configure the second peer, use the files | ||
516 | @file{$PREFIX/share/gnunet/config.d} as a template for your main | ||
517 | configuration file: | ||
518 | |||
519 | @example | ||
520 | $ cat $PREFIX/share/gnunet/config.d/*.conf > peer2.conf | ||
521 | @end example | ||
522 | |||
523 | @noindent | ||
524 | Now you have to edit @file{peer2.conf} and change: | ||
525 | |||
526 | @itemize | ||
527 | @item @code{GNUNET\_TEST\_HOME} under @code{PATHS} | ||
528 | @item Every (uncommented) value for ``@code{PORT}'' (add 10000) in any | ||
529 | section (the option may be commented out if @code{PORT} is | ||
530 | prefixed by "\#", in this case, UNIX domain sockets are used | ||
531 | and the PORT option does not need to be touched) | ||
532 | @item Every value for ``@code{UNIXPATH}'' in any section | ||
533 | (e.g. by adding a "-p2" suffix) | ||
534 | @end itemize | ||
535 | |||
536 | to a fresh, unique value. Make sure that the PORT numbers stay | ||
537 | below 65536. From now on, whenever you interact with the second peer, | ||
538 | you need to specify @command{-c peer2.conf} as an additional | ||
539 | command line argument. | ||
540 | |||
541 | Now, generate the 2nd peer's private key: | ||
542 | |||
543 | @example | ||
544 | $ gnunet-peerinfo -s -c peer2.conf | ||
545 | @end example | ||
546 | |||
547 | @noindent | ||
548 | This may take a while, generate entropy using your keyboard or mouse | ||
549 | as needed. Also, make sure the output is different from the | ||
550 | gnunet-peerinfo output for the first peer (otherwise you made an | ||
551 | error in the configuration). | ||
552 | |||
553 | @node Start the second peer and connect the peers | ||
554 | @subsection Start the second peer and connect the peers | ||
555 | |||
556 | Then, you can start a second peer using: | ||
557 | |||
558 | @example | ||
559 | $ gnunet-arm -c peer2.conf -s | ||
560 | $ gnunet-arm -c peer2.conf -i dht | ||
561 | $ ~/gnunet/src/dht/gnunet-dht-put -c peer2.conf -k KEY -d VALUE | ||
562 | $ ~/gnunet/src/dht/gnunet-dht-get -c peer2.conf -k KEY | ||
563 | @end example | ||
564 | |||
565 | If you want the two peers to connect, you have multiple options: | ||
566 | |||
567 | @itemize | ||
568 | @item UDP neighbour discovery (automatic) | ||
569 | @item Setup a bootstrap server | ||
570 | @item Connect manually | ||
571 | @end itemize | ||
572 | |||
573 | To setup peer 1 as bootstrapping server change the configuration of | ||
574 | the first one to be a hostlist server by adding the following lines to | ||
575 | @file{peer1.conf} to enable bootstrapping server: | ||
576 | |||
577 | @example | ||
578 | [hostlist] | ||
579 | OPTIONS = -p | ||
580 | @end example | ||
581 | |||
582 | @noindent | ||
583 | Then change @file{peer2.conf} and replace the ``@code{SERVERS}'' | ||
584 | line in the ``@code{[hostlist]}'' section with | ||
585 | ``@code{http://localhost:8080/}''. Restart both peers using: | ||
586 | |||
587 | @example | ||
588 | # stop first peer | ||
589 | $ gnunet-arm -c peer1.conf -e | ||
590 | # start first peer | ||
591 | $ gnunet-arm -c peer1.conf -s | ||
592 | # start second peer | ||
593 | $ gnunet-arm -c peer2.conf -s | ||
594 | @end example | ||
595 | |||
596 | @noindent | ||
597 | Note that if you start your peers without changing these settings, they | ||
598 | will use the ``global'' hostlist servers of the GNUnet P2P network and | ||
599 | likely connect to those peers. At that point, debugging might become | ||
600 | tricky as you're going to be connected to many more peers and would | ||
601 | likely observe traffic and behaviors that are not explicitly controlled | ||
602 | by you. | ||
603 | |||
604 | @node How to connect manually | ||
605 | @subsection How to connect manually | ||
606 | |||
607 | If you want to use the @code{peerinfo} tool to connect your | ||
608 | peers, you should: | ||
609 | |||
610 | @itemize | ||
611 | @item Set @code{FORCESTART = NO} in section @code{hostlist} | ||
612 | (to not connect to the global GNUnet) | ||
613 | @item Start both peers running @command{gnunet-arm -c peer1.conf -s} | ||
614 | and @command{gnunet-arm -c peer2.conf -s} | ||
615 | @item Get @code{HELLO} message of the first peer running | ||
616 | @command{gnunet-peerinfo -c peer1.conf -g} | ||
617 | @item Give the output to the second peer by running | ||
618 | @command{gnunet-peerinfo -c peer2.conf -p '<output>'} | ||
619 | @end itemize | ||
620 | |||
621 | Check that they are connected using @command{gnunet-core -c peer1.conf}, | ||
622 | which should give you the other peer's peer identity: | ||
623 | |||
624 | @example | ||
625 | $ gnunet-core -c peer1.conf | ||
626 | Peer `9TVUCS8P5A7ILLBGO6 [...shortened...] 1KNBJ4NGCHP3JPVULDG' | ||
627 | @end example | ||
628 | |||
629 | @node Starting Peers Using the Testbed Service | ||
630 | @section Starting Peers Using the Testbed Service | ||
631 | @c \label{sec:testbed} | ||
632 | |||
633 | GNUnet's testbed service is used for testing scenarios where | ||
634 | a number of peers are to be started. The testbed can manage peers | ||
635 | on a single host or on multiple hosts in a distributed fashion. | ||
636 | On a single affordable computer, it should be possible to run | ||
637 | around tens of peers without drastically increasing the load on the | ||
638 | system. | ||
639 | |||
640 | The testbed service can be access through its API | ||
641 | @file{include/gnunet\_testbed\_service.h}. The API provides many | ||
642 | routines for managing a group of peers. It also provides a helper | ||
643 | function @code{GNUNET\_TESTBED\_test\_run()} to quickly setup a | ||
644 | minimalistic testing environment on a single host. | ||
645 | |||
646 | This function takes a configuration file which will be used as a | ||
647 | template configuration for the peers. The testbed takes care of | ||
648 | modifying relevant options in the peers' configuration such as | ||
649 | @code{SERVICEHOME}, @code{PORT}, @code{UNIXPATH} to unique values | ||
650 | so that peers run without running into conflicts. It also checks | ||
651 | and assigns the ports in configurations only if they are free. | ||
652 | |||
653 | Additionally, the testbed service also reads its options from the | ||
654 | same configuration file. Various available options and details | ||
655 | about them can be found in the testbed default configuration file | ||
656 | @file{src/testbed/testbed.conf}. | ||
657 | |||
658 | With the testbed API, a sample test case can be structured as follows: | ||
659 | |||
660 | @example | ||
661 | @verbatiminclude testbed_test.c | ||
662 | @end example | ||
663 | |||
664 | @noindent | ||
665 | The source code for the above listing can be found at | ||
666 | @uref{https://gnunet.org/git/gnunet.git/tree/doc/testbed_test.c} | ||
667 | or in the @file{doc/} folder of your repository check-out. | ||
668 | After installing GNUnet, the above source code can be compiled as: | ||
669 | |||
670 | @example | ||
671 | $ export CPPFLAGS="-I/path/to/gnunet/headers" | ||
672 | $ export LDFLAGS="-L/path/to/gnunet/libraries" | ||
673 | $ gcc $CPPFLAGS $LDFLAGS -o testbed-test testbed_test.c \ | ||
674 | -lgnunettestbed -lgnunetdht -lgnunetutil | ||
675 | # Generate (empty) configuration | ||
676 | $ touch template.conf | ||
677 | # run it (press CTRL-C to stop) | ||
678 | $ ./testbed-test | ||
679 | @end example | ||
680 | |||
681 | @noindent | ||
682 | The @code{CPPFLAGS} and @code{LDFLAGS} are necessary if GNUnet | ||
683 | is installed into a different directory other than @file{/usr/local}. | ||
684 | |||
685 | All of testbed API's peer management functions treat management | ||
686 | actions as operations and return operation handles. It is expected | ||
687 | that the operations begin immediately, but they may get delayed (to | ||
688 | balance out load on the system). The program using the API then has | ||
689 | to take care of marking the operation as ``done'' so that its | ||
690 | associated resources can be freed immediately and other waiting | ||
691 | operations can be executed. Operations will be canceled if they are | ||
692 | marked as ``done'' before their completion. | ||
693 | |||
694 | An operation is treated as completed when it succeeds or fails. | ||
695 | Completion of an operation is either conveyed as events through | ||
696 | @i{controller event callback} or through respective operation | ||
697 | completion callbacks. In functions which support completion | ||
698 | notification through both controller event callback and operation | ||
699 | completion callback, first the controller event callback will be | ||
700 | called. If the operation is not marked as done in that callback | ||
701 | or if the callback is given as NULL when creating the operation, | ||
702 | the operation completion callback will be called. The API | ||
703 | documentation shows which event are to be expected in the | ||
704 | controller event notifications. It also documents any exceptional | ||
705 | behaviour. | ||
706 | |||
707 | Once the peers are started, test cases often need to connect | ||
708 | some of the peers' services. Normally, opening a connect to | ||
709 | a peer's service requires the peer's configuration. While using | ||
710 | testbed, the testbed automatically generates per-peer configuration. | ||
711 | Accessing those configurations directly through file system is | ||
712 | discouraged as their locations are dynamically created and will be | ||
713 | different among various runs of testbed. To make access to these | ||
714 | configurations easy, testbed API provides the function | ||
715 | @code{GNUNET\_TESTBED\_service\_connect()}. This function fetches | ||
716 | the configuration of a given peer and calls the @i{Connect Adapter}. | ||
717 | In the example code, it is the @code{dht\_ca}. A connect adapter is | ||
718 | expected to open the connection to the needed service by using the | ||
719 | provided configuration and return the created service connection handle. | ||
720 | Successful connection to the needed service is signaled through | ||
721 | @code{service\_connect\_comp\_cb}. | ||
722 | |||
723 | A dual to connect adapter is the @i{Disconnect Adapter}. This callback | ||
724 | is called after the connect adapter has been called when the operation | ||
725 | from @code{GNUNET\_TESTBED\_service\_connect()} is marked as ``done''. | ||
726 | It has to disconnect from the service with the provided service | ||
727 | handle (@code{op\_result}). | ||
728 | |||
729 | Exercise: Find out how many peers you can run on your system. | ||
730 | |||
731 | Exercise: Find out how to create a 2D torus topology by changing the | ||
732 | options in the configuration file. | ||
733 | See @uref{https://gnunet.org/supported-topologies}, then use the | ||
734 | DHT API to store and retrieve values in the network. | ||
735 | |||
736 | @node Developing Applications | ||
737 | @chapter Developing Applications | ||
738 | |||
739 | @menu | ||
740 | * gnunet-ext:: | ||
741 | * Adapting the Template:: | ||
742 | * Writing a Client Application:: | ||
743 | * Writing a Service:: | ||
744 | * Interacting directly with other Peers using the CORE Service:: | ||
745 | * Storing peer-specific data using the PEERSTORE service:: | ||
746 | * Using the DHT:: | ||
747 | * Debugging with gnunet-arm:: | ||
748 | @end menu | ||
749 | |||
750 | @node gnunet-ext | ||
751 | @section gnunet-ext | ||
752 | To develop a new peer-to-peer application or to extend GNUnet we provide | ||
753 | a template build system for writing GNUnet extensions in C. It can be | ||
754 | obtained as follows: | ||
755 | |||
756 | @example | ||
757 | $ git clone https://gnunet.org/git/gnunet-ext | ||
758 | $ cd gnunet-ext/ | ||
759 | $ ./bootstrap | ||
760 | $ ./configure --prefix=$PREFIX --with-gnunet=$PREFIX | ||
761 | $ make | ||
762 | $ make install | ||
763 | $ make check | ||
764 | @end example | ||
765 | |||
766 | @noindent | ||
767 | The GNUnet ext template includes examples and a working buildsystem | ||
768 | for a new GNUnet service. A common GNUnet service consists of the | ||
769 | following parts which will be discussed in detail in the remainder | ||
770 | of this document. The functionality of a GNUnet service is implemented in: | ||
771 | |||
772 | @itemize | ||
773 | @item the GNUnet service (gnunet-ext/src/ext/gnunet-service-ext.c) | ||
774 | @item the client API (gnunet-ext/src/ext/ext_api.c) | ||
775 | @item the client application using the service API | ||
776 | (gnunet-ext/src/ext/gnunet-ext.c) | ||
777 | @end itemize | ||
778 | |||
779 | The interfaces for these entities are defined in: | ||
780 | |||
781 | @itemize | ||
782 | @item client API interface (gnunet-ext/src/ext/ext.h) | ||
783 | @item the service interface (gnunet-ext/src/include/gnunet_service_SERVICE.h) | ||
784 | @item the P2P protocol (gnunet-ext/src/include/gnunet_protocols_ext.h) | ||
785 | @end itemize | ||
786 | |||
787 | |||
788 | In addition the ext systems provides: | ||
789 | |||
790 | @itemize | ||
791 | @item a test testing the API (gnunet-ext/src/ext/test_ext_api.c) | ||
792 | @item a configuration template for the service | ||
793 | (gnunet-ext/src/ext/ext.conf.in) | ||
794 | @end itemize | ||
795 | |||
796 | @node Adapting the Template | ||
797 | @section Adapting the Template | ||
798 | |||
799 | The first step for writing any extension with a new service is to | ||
800 | ensure that the @file{ext.conf.in} file contains entries for the | ||
801 | @code{UNIXPATH}, @code{PORT} and @code{BINARY} for the service in a | ||
802 | section named after the service. | ||
803 | |||
804 | If you want to adapt the template rename the @file{ext.conf.in} to | ||
805 | match your services name, you have to modify the @code{AC\_OUTPUT} | ||
806 | section in @file{configure.ac} in the @file{gnunet-ext} root. | ||
807 | |||
808 | @node Writing a Client Application | ||
809 | @section Writing a Client Application | ||
810 | |||
811 | When writing any client application (for example, a command-line | ||
812 | tool), the basic structure is to start with the | ||
813 | @code{GNUNET\_PROGRAM\_run} function. This function will parse | ||
814 | command-line options, setup the scheduler and then invoke the | ||
815 | @code{run} function (with the remaining non-option arguments) | ||
816 | and a handle to the parsed configuration (and the configuration | ||
817 | file name that was used, which is typically not needed): | ||
818 | |||
819 | @example | ||
820 | @verbatiminclude tutorial-examples/001.c | ||
821 | @end example | ||
822 | |||
823 | @menu | ||
824 | * Handling command-line options:: | ||
825 | * Writing a Client Library:: | ||
826 | * Writing a user interface:: | ||
827 | @end menu | ||
828 | |||
829 | @node Handling command-line options | ||
830 | @subsection Handling command-line options | ||
831 | |||
832 | Options can then be added easily by adding global variables and | ||
833 | expanding the @code{options} array. For example, the following would | ||
834 | add a string-option and a binary flag (defaulting to @code{NULL} and | ||
835 | @code{GNUNET\_NO} respectively): | ||
836 | |||
837 | @example | ||
838 | @verbatiminclude tutorial-examples/002.c | ||
839 | @end example | ||
840 | |||
841 | Issues such as displaying some helpful text describing options using | ||
842 | the @code{--help} argument and error handling are taken care of when | ||
843 | using this approach. Other @code{GNUNET\_GETOPT\_}-functions can be used | ||
844 | to obtain integer value options, increment counters, etc. You can | ||
845 | even write custom option parsers for special circumstances not covered | ||
846 | by the available handlers. To check if an argument was specified by the | ||
847 | user you initialize the variable with a specific value (e.g. NULL for | ||
848 | a string and GNUNET\_SYSERR for a integer) and check after parsing | ||
849 | happened if the values were modified. | ||
850 | |||
851 | Inside the @code{run} method, the program would perform the | ||
852 | application-specific logic, which typically involves initializing and | ||
853 | using some client library to interact with the service. The client | ||
854 | library is supposed to implement the IPC whereas the service provides | ||
855 | more persistent P2P functions. | ||
856 | |||
857 | Exercise: Add a few command-line options and print them inside | ||
858 | of @code{run}. What happens if the user gives invalid arguments? | ||
859 | |||
860 | @node Writing a Client Library | ||
861 | @subsection Writing a Client Library | ||
862 | |||
863 | The first and most important step in writing a client library is to | ||
864 | decide on an API for the library. Typical API calls include | ||
865 | connecting to the service, performing application-specific requests | ||
866 | and cleaning up. Many examples for such service APIs can be found | ||
867 | in the @file{gnunet/src/include/gnunet\_*\_service.h} files. | ||
868 | |||
869 | Then, a client-service protocol needs to be designed. This typically | ||
870 | involves defining various message formats in a header that will be | ||
871 | included by both the service and the client library (but is otherwise | ||
872 | not shared and hence located within the service's directory and not | ||
873 | installed by @command{make install}). Each message must start with a | ||
874 | @code{struct GNUNET\_MessageHeader} and must be shorter than 64k. By | ||
875 | convention, all fields in IPC (and P2P) messages must be in big-endian | ||
876 | format (and thus should be read using @code{ntohl} and similar | ||
877 | functions and written using @code{htonl} and similar functions). | ||
878 | Unique message types must be defined for each message struct in the | ||
879 | @file{gnunet\_protocols.h} header (or an extension-specific include | ||
880 | file). | ||
881 | |||
882 | @menu | ||
883 | * Connecting to the Service:: | ||
884 | * Sending messages:: | ||
885 | * Receiving Replies from the Service:: | ||
886 | @end menu | ||
887 | |||
888 | @node Connecting to the Service | ||
889 | @subsubsection Connecting to the Service | ||
890 | |||
891 | Before a client library can implement the application-specific protocol | ||
892 | with the service, a connection must be created: | ||
893 | |||
894 | @example | ||
895 | @verbatiminclude tutorial-examples/003.c | ||
896 | @end example | ||
897 | |||
898 | @noindent | ||
899 | As a result a @code{GNUNET\_MQ\_Handle} is returned | ||
900 | which can to used henceforth to transmit messages to the service. | ||
901 | The complete MQ API can be found in @file{gnunet\_mq\_lib.h}. | ||
902 | The @code{hanlders} array in the example above is incomplete. | ||
903 | Here is where you will define which messages you expect to | ||
904 | receive from the service, and which functions handle them. | ||
905 | The @code{error\_cb} is a function that is to be called whenever | ||
906 | there are errors communicating with the service. | ||
907 | |||
908 | @node Sending messages | ||
909 | @subsubsection Sending messages | ||
910 | |||
911 | In GNUnet, messages are always sent beginning with a | ||
912 | @code{struct GNUNET\_MessageHeader} in big endian format. | ||
913 | This header defines the size and the type of the | ||
914 | message, the payload follows after this header. | ||
915 | |||
916 | @example | ||
917 | @verbatiminclude tutorial-examples/004.c | ||
918 | @end example | ||
919 | |||
920 | @noindent | ||
921 | Existing message types are defined in @file{gnunet\_protocols.h}. | ||
922 | A common way to create a message is with an envelope: | ||
923 | |||
924 | @example | ||
925 | @verbatiminclude tutorial-examples/005.c | ||
926 | @end example | ||
927 | |||
928 | @noindent | ||
929 | Exercise: Define a message struct that includes a 32-bit | ||
930 | unsigned integer in addition to the standard GNUnet MessageHeader. | ||
931 | Add a C struct and define a fresh protocol number for your message. | ||
932 | Protocol numbers in gnunet-ext are defined | ||
933 | in @file{gnunet-ext/src/include/gnunet_protocols_ext.h} | ||
934 | |||
935 | Exercise: Find out how you can determine the number of messages | ||
936 | in a message queue. | ||
937 | |||
938 | Exercise: Find out how you can determine when a message you | ||
939 | have queued was actually transmitted. | ||
940 | |||
941 | Exercise: Define a helper function to transmit a 32-bit | ||
942 | unsigned integer (as payload) to a service using some given client | ||
943 | handle. | ||
944 | |||
945 | @node Receiving Replies from the Service | ||
946 | @subsubsection Receiving Replies from the Service | ||
947 | |||
948 | Clients can receive messages from the service using the handlers | ||
949 | specified in the @code{handlers} array we specified when connecting | ||
950 | to the service. Entries in the the array are usually created using | ||
951 | one of two macros, depending on whether the message is fixed size | ||
952 | or variable size. Variable size messages are managed using two | ||
953 | callbacks, one to check that the message is well-formed, the other | ||
954 | to actually process the message. Fixed size messages are fully | ||
955 | checked by the MQ-logic, and thus only need to provide the handler | ||
956 | to process the message. Note that the prefixes @code{check\_} | ||
957 | and @code{handle\_} are mandatory. | ||
958 | |||
959 | @example | ||
960 | @verbatiminclude tutorial-examples/006.c | ||
961 | @end example | ||
962 | |||
963 | @noindent | ||
964 | Exercise: Expand your helper function to receive a response message | ||
965 | (for example, containing just the @code{struct GNUnet MessageHeader} | ||
966 | without any payload). Upon receiving the service's response, you | ||
967 | should call a callback provided to your helper function's API. | ||
968 | |||
969 | Exercise: Figure out where you can pass values to the | ||
970 | closures (@code{cls}). | ||
971 | |||
972 | @node Writing a user interface | ||
973 | @subsection Writing a user interface | ||
974 | |||
975 | Given a client library, all it takes to access a service now is to | ||
976 | combine calls to the client library with parsing command-line | ||
977 | options. | ||
978 | |||
979 | Exercise: Call your client API from your @code{run()} method in your | ||
980 | client application to send a request to the service. For example, | ||
981 | send a 32-bit integer value based on a number given at the | ||
982 | command-line to the service. | ||
983 | |||
984 | @node Writing a Service | ||
985 | @section Writing a Service | ||
986 | |||
987 | Before you can test the client you've written so far, you'll | ||
988 | need to also implement the corresponding service. | ||
989 | |||
990 | @menu | ||
991 | * Code Placement:: | ||
992 | * Starting a Service:: | ||
993 | @end menu | ||
994 | |||
995 | @node Code Placement | ||
996 | @subsection Code Placement | ||
997 | |||
998 | New services are placed in their own subdirectory under | ||
999 | @file{gnunet/src}. This subdirectory should contain the API | ||
1000 | implementation file @file{SERVICE\_api.c}, the description of | ||
1001 | the client-service protocol @file{SERVICE.h} and P2P protocol | ||
1002 | @file{SERVICE\_protocol.h}, the implementation of the service itself | ||
1003 | @file{gnunet-service-SERVICE.h} and several files for tests, | ||
1004 | including test code and configuration files. | ||
1005 | |||
1006 | @node Starting a Service | ||
1007 | @subsection Starting a Service | ||
1008 | |||
1009 | The key API definition for creating a service is the | ||
1010 | @code{GNUNET\_SERVICE\_MAIN} macro: | ||
1011 | |||
1012 | @example | ||
1013 | @verbatiminclude tutorial-examples/007.c | ||
1014 | @end example | ||
1015 | |||
1016 | @noindent | ||
1017 | In addition to the service name and flags, the macro takes three | ||
1018 | functions, typically called @code{run}, @code{client\_connect\_cb} and | ||
1019 | @code{client\_disconnect\_cb} as well as an array of message handlers | ||
1020 | that will be called for incoming messages from clients. | ||
1021 | |||
1022 | A minimal version of the three central service funtions would look | ||
1023 | like this: | ||
1024 | |||
1025 | @example | ||
1026 | @verbatiminclude tutorial-examples/008.c | ||
1027 | @end example | ||
1028 | |||
1029 | @noindent | ||
1030 | Exercise: Write a stub service that processes no messages at all | ||
1031 | in your code. Create a default configuration for it, integrate it | ||
1032 | with the build system and start the service from | ||
1033 | @command{gnunet-service-arm} using @command{gnunet-arm -i NAME}. | ||
1034 | |||
1035 | Exercise: Figure out how to set the closure (@code{cls}) for handlers | ||
1036 | of a service. | ||
1037 | |||
1038 | Exercise: Figure out how to send messages from the service back to the | ||
1039 | client. | ||
1040 | |||
1041 | Each handler function in the service @b{must} eventually (possibly in some | ||
1042 | asynchronous continuation) call | ||
1043 | @code{GNUNET\_SERVICE\_client\_continue()}. Only after this call | ||
1044 | additional messages from the same client may | ||
1045 | be processed. This way, the service can throttle processing messages | ||
1046 | from the same client. | ||
1047 | |||
1048 | Exercise: Change the service to ``handle'' the message from your | ||
1049 | client (for now, by printing a message). What happens if you | ||
1050 | forget to call @code{GNUNET\_SERVICE\_client\_continue()}? | ||
1051 | |||
1052 | @node Interacting directly with other Peers using the CORE Service | ||
1053 | @section Interacting directly with other Peers using the CORE Service | ||
1054 | |||
1055 | FIXME: This section still needs to be updated to the lastest API! | ||
1056 | |||
1057 | One of the most important services in GNUnet is the @code{CORE} service | ||
1058 | managing connections between peers and handling encryption between peers. | ||
1059 | |||
1060 | One of the first things any service that extends the P2P protocol | ||
1061 | typically does is connect to the @code{CORE} service using: | ||
1062 | |||
1063 | @example | ||
1064 | @verbatiminclude tutorial-examples/009.c | ||
1065 | @end example | ||
1066 | |||
1067 | @menu | ||
1068 | * New P2P connections:: | ||
1069 | * Receiving P2P Messages:: | ||
1070 | * Sending P2P Messages:: | ||
1071 | * End of P2P connections:: | ||
1072 | @end menu | ||
1073 | |||
1074 | @node New P2P connections | ||
1075 | @subsection New P2P connections | ||
1076 | |||
1077 | Before any traffic with a different peer can be exchanged, the peer must | ||
1078 | be known to the service. This is notified by the @code{CORE} | ||
1079 | @code{connects} callback, which communicates the identity of the new | ||
1080 | peer to the service: | ||
1081 | |||
1082 | @example | ||
1083 | @verbatiminclude tutorial-examples/010.c | ||
1084 | @end example | ||
1085 | |||
1086 | @noindent | ||
1087 | Note that whatever you return from @code{connects} is given as the | ||
1088 | @i{cls} argument to the message handlers for messages from | ||
1089 | the respective peer. | ||
1090 | |||
1091 | Exercise: Create a service that connects to the @code{CORE}. Then | ||
1092 | start (and connect) two peers and print a message once your connect | ||
1093 | callback is invoked. | ||
1094 | |||
1095 | @node Receiving P2P Messages | ||
1096 | @subsection Receiving P2P Messages | ||
1097 | |||
1098 | To receive messages from @code{CORE}, you pass the desired | ||
1099 | @i{handlers} to the @code{GNUNET\_CORE\_connect()} function, | ||
1100 | just as we showed for services. | ||
1101 | |||
1102 | It is your responsibility to process messages fast enough or | ||
1103 | to implement flow control. If an application does not process | ||
1104 | CORE messages fast enough, CORE will randomly drop messages | ||
1105 | to not keep a very long queue in memory. | ||
1106 | |||
1107 | Exercise: Start one peer with a new service that has a message | ||
1108 | handler and start a second peer that only has your ``old'' service | ||
1109 | without message handlers. Which ``connect'' handlers are invoked when | ||
1110 | the two peers are connected? Why? | ||
1111 | |||
1112 | @node Sending P2P Messages | ||
1113 | @subsection Sending P2P Messages | ||
1114 | |||
1115 | You can transmit messages to other peers using the @i{mq} you were | ||
1116 | given during the @code{connect} callback. Note that the @i{mq} | ||
1117 | automatically is released upon @code{disconnect} and that you must | ||
1118 | not use it afterwards. | ||
1119 | |||
1120 | It is your responsibility to not over-fill the message queue, GNUnet | ||
1121 | will send the messages roughly in the order given as soon as possible. | ||
1122 | |||
1123 | Exercise: Write a service that upon connect sends messages as | ||
1124 | fast as possible to the other peer (the other peer should run a | ||
1125 | service that ``processes'' those messages). How fast is the | ||
1126 | transmission? Count using the STATISTICS service on both ends. Are | ||
1127 | messages lost? How can you transmit messages faster? What happens if | ||
1128 | you stop the peer that is receiving your messages? | ||
1129 | |||
1130 | @node End of P2P connections | ||
1131 | @subsection End of P2P connections | ||
1132 | |||
1133 | If a message handler returns @code{GNUNET\_SYSERR}, the remote | ||
1134 | peer shuts down or there is an unrecoverable network | ||
1135 | disconnection, CORE notifies the service that the peer disconnected. | ||
1136 | After this notification no more messages will be received from the | ||
1137 | peer and the service is no longer allowed to send messages to the peer. | ||
1138 | The disconnect callback looks like the following: | ||
1139 | |||
1140 | @example | ||
1141 | @verbatiminclude tutorial-examples/011.c | ||
1142 | @end example | ||
1143 | |||
1144 | @noindent | ||
1145 | Exercise: Fix your service to handle peer disconnects. | ||
1146 | |||
1147 | @node Storing peer-specific data using the PEERSTORE service | ||
1148 | @section Storing peer-specific data using the PEERSTORE service | ||
1149 | |||
1150 | GNUnet's PEERSTORE service offers a persistorage for arbitrary | ||
1151 | peer-specific data. Other GNUnet services can use the PEERSTORE | ||
1152 | to store, retrieve and monitor data records. Each data record | ||
1153 | stored with PEERSTORE contains the following fields: | ||
1154 | |||
1155 | @itemize | ||
1156 | @item subsystem: Name of the subsystem responsible for the record. | ||
1157 | @item peerid: Identity of the peer this record is related to. | ||
1158 | @item key: a key string identifying the record. | ||
1159 | @item value: binary record value. | ||
1160 | @item expiry: record expiry date. | ||
1161 | @end itemize | ||
1162 | |||
1163 | The first step is to start a connection to the PEERSTORE service: | ||
1164 | @example | ||
1165 | @verbatiminclude tutorial-examples/012.c | ||
1166 | @end example | ||
1167 | |||
1168 | The service handle @code{peerstore_handle} will be needed for | ||
1169 | all subsequent PEERSTORE operations. | ||
1170 | |||
1171 | @menu | ||
1172 | * Storing records:: | ||
1173 | * Retrieving records:: | ||
1174 | * Monitoring records:: | ||
1175 | * Disconnecting from PEERSTORE:: | ||
1176 | @end menu | ||
1177 | |||
1178 | @node Storing records | ||
1179 | @subsection Storing records | ||
1180 | |||
1181 | To store a new record, use the following function: | ||
1182 | |||
1183 | @example | ||
1184 | @verbatiminclude tutorial-examples/013.c | ||
1185 | @end example | ||
1186 | |||
1187 | @noindent | ||
1188 | The @code{options} parameter can either be | ||
1189 | @code{GNUNET_PEERSTORE_STOREOPTION_MULTIPLE} which means that multiple | ||
1190 | values can be stored under the same key combination | ||
1191 | (subsystem, peerid, key), or @code{GNUNET_PEERSTORE_STOREOPTION_REPLACE} | ||
1192 | which means that PEERSTORE will replace any existing values under the | ||
1193 | given key combination (subsystem, peerid, key) with the new given value. | ||
1194 | |||
1195 | The continuation function @code{cont} will be called after the store | ||
1196 | request is successfully sent to the PEERSTORE service. This does not | ||
1197 | guarantee that the record is successfully stored, only that it was | ||
1198 | received by the service. | ||
1199 | |||
1200 | The @code{GNUNET_PEERSTORE_store} function returns a handle to the store | ||
1201 | operation. This handle can be used to cancel the store operation only | ||
1202 | before the continuation function is called: | ||
1203 | |||
1204 | @example | ||
1205 | @verbatiminclude tutorial-examples/013.1.c | ||
1206 | @end example | ||
1207 | |||
1208 | @node Retrieving records | ||
1209 | @subsection Retrieving records | ||
1210 | |||
1211 | To retrieve stored records, use the following function: | ||
1212 | |||
1213 | @example | ||
1214 | @verbatiminclude tutorial-examples/014.c | ||
1215 | @end example | ||
1216 | |||
1217 | @noindent | ||
1218 | The values of @code{peer} and @code{key} can be @code{NULL}. This | ||
1219 | allows the iteration over values stored under any of the following | ||
1220 | key combinations: | ||
1221 | |||
1222 | @itemize | ||
1223 | @item (subsystem) | ||
1224 | @item (subsystem, peerid) | ||
1225 | @item (subsystem, key) | ||
1226 | @item (subsystem, peerid, key) | ||
1227 | @end itemize | ||
1228 | |||
1229 | The @code{callback} function will be called once with each retrieved | ||
1230 | record and once more with a @code{NULL} record to signal the end of | ||
1231 | results. | ||
1232 | |||
1233 | The @code{GNUNET_PEERSTORE_iterate} function returns a handle to the | ||
1234 | iterate operation. This handle can be used to cancel the iterate | ||
1235 | operation only before the callback function is called with a | ||
1236 | @code{NULL} record. | ||
1237 | |||
1238 | @node Monitoring records | ||
1239 | @subsection Monitoring records | ||
1240 | |||
1241 | PEERSTORE offers the functionality of monitoring for new records | ||
1242 | stored under a specific key combination (subsystem, peerid, key). | ||
1243 | To start the monitoring, use the following function: | ||
1244 | |||
1245 | @example | ||
1246 | @verbatiminclude tutorial-examples/015.c | ||
1247 | @end example | ||
1248 | |||
1249 | @noindent | ||
1250 | Whenever a new record is stored under the given key combination, | ||
1251 | the @code{callback} function will be called with this new | ||
1252 | record. This will continue until the connection to the PEERSTORE | ||
1253 | service is broken or the watch operation is canceled: | ||
1254 | |||
1255 | @example | ||
1256 | @verbatiminclude tutorial-examples/016.c | ||
1257 | @end example | ||
1258 | |||
1259 | @node Disconnecting from PEERSTORE | ||
1260 | @subsection Disconnecting from PEERSTORE | ||
1261 | |||
1262 | When the connection to the PEERSTORE service is no longer needed, | ||
1263 | disconnect using the following function: | ||
1264 | |||
1265 | @example | ||
1266 | @verbatiminclude tutorial-examples/017.c | ||
1267 | @end example | ||
1268 | |||
1269 | @noindent | ||
1270 | If the @code{sync_first} flag is set to @code{GNUNET_YES}, | ||
1271 | the API will delay the disconnection until all store requests | ||
1272 | are received by the PEERSTORE service. Otherwise, it will | ||
1273 | disconnect immediately. | ||
1274 | |||
1275 | @node Using the DHT | ||
1276 | @section Using the DHT | ||
1277 | |||
1278 | The DHT allows to store data so other peers in the P2P network can | ||
1279 | access it and retrieve data stored by any peers in the network. | ||
1280 | This section will explain how to use the DHT. Of course, the first | ||
1281 | thing to do is to connect to the DHT service: | ||
1282 | |||
1283 | @example | ||
1284 | @verbatiminclude tutorial-examples/018.c | ||
1285 | @end example | ||
1286 | |||
1287 | @noindent | ||
1288 | The second parameter indicates how many requests in parallel to expect. | ||
1289 | It is not a hard limit, but a good approximation will make the DHT more | ||
1290 | efficient. | ||
1291 | |||
1292 | @menu | ||
1293 | * Storing data in the DHT:: | ||
1294 | * Obtaining data from the DHT:: | ||
1295 | * Implementing a block plugin:: | ||
1296 | * Monitoring the DHT:: | ||
1297 | @end menu | ||
1298 | |||
1299 | @node Storing data in the DHT | ||
1300 | @subsection Storing data in the DHT | ||
1301 | Since the DHT is a dynamic environment (peers join and leave frequently) | ||
1302 | the data that we put in the DHT does not stay there indefinitely. It is | ||
1303 | important to ``refresh'' the data periodically by simply storing it | ||
1304 | again, in order to make sure other peers can access it. | ||
1305 | |||
1306 | The put API call offers a callback to signal that the PUT request has been | ||
1307 | sent. This does not guarantee that the data is accessible to others peers, | ||
1308 | or even that is has been stored, only that the service has requested to | ||
1309 | a neighboring peer the retransmission of the PUT request towards its final | ||
1310 | destination. Currently there is no feedback about whether or not the data | ||
1311 | has been sucessfully stored or where it has been stored. In order to | ||
1312 | improve the availablilty of the data and to compensate for possible | ||
1313 | errors, peers leaving and other unfavorable events, just make several | ||
1314 | PUT requests! | ||
1315 | |||
1316 | @example | ||
1317 | @verbatiminclude tutorial-examples/019.c | ||
1318 | @end example | ||
1319 | |||
1320 | @noindent | ||
1321 | Exercise: Store a value in the DHT periodically to make sure it | ||
1322 | is available over time. You might consider using the function | ||
1323 | @code{GNUNET\_SCHEDULER\_add\_delayed} and call | ||
1324 | @code{GNUNET\_DHT\_put} from inside a helper function. | ||
1325 | |||
1326 | @node Obtaining data from the DHT | ||
1327 | @subsection Obtaining data from the DHT | ||
1328 | |||
1329 | As we saw in the previous example, the DHT works in an asynchronous mode. | ||
1330 | Each request to the DHT is executed ``in the background'' and the API | ||
1331 | calls return immediately. In order to receive results from the DHT, the | ||
1332 | API provides a callback. Once started, the request runs in the service, | ||
1333 | the service will try to get as many results as possible (filtering out | ||
1334 | duplicates) until the timeout expires or we explicitly stop the request. | ||
1335 | It is possible to give a ``forever'' timeout with | ||
1336 | @code{GNUNET\_TIME\_UNIT\_FOREVER\_REL}. | ||
1337 | |||
1338 | If we give a route option @code{GNUNET\_DHT\_RO\_RECORD\_ROUTE} | ||
1339 | the callback will get a list of all the peers the data has travelled, | ||
1340 | both on the PUT path and on the GET path. | ||
1341 | |||
1342 | @example | ||
1343 | @verbatiminclude tutorial-examples/020.c | ||
1344 | @end example | ||
1345 | |||
1346 | @noindent | ||
1347 | Exercise: Store a value in the DHT and after a while retrieve it. | ||
1348 | Show the IDs of all the peers the requests have gone through. | ||
1349 | In order to convert a peer ID to a string, use the function | ||
1350 | @code{GNUNET\_i2s}. Pay attention to the route option parameters | ||
1351 | in both calls! | ||
1352 | |||
1353 | @node Implementing a block plugin | ||
1354 | @subsection Implementing a block plugin | ||
1355 | |||
1356 | In order to store data in the DHT, it is necessary to provide a block | ||
1357 | plugin. The DHT uses the block plugin to ensure that only well-formed | ||
1358 | requests and replies are transmitted over the network. | ||
1359 | |||
1360 | The block plugin should be put in a file @file{plugin\_block\_SERVICE.c} | ||
1361 | in the service's respective directory. The | ||
1362 | mandatory functions that need to be implemented for a block plugin are | ||
1363 | described in the following sections. | ||
1364 | |||
1365 | @menu | ||
1366 | * Validating requests and replies:: | ||
1367 | * Deriving a key from a reply:: | ||
1368 | * Initialization of the plugin:: | ||
1369 | * Shutdown of the plugin:: | ||
1370 | * Integration of the plugin with the build system:: | ||
1371 | @end menu | ||
1372 | |||
1373 | @node Validating requests and replies | ||
1374 | @subsubsection Validating requests and replies | ||
1375 | |||
1376 | The evaluate function should validate a reply or a request. It returns | ||
1377 | a @code{GNUNET\_BLOCK\_EvaluationResult}, which is an enumeration. All | ||
1378 | possible answers are in @file{gnunet\_block\_lib.h}. The function will | ||
1379 | be called with a @code{reply\_block} argument of @code{NULL} for | ||
1380 | requests. Note that depending on how @code{evaluate} is called, only | ||
1381 | some of the possible return values are valid. The specific meaning of | ||
1382 | the @code{xquery} argument is application-specific. Applications that | ||
1383 | do not use an extended query should check that the @code{xquery\_size} | ||
1384 | is zero. The block group is typically used to filter duplicate | ||
1385 | replies. | ||
1386 | |||
1387 | @example | ||
1388 | @verbatiminclude tutorial-examples/021.c | ||
1389 | @end example | ||
1390 | |||
1391 | @noindent | ||
1392 | Note that it is mandatory to detect duplicate replies in this function | ||
1393 | and return the respective status code. Duplicate detection is | ||
1394 | typically done using the Bloom filter block group provided by | ||
1395 | @file{libgnunetblockgroup.so}. Failure to do so may cause replies to | ||
1396 | circle in the network. | ||
1397 | |||
1398 | @node Deriving a key from a reply | ||
1399 | @subsubsection Deriving a key from a reply | ||
1400 | |||
1401 | The DHT can operate more efficiently if it is possible to derive a key | ||
1402 | from the value of the corresponding block. The @code{get\_key} | ||
1403 | function is used to obtain the key of a block --- for example, by | ||
1404 | means of hashing. If deriving the key is not possible, the function | ||
1405 | should simply return @code{GNUNET\_SYSERR} (the DHT will still work | ||
1406 | just fine with such blocks). | ||
1407 | |||
1408 | @example | ||
1409 | @verbatiminclude tutorial-examples/022.c | ||
1410 | @end example | ||
1411 | |||
1412 | @node Initialization of the plugin | ||
1413 | @subsubsection Initialization of the plugin | ||
1414 | |||
1415 | The plugin is realized as a shared C library. The library must export | ||
1416 | an initialization function which should initialize the plugin. The | ||
1417 | initialization function specifies what block types the plugin cares | ||
1418 | about and returns a struct with the functions that are to be used for | ||
1419 | validation and obtaining keys (the ones just defined above). | ||
1420 | |||
1421 | @example | ||
1422 | @verbatiminclude tutorial-examples/023.c | ||
1423 | @end example | ||
1424 | |||
1425 | @node Shutdown of the plugin | ||
1426 | @subsubsection Shutdown of the plugin | ||
1427 | |||
1428 | Following GNUnet's general plugin API concept, the plugin must | ||
1429 | export a second function for cleaning up. It usually does very | ||
1430 | little. | ||
1431 | |||
1432 | @example | ||
1433 | @verbatiminclude tutorial-examples/024.c | ||
1434 | @end example | ||
1435 | |||
1436 | @node Integration of the plugin with the build system | ||
1437 | @subsubsection Integration of the plugin with the build system | ||
1438 | |||
1439 | In order to compile the plugin, the @file{Makefile.am} file for the | ||
1440 | service SERVICE should contain a rule similar to this: | ||
1441 | @c Actually this is a Makefile not C. But the whole structure of examples | ||
1442 | @c must be improved. | ||
1443 | |||
1444 | @example | ||
1445 | @verbatiminclude tutorial-examples/025.c | ||
1446 | @end example | ||
1447 | |||
1448 | @noindent | ||
1449 | Exercise: Write a block plugin that accepts all queries | ||
1450 | and all replies but prints information about queries and replies | ||
1451 | when the respective validation hooks are called. | ||
1452 | |||
1453 | @node Monitoring the DHT | ||
1454 | @subsection Monitoring the DHT | ||
1455 | |||
1456 | It is possible to monitor the functioning of the local | ||
1457 | DHT service. When monitoring the DHT, the service will | ||
1458 | alert the monitoring program of any events, both started | ||
1459 | locally or received for routing from another peer. | ||
1460 | The are three different types of events possible: a | ||
1461 | GET request, a PUT request or a response (a reply to a GET). | ||
1462 | |||
1463 | Since the different events have different associated data, | ||
1464 | the API gets 3 different callbacks (one for each message type) | ||
1465 | and optional type and key parameters, to allow for filtering of | ||
1466 | messages. When an event happens, the appropiate callback is | ||
1467 | called with all the information about the event. | ||
1468 | |||
1469 | @example | ||
1470 | @verbatiminclude tutorial-examples/026.c | ||
1471 | @end example | ||
1472 | |||
1473 | @node Debugging with gnunet-arm | ||
1474 | @section Debugging with gnunet-arm | ||
1475 | |||
1476 | Even if services are managed by @command{gnunet-arm}, you can | ||
1477 | start them with @command{gdb} or @command{valgrind}. For | ||
1478 | example, you could add the following lines to your | ||
1479 | configuration file to start the DHT service in a @command{gdb} | ||
1480 | session in a fresh @command{xterm}: | ||
1481 | |||
1482 | @example | ||
1483 | [dht] | ||
1484 | PREFIX=xterm -e gdb --args | ||
1485 | @end example | ||
1486 | |||
1487 | @noindent | ||
1488 | Alternatively, you can stop a service that was started via | ||
1489 | ARM and run it manually: | ||
1490 | |||
1491 | @example | ||
1492 | $ gnunet-arm -k dht | ||
1493 | $ gdb --args gnunet-service-dht -L DEBUG | ||
1494 | $ valgrind gnunet-service-dht -L DEBUG | ||
1495 | @end example | ||
1496 | |||
1497 | @noindent | ||
1498 | Assuming other services are well-written, they will automatically | ||
1499 | re-integrate the restarted service with the peer. | ||
1500 | |||
1501 | GNUnet provides a powerful logging mechanism providing log | ||
1502 | levels @code{ERROR}, @code{WARNING}, @code{INFO} and @code{DEBUG}. | ||
1503 | The current log level is configured using the @code{$GNUNET_FORCE_LOG} | ||
1504 | environmental variable. The @code{DEBUG} level is only available if | ||
1505 | @command{--enable-logging=verbose} was used when running | ||
1506 | @command{configure}. More details about logging can be found under | ||
1507 | @uref{https://gnunet.org/logging}. | ||
1508 | |||
1509 | You should also probably enable the creation of core files, by setting | ||
1510 | @code{ulimit}, and echo'ing @code{1} into | ||
1511 | @file{/proc/sys/kernel/core\_uses\_pid}. Then you can investigate the | ||
1512 | core dumps with @command{gdb}, which is often the fastest method to | ||
1513 | find simple errors. | ||
1514 | |||
1515 | Exercise: Add a memory leak to your service and obtain a trace | ||
1516 | pointing to the leak using @command{valgrind} while running the service | ||
1517 | from @command{gnunet-service-arm}. | ||
1518 | |||
1519 | @bye | ||