diff options
Diffstat (limited to 'doc/documentation/gnunet-c-tutorial.texi')
-rw-r--r-- | doc/documentation/gnunet-c-tutorial.texi | 1540 |
1 files changed, 1540 insertions, 0 deletions
diff --git a/doc/documentation/gnunet-c-tutorial.texi b/doc/documentation/gnunet-c-tutorial.texi new file mode 100644 index 000000000..f39c7de64 --- /dev/null +++ b/doc/documentation/gnunet-c-tutorial.texi | |||
@@ -0,0 +1,1540 @@ | |||
1 | \input texinfo | ||
2 | @c %**start of header | ||
3 | @setfilename gnunet-c-tutorial.info | ||
4 | @documentencoding UTF-8 | ||
5 | @settitle GNUnet C Tutorial | ||
6 | @exampleindent 2 | ||
7 | @c %**end of header | ||
8 | |||
9 | @c including 'version.texi' makes makeinfo throw errors. | ||
10 | @include version2.texi | ||
11 | |||
12 | @copying | ||
13 | Copyright @copyright{} 2001-2017 GNUnet e.V. | ||
14 | |||
15 | Permission is granted to copy, distribute and/or modify this document | ||
16 | under the terms of the GNU Free Documentation License, Version 1.3 or | ||
17 | any later version published by the Free Software Foundation; with no | ||
18 | Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A | ||
19 | copy of the license is included in the section entitled ``GNU Free | ||
20 | Documentation License''. | ||
21 | |||
22 | A copy of the license is also available from the Free Software | ||
23 | Foundation Web site at @url{http://www.gnu.org/licenses/fdl.html}. | ||
24 | |||
25 | Alternately, this document is also available under the General | ||
26 | Public License, version 3 or later, as published by the Free Software | ||
27 | Foundation. A copy of the license is included in the section entitled | ||
28 | ``GNU General Public License''. | ||
29 | |||
30 | A copy of the license is also available from the Free Software | ||
31 | Foundation Web site at @url{http://www.gnu.org/licenses/gpl.html}. | ||
32 | @end copying | ||
33 | |||
34 | @dircategory Tutorial | ||
35 | @direntry | ||
36 | * GNUnet-C-Tutorial: (gnunet-c-tutorial). C Tutorial for GNunet | ||
37 | @end direntry | ||
38 | |||
39 | |||
40 | @titlepage | ||
41 | @title GNUnet C Tutorial | ||
42 | @subtitle A Tutorial for GNUnet @value{VERSION} (C version) | ||
43 | @author The GNUnet Developers | ||
44 | |||
45 | @page | ||
46 | @vskip 0pt plus 1filll | ||
47 | |||
48 | @insertcopying | ||
49 | @end titlepage | ||
50 | |||
51 | @contents | ||
52 | |||
53 | @c **** TODO | ||
54 | @c 1. Update content? | ||
55 | @c 2. Either reference main documentation or | ||
56 | @c 3. Merge this into main documentation | ||
57 | |||
58 | @node Top | ||
59 | @top Introduction | ||
60 | |||
61 | This tutorials explains how to install GNUnet on a | ||
62 | GNU/Linux system and gives an introduction on how | ||
63 | GNUnet can be used to develop a Peer-to-Peer application. | ||
64 | Detailed installation instructions for | ||
65 | various operating systems and a detailed list of all | ||
66 | dependencies can be found on our website at | ||
67 | @uref{https://gnunet.org/installation} and in our | ||
68 | Reference Documentation (GNUnet Handbook). | ||
69 | |||
70 | Please read this tutorial carefully since every single step is | ||
71 | important and do not hesitate to contact the GNUnet team if you have | ||
72 | any questions or problems! Check here how to contact the GNUnet | ||
73 | team: @uref{https://gnunet.org/contact_information} | ||
74 | |||
75 | @menu | ||
76 | |||
77 | * Installing GNUnet:: Installing GNUnet | ||
78 | * Introduction to GNUnet Architecture:: Introduction to GNUnet Architecture | ||
79 | * First Steps with GNUnet:: First Steps with GNUnet | ||
80 | * Developing Applications:: Developing Applications | ||
81 | |||
82 | @detailmenu | ||
83 | --- The Detailed Node Listing --- | ||
84 | |||
85 | Installing GNUnet | ||
86 | |||
87 | * Obtaining a stable version:: | ||
88 | * Installing Build Tool Chain and Dependencies:: | ||
89 | * Obtaining the latest version from Git:: | ||
90 | * Compiling and Installing GNUnet:: | ||
91 | * Common Issues - Check your GNUnet installation:: | ||
92 | |||
93 | Introduction to GNUnet Architecture | ||
94 | |||
95 | First Steps with GNUnet | ||
96 | |||
97 | * Configure your peer:: | ||
98 | * Start a peer:: | ||
99 | * Monitor a peer:: | ||
100 | * Starting Two Peers by Hand:: | ||
101 | * Starting Peers Using the Testbed Service:: | ||
102 | |||
103 | Developing Applications | ||
104 | |||
105 | * gnunet-ext:: | ||
106 | * Adapting the Template:: | ||
107 | * Writing a Client Application:: | ||
108 | * Writing a Service:: | ||
109 | * Interacting directly with other Peers using the CORE Service:: | ||
110 | * Storing peer-specific data using the PEERSTORE service:: | ||
111 | * Using the DHT:: | ||
112 | * Debugging with gnunet-arm:: | ||
113 | |||
114 | @end detailmenu | ||
115 | @end menu | ||
116 | |||
117 | @node Installing GNUnet | ||
118 | @chapter Installing GNUnet | ||
119 | |||
120 | First of all you have to install a current version of GNUnet. | ||
121 | You can download a tarball of a stable version from GNU FTP mirrors | ||
122 | or obtain the latest development version from our Git repository. | ||
123 | |||
124 | Most of the time you should prefer to download the stable version | ||
125 | since with the latest development version things can be broken, | ||
126 | functionality can be changed or tests can fail. You should only use | ||
127 | the development version if you know that you require a certain | ||
128 | feature or a certain issue has been fixed since the last release. | ||
129 | |||
130 | @menu | ||
131 | * Obtaining a stable version:: | ||
132 | * Installing Build Tool Chain and Dependencies:: | ||
133 | * Obtaining the latest version from Git:: | ||
134 | * Compiling and Installing GNUnet:: | ||
135 | * Common Issues - Check your GNUnet installation:: | ||
136 | @end menu | ||
137 | |||
138 | @node Obtaining a stable version | ||
139 | @section Obtaining a stable version | ||
140 | |||
141 | Download the tarball from | ||
142 | @indicateurl{https://ftp.gnu.org/gnu/gnunet/gnunet-@value{VERSION}.tar.gz}. | ||
143 | |||
144 | Make sure to download the associated @file{.sig} file and to verify the | ||
145 | authenticity of the tarball against it, like this: | ||
146 | |||
147 | @example | ||
148 | $ wget https://ftp.gnu.org/gnu/gnunet/gnunet-@value{VERSION}.tar.gz.sig | ||
149 | $ gpg --verify-files gnunet-@value{VERSION}.tar.gz.sig | ||
150 | @end example | ||
151 | |||
152 | @noindent | ||
153 | If this command fails because you do not have the required public key, | ||
154 | then you need to run this command to import it: | ||
155 | |||
156 | @example | ||
157 | $ gpg --keyserver keys.gnupg.net --recv-keys 48426C7E | ||
158 | @end example | ||
159 | |||
160 | @noindent | ||
161 | and rerun the @code{gpg --verify-files} command. | ||
162 | |||
163 | @b{Note:}@ | ||
164 | @b{The pub key to sign the 0.10.1 release has been | ||
165 | revoked}. You will get an error message stating that | ||
166 | @b{there is no known public key or that it has been revoked}. | ||
167 | The next release of GNUnet will have a valid signature | ||
168 | again. We are sorry for the inconvenience this causes. | ||
169 | Another possible source you could use is our | ||
170 | "gnunet" git repository which has mandatory signed commits | ||
171 | by every developer. | ||
172 | |||
173 | Now you can extract the tarball and rename the resulting | ||
174 | directory to @file{gnunet} which we will be using in the | ||
175 | remainder of this document. | ||
176 | |||
177 | @example | ||
178 | $ tar xvzf gnunet-@value{VERSION}.tar.gz | ||
179 | $ mv gnunet-@value{VERSION} gnunet | ||
180 | $ cd gnunet | ||
181 | @end example | ||
182 | |||
183 | @noindent | ||
184 | However, please note that stable versions can be very outdated. | ||
185 | As a developer you are @b{strongly} encouraged to use the version | ||
186 | from @uref{https://gnunet.org/git/, git}. | ||
187 | |||
188 | @node Installing Build Tool Chain and Dependencies | ||
189 | @section Installing Build Tool Chain and Dependencies | ||
190 | |||
191 | To successfully compile GNUnet, you need the tools to build GNUnet and | ||
192 | the required dependencies. Please take a look at the | ||
193 | GNUnet Reference Documentation | ||
194 | (@pxref{Dependencies, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation}) | ||
195 | for a list of required dependencies | ||
196 | and | ||
197 | (@pxref{Generic installation instructions, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation}) | ||
198 | read its Installation chapter for specific instructions for | ||
199 | your operating system. | ||
200 | Please check the notes at the end of the configure process about | ||
201 | required dependencies. | ||
202 | |||
203 | For GNUnet bootstrapping support and the HTTP(S) plugin you should | ||
204 | install @uref{https://gnunet.org/gnurl, libgnurl}. | ||
205 | For the filesharing service you should install at least one of the | ||
206 | datastore backends. MySQL, SQlite and PostgreSQL are supported. | ||
207 | |||
208 | @node Obtaining the latest version from Git | ||
209 | @section Obtaining the latest version from Git | ||
210 | |||
211 | The latest development version can obtained from our Git repository. | ||
212 | To obtain the code you need to have @code{Git} installed, which is | ||
213 | required for obtaining the repository via: | ||
214 | |||
215 | @example | ||
216 | $ git clone https://gnunet.org/git/gnunet | ||
217 | @end example | ||
218 | |||
219 | @noindent | ||
220 | After cloning the repository you have to execute the @file{bootstrap} | ||
221 | script in the new directory: | ||
222 | |||
223 | @example | ||
224 | $ cd gnunet | ||
225 | $ ./bootstrap | ||
226 | @end example | ||
227 | |||
228 | @noindent | ||
229 | The remainder of this tutorial will assume that you have the | ||
230 | Git branch ``master'' checked out. | ||
231 | |||
232 | @node Compiling and Installing GNUnet | ||
233 | @section Compiling and Installing GNUnet | ||
234 | |||
235 | Note: This section is a duplication of the more in depth | ||
236 | @pxref{GNUnet Installation Handbook, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation}. | ||
237 | |||
238 | First, you need to install libgnupgerror @geq{} 1.27 and | ||
239 | libgcrypt @geq{} 1.7.6: | ||
240 | |||
241 | @example | ||
242 | $ export GNUPGFTP="https://www.gnupg.org/ftp/gcrypt" | ||
243 | $ wget $GNUPGFTP/libgpg-error/libgpg-error-1.27.tar.bz2 | ||
244 | $ tar xf libgpg-error-1.27.tar.bz2 | ||
245 | $ cd libgpg-error-1.27 | ||
246 | $ ./configure | ||
247 | $ sudo make install | ||
248 | $ cd .. | ||
249 | @end example | ||
250 | |||
251 | @example | ||
252 | $ export GNUPGFTP="https://www.gnupg.org/ftp/gcrypt" | ||
253 | $ wget $GNUPGFTP/libgcrypt/libgcrypt-1.7.6.tar.bz2 | ||
254 | $ tar xf libgcrypt-1.7.6.tar.bz2 | ||
255 | $ cd libgcrypt-1.7.6 | ||
256 | $ ./configure | ||
257 | $ sudo make install | ||
258 | $ cd .. | ||
259 | @end example | ||
260 | |||
261 | @menu | ||
262 | * Installation:: | ||
263 | @end menu | ||
264 | |||
265 | @node Installation | ||
266 | @subsection Installation | ||
267 | Assuming all dependencies are installed, the following commands will | ||
268 | compile and install GNUnet in your home directory. You can specify the | ||
269 | directory where GNUnet will be installed by changing the | ||
270 | @code{--prefix} value when calling @command{./configure}. If | ||
271 | you do not specifiy a prefix, GNUnet is installed in the directory | ||
272 | @file{/usr/local}. When developing new applications you may want | ||
273 | to enable verbose logging by adding @code{--enable-logging=verbose}: | ||
274 | |||
275 | @example | ||
276 | $ ./configure --prefix=$PREFIX --enable-logging | ||
277 | $ make | ||
278 | $ make install | ||
279 | @end example | ||
280 | |||
281 | @noindent | ||
282 | After installing GNUnet you have to add your GNUnet installation | ||
283 | to your path environmental variable. In addition you have to | ||
284 | create the @file{.config} directory in your home directory | ||
285 | (unless it already exists) where GNUnet stores its data and an | ||
286 | empty GNUnet configuration file: | ||
287 | |||
288 | @example | ||
289 | $ export PATH=$PATH:$PREFIX/bin | ||
290 | $ echo export PATH=$PREFIX/bin:\\$PATH >> ~/.bashrc | ||
291 | $ mkdir ~/.config/ | ||
292 | $ touch ~/.config/gnunet.conf | ||
293 | @end example | ||
294 | |||
295 | @node Common Issues - Check your GNUnet installation | ||
296 | @section Common Issues - Check your GNUnet installation | ||
297 | |||
298 | You should check your installation to ensure that installing GNUnet | ||
299 | was successful up to this point. You should be able to access GNUnet's | ||
300 | binaries and run GNUnet's self check. | ||
301 | |||
302 | @example | ||
303 | $ which gnunet-arm | ||
304 | @end example | ||
305 | |||
306 | @noindent | ||
307 | should return $PREFIX/bin/gnunet-arm. It should be located in your | ||
308 | GNUnet installation and the output should not be empty. | ||
309 | If you see an output like: | ||
310 | |||
311 | @example | ||
312 | $ which gnunet-arm | ||
313 | @end example | ||
314 | |||
315 | @noindent | ||
316 | check your PATH variable to ensure GNUnet's @file{bin} directory is | ||
317 | included. | ||
318 | |||
319 | GNUnet provides tests for all of its subcomponents. Run | ||
320 | |||
321 | @example | ||
322 | $ make check | ||
323 | @end example | ||
324 | |||
325 | @noindent | ||
326 | to execute tests for all components. @command{make check} traverses all | ||
327 | subdirectories in @file{src}. For every subdirectory you should | ||
328 | get a message like this: | ||
329 | |||
330 | @example | ||
331 | make[2]: Entering directory `/home/$USER/gnunet/contrib' | ||
332 | PASS: test_gnunet_prefix | ||
333 | ============= | ||
334 | 1 test passed | ||
335 | ============= | ||
336 | @end example | ||
337 | |||
338 | @node Introduction to GNUnet Architecture | ||
339 | @chapter Introduction to GNUnet Architecture | ||
340 | |||
341 | GNUnet is organized in layers and services. Each service is composed of a | ||
342 | main service implementation and a client library for other programs to use | ||
343 | the service's functionality, described by an API. | ||
344 | @c This approach is shown in | ||
345 | @c FIXME: enable this once the commented block below works: | ||
346 | @c figure~\ref fig:service. | ||
347 | Some services provide an additional command line tool to enable the user | ||
348 | to interact with the service. | ||
349 | |||
350 | Very often it is other GNUnet services that will use these APIs to build | ||
351 | the higher layers of GNUnet on top of the lower ones. Each layer expands | ||
352 | or extends the functionality of the service below (for instance, to build | ||
353 | a mesh on top of a DHT). | ||
354 | @c FXIME: See comment above. | ||
355 | @c See figure ~\ref fig:interaction for an illustration of this approach. | ||
356 | |||
357 | @c ** @image filename[, width[, height[, alttext[, extension]]]] | ||
358 | @c FIXME: Texlive (?) 20112 makes the assumption that this means | ||
359 | @c 'images/OBJECTNAME.txt' but later versions of it (2017) use this | ||
360 | @c syntax as described below. | ||
361 | @c TODO: Checkout the makedoc script Guile uses. | ||
362 | |||
363 | @c FIXME!!! | ||
364 | @c @image{images/gnunet-tutorial-service,,5in,Service with API and network protocol,.png} | ||
365 | @c @image{images/gnunet-tutorial-system,,5in,The layered system architecture of GNUnet,.png} | ||
366 | |||
367 | @c \begin{figure}[!h] | ||
368 | @c \begin{center} | ||
369 | @c % \begin{subfigure} | ||
370 | @c \begin{subfigure}[b]{0.3\textwidth} | ||
371 | @c \centering | ||
372 | @c \includegraphics[width=\textwidth]{figs/Service.pdf} | ||
373 | @c \caption{Service with API and network protocol} | ||
374 | @c \label{fig:service} | ||
375 | @c \end{subfigure} | ||
376 | @c ~~~~~~~~~~ | ||
377 | @c \begin{subfigure}[b]{0.3\textwidth} | ||
378 | @c \centering | ||
379 | @c \includegraphics[width=\textwidth]{figs/System.pdf} | ||
380 | @c \caption{Service interaction} | ||
381 | @c \label{fig:interaction} | ||
382 | @c \end{subfigure} | ||
383 | @c \end{center} | ||
384 | @c \caption{GNUnet's layered system architecture} | ||
385 | @c \end{figure} | ||
386 | |||
387 | The main service implementation runs as a standalone process in the | ||
388 | operating system and the client code runs as part of the client program, | ||
389 | so crashes of a client do not affect the service process or other clients. | ||
390 | The service and the clients communicate via a message protocol to be | ||
391 | defined and implemented by the programmer. | ||
392 | |||
393 | @node First Steps with GNUnet | ||
394 | @chapter First Steps with GNUnet | ||
395 | |||
396 | @menu | ||
397 | * Configure your peer:: | ||
398 | * Start a peer:: | ||
399 | * Monitor a peer:: | ||
400 | * Starting Two Peers by Hand:: | ||
401 | * Starting Peers Using the Testbed Service:: | ||
402 | @end menu | ||
403 | |||
404 | @node Configure your peer | ||
405 | @section Configure your peer | ||
406 | |||
407 | First of all we need to configure your peer. Each peer is started with | ||
408 | a configuration containing settings for GNUnet itself and its services. | ||
409 | This configuration is based on the default configuration shipped with | ||
410 | GNUnet and can be modified. The default configuration is located in the | ||
411 | @file{$PREFIX/share/gnunet/config.d} directory. When starting a peer, you | ||
412 | can specify a customized configuration using the the @command{-c} command | ||
413 | line switch when starting the ARM service and all other services. When | ||
414 | using a modified configuration the default values are loaded and only | ||
415 | values specified in the configuration file will replace the default | ||
416 | values. | ||
417 | |||
418 | Since we want to start additional peers later, we need some modifications | ||
419 | from the default configuration. We need to create a separate service | ||
420 | home and a file containing our modifications for this peer: | ||
421 | |||
422 | @example | ||
423 | $ mkdir ~/gnunet1/ | ||
424 | $ touch peer1.conf | ||
425 | @end example | ||
426 | |||
427 | @noindent | ||
428 | Now add the following lines to @file{peer1.conf} to use this directory. | ||
429 | For simplified usage we want to prevent the peer to connect to the GNUnet | ||
430 | network since this could lead to confusing output. This modifications | ||
431 | will replace the default settings: | ||
432 | |||
433 | @example | ||
434 | [PATHS] | ||
435 | # Use this directory to store GNUnet data | ||
436 | GNUNET_HOME = ~/gnunet1/ | ||
437 | [hostlist] | ||
438 | # prevent bootstrapping | ||
439 | SERVERS = | ||
440 | @end example | ||
441 | |||
442 | @node Start a peer | ||
443 | @section Start a peer | ||
444 | Each GNUnet instance (called peer) has an identity (peer ID) based on a | ||
445 | cryptographic public private key pair. The peer ID is the printable hash | ||
446 | of the public key. | ||
447 | |||
448 | GNUnet services are controlled by a master service, the so called | ||
449 | @dfn{Automatic Restart Manager} (ARM). ARM starts, stops and even | ||
450 | restarts services automatically or on demand when a client connects. | ||
451 | You interact with the ARM service using the @command{gnunet-arm} tool. | ||
452 | GNUnet can then be started with @command{gnunet-arm -s} and stopped with | ||
453 | @command{gnunet-arm -e}. An additional service not automatically started | ||
454 | can be started using @command{gnunet-arm -i <service name>} and stopped | ||
455 | using @command{gnunet-arm -k <servicename>}. | ||
456 | |||
457 | Once you have started your peer, you can use many other GNUnet commands | ||
458 | to interact with it. For example, you can run: | ||
459 | |||
460 | @example | ||
461 | $ gnunet-peerinfo -s | ||
462 | @end example | ||
463 | |||
464 | @noindent | ||
465 | to obtain the public key of your peer. | ||
466 | |||
467 | You should see an output containing the peer ID similar to: | ||
468 | |||
469 | @example | ||
470 | I am peer `0PA02UVRKQTS2C .. JL5Q78F6H0B1ACPV1CJI59MEQUMQCC5G'. | ||
471 | @end example | ||
472 | |||
473 | @node Monitor a peer | ||
474 | @section Monitor a peer | ||
475 | |||
476 | In this section, we will monitor the behaviour of our peer's DHT | ||
477 | service with respect to a specific key. First we will start | ||
478 | GNUnet and then start the DHT service and use the DHT monitor tool | ||
479 | to monitor the PUT and GET commands we issue ussing the | ||
480 | @command{gnunet-dht-put} and @command{gnunet-dht-get} commands. | ||
481 | Using the ``monitor'' line given below, you can observe the behavior | ||
482 | of your own peer's DHT with respect to the specified KEY: | ||
483 | |||
484 | @example | ||
485 | # start gnunet with all default services: | ||
486 | $ gnunet-arm -c ~/peer1.conf -s | ||
487 | # start DHT service: | ||
488 | $ gnunet-arm -c ~/peer1.conf -i dht | ||
489 | $ cd ~/gnunet/src/dht; | ||
490 | $ ./gnunet-dht-monitor -c ~/peer1.conf -k KEY | ||
491 | @end example | ||
492 | |||
493 | @noindent | ||
494 | Now open a separate terminal and change again to | ||
495 | the @file{gnunet/src/dht} directory: | ||
496 | |||
497 | @example | ||
498 | $ cd ~/gnunet/src/dht | ||
499 | # put VALUE under KEY in the DHT: | ||
500 | $ ./gnunet-dht-put -c ~/peer1.conf -k KEY -d VALUE | ||
501 | # get key KEY from the DHT: | ||
502 | $ ./gnunet/src/dht/gnunet-dht-get -c ~/peer1.conf -k KEY | ||
503 | # print statistics about current GNUnet state: | ||
504 | $ gnunet-statistics -c ~/peer1.conf | ||
505 | # print statistics about DHT service: | ||
506 | $ gnunet-statistics -c ~/peer1.conf -s dht | ||
507 | @end example | ||
508 | |||
509 | @node Starting Two Peers by Hand | ||
510 | @section Starting Two Peers by Hand | ||
511 | |||
512 | This section describes how to start two peers on the same machine by hand. | ||
513 | The process is rather painful, but the description is somewhat | ||
514 | instructive. In practice, you might prefer the automated method | ||
515 | (@pxref{Starting Peers Using the Testbed Service}). | ||
516 | |||
517 | @menu | ||
518 | * Setup a second peer:: | ||
519 | * Start the second peer and connect the peers:: | ||
520 | * How to connect manually:: | ||
521 | @end menu | ||
522 | |||
523 | @node Setup a second peer | ||
524 | @subsection Setup a second peer | ||
525 | We will now start a second peer on your machine. | ||
526 | For the second peer, you will need to manually create a modified | ||
527 | configuration file to avoid conflicts with ports and directories. | ||
528 | A peers configuration file is by default located | ||
529 | in @file{~/.gnunet/gnunet.conf}. This file is typically very short | ||
530 | or even empty as only the differences to the defaults need to be | ||
531 | specified. The defaults are located in many files in the | ||
532 | @file{$PREFIX/share/gnunet/config.d} directory. | ||
533 | |||
534 | To configure the second peer, use the files | ||
535 | @file{$PREFIX/share/gnunet/config.d} as a template for your main | ||
536 | configuration file: | ||
537 | |||
538 | @example | ||
539 | $ cat $PREFIX/share/gnunet/config.d/*.conf > peer2.conf | ||
540 | @end example | ||
541 | |||
542 | @noindent | ||
543 | Now you have to edit @file{peer2.conf} and change: | ||
544 | |||
545 | @itemize | ||
546 | @item @code{GNUNET\_TEST\_HOME} under @code{PATHS} | ||
547 | @item Every (uncommented) value for ``@code{PORT}'' (add 10000) in any | ||
548 | section (the option may be commented out if @code{PORT} is | ||
549 | prefixed by "\#", in this case, UNIX domain sockets are used | ||
550 | and the PORT option does not need to be touched) | ||
551 | @item Every value for ``@code{UNIXPATH}'' in any section | ||
552 | (e.g. by adding a "-p2" suffix) | ||
553 | @end itemize | ||
554 | |||
555 | to a fresh, unique value. Make sure that the PORT numbers stay | ||
556 | below 65536. From now on, whenever you interact with the second peer, | ||
557 | you need to specify @command{-c peer2.conf} as an additional | ||
558 | command line argument. | ||
559 | |||
560 | Now, generate the 2nd peer's private key: | ||
561 | |||
562 | @example | ||
563 | $ gnunet-peerinfo -s -c peer2.conf | ||
564 | @end example | ||
565 | |||
566 | @noindent | ||
567 | This may take a while, generate entropy using your keyboard or mouse | ||
568 | as needed. Also, make sure the output is different from the | ||
569 | gnunet-peerinfo output for the first peer (otherwise you made an | ||
570 | error in the configuration). | ||
571 | |||
572 | @node Start the second peer and connect the peers | ||
573 | @subsection Start the second peer and connect the peers | ||
574 | |||
575 | Then, you can start a second peer using: | ||
576 | |||
577 | @example | ||
578 | $ gnunet-arm -c peer2.conf -s | ||
579 | $ gnunet-arm -c peer2.conf -i dht | ||
580 | $ ~/gnunet/src/dht/gnunet-dht-put -c peer2.conf -k KEY -d VALUE | ||
581 | $ ~/gnunet/src/dht/gnunet-dht-get -c peer2.conf -k KEY | ||
582 | @end example | ||
583 | |||
584 | If you want the two peers to connect, you have multiple options: | ||
585 | |||
586 | @itemize | ||
587 | @item UDP neighbour discovery (automatic) | ||
588 | @item Setup a bootstrap server | ||
589 | @item Connect manually | ||
590 | @end itemize | ||
591 | |||
592 | To setup peer 1 as bootstrapping server change the configuration of | ||
593 | the first one to be a hostlist server by adding the following lines to | ||
594 | @file{peer1.conf} to enable bootstrapping server: | ||
595 | |||
596 | @example | ||
597 | [hostlist] | ||
598 | OPTIONS = -p | ||
599 | @end example | ||
600 | |||
601 | @noindent | ||
602 | Then change @file{peer2.conf} and replace the ``@code{SERVERS}'' | ||
603 | line in the ``@code{[hostlist]}'' section with | ||
604 | ``@code{http://localhost:8080/}''. Restart both peers using: | ||
605 | |||
606 | @example | ||
607 | # stop first peer | ||
608 | $ gnunet-arm -c peer1.conf -e | ||
609 | # start first peer | ||
610 | $ gnunet-arm -c peer1.conf -s | ||
611 | # start second peer | ||
612 | $ gnunet-arm -c peer2.conf -s | ||
613 | @end example | ||
614 | |||
615 | @noindent | ||
616 | Note that if you start your peers without changing these settings, they | ||
617 | will use the ``global'' hostlist servers of the GNUnet P2P network and | ||
618 | likely connect to those peers. At that point, debugging might become | ||
619 | tricky as you're going to be connected to many more peers and would | ||
620 | likely observe traffic and behaviors that are not explicitly controlled | ||
621 | by you. | ||
622 | |||
623 | @node How to connect manually | ||
624 | @subsection How to connect manually | ||
625 | |||
626 | If you want to use the @code{peerinfo} tool to connect your | ||
627 | peers, you should: | ||
628 | |||
629 | @itemize | ||
630 | @item Set @code{FORCESTART = NO} in section @code{hostlist} | ||
631 | (to not connect to the global GNUnet) | ||
632 | @item Start both peers running @command{gnunet-arm -c peer1.conf -s} | ||
633 | and @command{gnunet-arm -c peer2.conf -s} | ||
634 | @item Get @code{HELLO} message of the first peer running | ||
635 | @command{gnunet-peerinfo -c peer1.conf -g} | ||
636 | @item Give the output to the second peer by running | ||
637 | @command{gnunet-peerinfo -c peer2.conf -p '<output>'} | ||
638 | @end itemize | ||
639 | |||
640 | Check that they are connected using @command{gnunet-core -c peer1.conf}, | ||
641 | which should give you the other peer's peer identity: | ||
642 | |||
643 | @example | ||
644 | $ gnunet-core -c peer1.conf | ||
645 | Peer `9TVUCS8P5A7ILLBGO6 [...shortened...] 1KNBJ4NGCHP3JPVULDG' | ||
646 | @end example | ||
647 | |||
648 | @node Starting Peers Using the Testbed Service | ||
649 | @section Starting Peers Using the Testbed Service | ||
650 | @c \label{sec:testbed} | ||
651 | |||
652 | GNUnet's testbed service is used for testing scenarios where | ||
653 | a number of peers are to be started. The testbed can manage peers | ||
654 | on a single host or on multiple hosts in a distributed fashion. | ||
655 | On a single affordable computer, it should be possible to run | ||
656 | around tens of peers without drastically increasing the load on the | ||
657 | system. | ||
658 | |||
659 | The testbed service can be access through its API | ||
660 | @file{include/gnunet\_testbed\_service.h}. The API provides many | ||
661 | routines for managing a group of peers. It also provides a helper | ||
662 | function @code{GNUNET\_TESTBED\_test\_run()} to quickly setup a | ||
663 | minimalistic testing environment on a single host. | ||
664 | |||
665 | This function takes a configuration file which will be used as a | ||
666 | template configuration for the peers. The testbed takes care of | ||
667 | modifying relevant options in the peers' configuration such as | ||
668 | @code{SERVICEHOME}, @code{PORT}, @code{UNIXPATH} to unique values | ||
669 | so that peers run without running into conflicts. It also checks | ||
670 | and assigns the ports in configurations only if they are free. | ||
671 | |||
672 | Additionally, the testbed service also reads its options from the | ||
673 | same configuration file. Various available options and details | ||
674 | about them can be found in the testbed default configuration file | ||
675 | @file{src/testbed/testbed.conf}. | ||
676 | |||
677 | With the testbed API, a sample test case can be structured as follows: | ||
678 | |||
679 | @example | ||
680 | @verbatiminclude testbed_test.c | ||
681 | @end example | ||
682 | |||
683 | @noindent | ||
684 | The source code for the above listing can be found at | ||
685 | @uref{https://gnunet.org/git/gnunet.git/tree/doc/ | ||
686 | documentation/testbed_test.c} | ||
687 | or in the @file{doc/documentation/} folder of your repository check-out. | ||
688 | After installing GNUnet, the above source code can be compiled as: | ||
689 | |||
690 | @example | ||
691 | $ export CPPFLAGS="-I/path/to/gnunet/headers" | ||
692 | $ export LDFLAGS="-L/path/to/gnunet/libraries" | ||
693 | $ gcc $CPPFLAGS $LDFLAGS -o testbed-test testbed_test.c \ | ||
694 | -lgnunettestbed -lgnunetdht -lgnunetutil | ||
695 | # Generate (empty) configuration | ||
696 | $ touch template.conf | ||
697 | # run it (press CTRL-C to stop) | ||
698 | $ ./testbed-test | ||
699 | @end example | ||
700 | |||
701 | @noindent | ||
702 | The @code{CPPFLAGS} and @code{LDFLAGS} are necessary if GNUnet | ||
703 | is installed into a different directory other than @file{/usr/local}. | ||
704 | |||
705 | All of testbed API's peer management functions treat management | ||
706 | actions as operations and return operation handles. It is expected | ||
707 | that the operations begin immediately, but they may get delayed (to | ||
708 | balance out load on the system). The program using the API then has | ||
709 | to take care of marking the operation as ``done'' so that its | ||
710 | associated resources can be freed immediately and other waiting | ||
711 | operations can be executed. Operations will be canceled if they are | ||
712 | marked as ``done'' before their completion. | ||
713 | |||
714 | An operation is treated as completed when it succeeds or fails. | ||
715 | Completion of an operation is either conveyed as events through | ||
716 | @dfn{controller event callback} or through respective | ||
717 | @dfn{operation completion callbacks}. | ||
718 | In functions which support completion notification | ||
719 | through both controller event callback and operation | ||
720 | completion callback, first the controller event callback will be | ||
721 | called. If the operation is not marked as done in that callback | ||
722 | or if the callback is given as NULL when creating the operation, | ||
723 | the operation completion callback will be called. The API | ||
724 | documentation shows which event are to be expected in the | ||
725 | controller event notifications. It also documents any exceptional | ||
726 | behaviour. | ||
727 | |||
728 | Once the peers are started, test cases often need to connect | ||
729 | some of the peers' services. Normally, opening a connect to | ||
730 | a peer's service requires the peer's configuration. While using | ||
731 | testbed, the testbed automatically generates per-peer configuration. | ||
732 | Accessing those configurations directly through file system is | ||
733 | discouraged as their locations are dynamically created and will be | ||
734 | different among various runs of testbed. To make access to these | ||
735 | configurations easy, testbed API provides the function | ||
736 | @code{GNUNET\_TESTBED\_service\_connect()}. This function fetches | ||
737 | the configuration of a given peer and calls the @dfn{Connect Adapter}. | ||
738 | In the example code, it is the @code{dht\_ca}. A connect adapter is | ||
739 | expected to open the connection to the needed service by using the | ||
740 | provided configuration and return the created service connection handle. | ||
741 | Successful connection to the needed service is signaled through | ||
742 | @code{service\_connect\_comp\_cb}. | ||
743 | |||
744 | A dual to connect adapter is the @dfn{Disconnect Adapter}. This callback | ||
745 | is called after the connect adapter has been called when the operation | ||
746 | from @code{GNUNET\_TESTBED\_service\_connect()} is marked as ``done''. | ||
747 | It has to disconnect from the service with the provided service | ||
748 | handle (@code{op\_result}). | ||
749 | |||
750 | Exercise: Find out how many peers you can run on your system. | ||
751 | |||
752 | Exercise: Find out how to create a 2D torus topology by changing the | ||
753 | options in the configuration file. | ||
754 | @xref{Supported Topologies, The GNUnet Reference Documentation ,, gnunet, The GNUnet Reference Documentation}, | ||
755 | then use the DHT API to store and retrieve values in the network. | ||
756 | |||
757 | @node Developing Applications | ||
758 | @chapter Developing Applications | ||
759 | |||
760 | @menu | ||
761 | * gnunet-ext:: | ||
762 | * Adapting the Template:: | ||
763 | * Writing a Client Application:: | ||
764 | * Writing a Service:: | ||
765 | * Interacting directly with other Peers using the CORE Service:: | ||
766 | * Storing peer-specific data using the PEERSTORE service:: | ||
767 | * Using the DHT:: | ||
768 | * Debugging with gnunet-arm:: | ||
769 | @end menu | ||
770 | |||
771 | @node gnunet-ext | ||
772 | @section gnunet-ext | ||
773 | To develop a new peer-to-peer application or to extend GNUnet we provide | ||
774 | a template build system for writing GNUnet extensions in C. It can be | ||
775 | obtained as follows: | ||
776 | |||
777 | @example | ||
778 | $ git clone https://gnunet.org/git/gnunet-ext | ||
779 | $ cd gnunet-ext/ | ||
780 | $ ./bootstrap | ||
781 | $ ./configure --prefix=$PREFIX --with-gnunet=$PREFIX | ||
782 | $ make | ||
783 | $ make install | ||
784 | $ make check | ||
785 | @end example | ||
786 | |||
787 | @noindent | ||
788 | The GNUnet ext template includes examples and a working buildsystem | ||
789 | for a new GNUnet service. A common GNUnet service consists of the | ||
790 | following parts which will be discussed in detail in the remainder | ||
791 | of this document. The functionality of a GNUnet service is implemented in: | ||
792 | |||
793 | @itemize | ||
794 | @item the GNUnet service (gnunet-ext/src/ext/gnunet-service-ext.c) | ||
795 | @item the client API (gnunet-ext/src/ext/ext_api.c) | ||
796 | @item the client application using the service API | ||
797 | (gnunet-ext/src/ext/gnunet-ext.c) | ||
798 | @end itemize | ||
799 | |||
800 | The interfaces for these entities are defined in: | ||
801 | |||
802 | @itemize | ||
803 | @item client API interface (gnunet-ext/src/ext/ext.h) | ||
804 | @item the service interface (gnunet-ext/src/include/gnunet_service_SERVICE.h) | ||
805 | @item the P2P protocol (gnunet-ext/src/include/gnunet_protocols_ext.h) | ||
806 | @end itemize | ||
807 | |||
808 | |||
809 | In addition the ext systems provides: | ||
810 | |||
811 | @itemize | ||
812 | @item a test testing the API (gnunet-ext/src/ext/test_ext_api.c) | ||
813 | @item a configuration template for the service | ||
814 | (gnunet-ext/src/ext/ext.conf.in) | ||
815 | @end itemize | ||
816 | |||
817 | @node Adapting the Template | ||
818 | @section Adapting the Template | ||
819 | |||
820 | The first step for writing any extension with a new service is to | ||
821 | ensure that the @file{ext.conf.in} file contains entries for the | ||
822 | @code{UNIXPATH}, @code{PORT} and @code{BINARY} for the service in a | ||
823 | section named after the service. | ||
824 | |||
825 | If you want to adapt the template rename the @file{ext.conf.in} to | ||
826 | match your services name, you have to modify the @code{AC\_OUTPUT} | ||
827 | section in @file{configure.ac} in the @file{gnunet-ext} root. | ||
828 | |||
829 | @node Writing a Client Application | ||
830 | @section Writing a Client Application | ||
831 | |||
832 | When writing any client application (for example, a command-line | ||
833 | tool), the basic structure is to start with the | ||
834 | @code{GNUNET\_PROGRAM\_run} function. This function will parse | ||
835 | command-line options, setup the scheduler and then invoke the | ||
836 | @code{run} function (with the remaining non-option arguments) | ||
837 | and a handle to the parsed configuration (and the configuration | ||
838 | file name that was used, which is typically not needed): | ||
839 | |||
840 | @example | ||
841 | @verbatiminclude tutorial-examples/001.c | ||
842 | @end example | ||
843 | |||
844 | @menu | ||
845 | * Handling command-line options:: | ||
846 | * Writing a Client Library:: | ||
847 | * Writing a user interface:: | ||
848 | @end menu | ||
849 | |||
850 | @node Handling command-line options | ||
851 | @subsection Handling command-line options | ||
852 | |||
853 | Options can then be added easily by adding global variables and | ||
854 | expanding the @code{options} array. For example, the following would | ||
855 | add a string-option and a binary flag (defaulting to @code{NULL} and | ||
856 | @code{GNUNET\_NO} respectively): | ||
857 | |||
858 | @example | ||
859 | @verbatiminclude tutorial-examples/002.c | ||
860 | @end example | ||
861 | |||
862 | Issues such as displaying some helpful text describing options using | ||
863 | the @code{--help} argument and error handling are taken care of when | ||
864 | using this approach. Other @code{GNUNET\_GETOPT\_}-functions can be used | ||
865 | to obtain integer value options, increment counters, etc. You can | ||
866 | even write custom option parsers for special circumstances not covered | ||
867 | by the available handlers. To check if an argument was specified by the | ||
868 | user you initialize the variable with a specific value (e.g. NULL for | ||
869 | a string and GNUNET\_SYSERR for a integer) and check after parsing | ||
870 | happened if the values were modified. | ||
871 | |||
872 | Inside the @code{run} method, the program would perform the | ||
873 | application-specific logic, which typically involves initializing and | ||
874 | using some client library to interact with the service. The client | ||
875 | library is supposed to implement the IPC whereas the service provides | ||
876 | more persistent P2P functions. | ||
877 | |||
878 | Exercise: Add a few command-line options and print them inside | ||
879 | of @code{run}. What happens if the user gives invalid arguments? | ||
880 | |||
881 | @node Writing a Client Library | ||
882 | @subsection Writing a Client Library | ||
883 | |||
884 | The first and most important step in writing a client library is to | ||
885 | decide on an API for the library. Typical API calls include | ||
886 | connecting to the service, performing application-specific requests | ||
887 | and cleaning up. Many examples for such service APIs can be found | ||
888 | in the @file{gnunet/src/include/gnunet\_*\_service.h} files. | ||
889 | |||
890 | Then, a client-service protocol needs to be designed. This typically | ||
891 | involves defining various message formats in a header that will be | ||
892 | included by both the service and the client library (but is otherwise | ||
893 | not shared and hence located within the service's directory and not | ||
894 | installed by @command{make install}). Each message must start with a | ||
895 | @code{struct GNUNET\_MessageHeader} and must be shorter than 64k. By | ||
896 | convention, all fields in IPC (and P2P) messages must be in big-endian | ||
897 | format (and thus should be read using @code{ntohl} and similar | ||
898 | functions and written using @code{htonl} and similar functions). | ||
899 | Unique message types must be defined for each message struct in the | ||
900 | @file{gnunet\_protocols.h} header (or an extension-specific include | ||
901 | file). | ||
902 | |||
903 | @menu | ||
904 | * Connecting to the Service:: | ||
905 | * Sending messages:: | ||
906 | * Receiving Replies from the Service:: | ||
907 | @end menu | ||
908 | |||
909 | @node Connecting to the Service | ||
910 | @subsubsection Connecting to the Service | ||
911 | |||
912 | Before a client library can implement the application-specific protocol | ||
913 | with the service, a connection must be created: | ||
914 | |||
915 | @example | ||
916 | @verbatiminclude tutorial-examples/003.c | ||
917 | @end example | ||
918 | |||
919 | @noindent | ||
920 | As a result a @code{GNUNET\_MQ\_Handle} is returned | ||
921 | which can to used henceforth to transmit messages to the service. | ||
922 | The complete MQ API can be found in @file{gnunet\_mq\_lib.h}. | ||
923 | The @code{hanlders} array in the example above is incomplete. | ||
924 | Here is where you will define which messages you expect to | ||
925 | receive from the service, and which functions handle them. | ||
926 | The @code{error\_cb} is a function that is to be called whenever | ||
927 | there are errors communicating with the service. | ||
928 | |||
929 | @node Sending messages | ||
930 | @subsubsection Sending messages | ||
931 | |||
932 | In GNUnet, messages are always sent beginning with a | ||
933 | @code{struct GNUNET\_MessageHeader} in big endian format. | ||
934 | This header defines the size and the type of the | ||
935 | message, the payload follows after this header. | ||
936 | |||
937 | @example | ||
938 | @verbatiminclude tutorial-examples/004.c | ||
939 | @end example | ||
940 | |||
941 | @noindent | ||
942 | Existing message types are defined in @file{gnunet\_protocols.h}. | ||
943 | A common way to create a message is with an envelope: | ||
944 | |||
945 | @example | ||
946 | @verbatiminclude tutorial-examples/005.c | ||
947 | @end example | ||
948 | |||
949 | @noindent | ||
950 | Exercise: Define a message struct that includes a 32-bit | ||
951 | unsigned integer in addition to the standard GNUnet MessageHeader. | ||
952 | Add a C struct and define a fresh protocol number for your message. | ||
953 | Protocol numbers in gnunet-ext are defined | ||
954 | in @file{gnunet-ext/src/include/gnunet_protocols_ext.h} | ||
955 | |||
956 | Exercise: Find out how you can determine the number of messages | ||
957 | in a message queue. | ||
958 | |||
959 | Exercise: Find out how you can determine when a message you | ||
960 | have queued was actually transmitted. | ||
961 | |||
962 | Exercise: Define a helper function to transmit a 32-bit | ||
963 | unsigned integer (as payload) to a service using some given client | ||
964 | handle. | ||
965 | |||
966 | @node Receiving Replies from the Service | ||
967 | @subsubsection Receiving Replies from the Service | ||
968 | |||
969 | Clients can receive messages from the service using the handlers | ||
970 | specified in the @code{handlers} array we specified when connecting | ||
971 | to the service. Entries in the the array are usually created using | ||
972 | one of two macros, depending on whether the message is fixed size | ||
973 | or variable size. Variable size messages are managed using two | ||
974 | callbacks, one to check that the message is well-formed, the other | ||
975 | to actually process the message. Fixed size messages are fully | ||
976 | checked by the MQ-logic, and thus only need to provide the handler | ||
977 | to process the message. Note that the prefixes @code{check\_} | ||
978 | and @code{handle\_} are mandatory. | ||
979 | |||
980 | @example | ||
981 | @verbatiminclude tutorial-examples/006.c | ||
982 | @end example | ||
983 | |||
984 | @noindent | ||
985 | Exercise: Expand your helper function to receive a response message | ||
986 | (for example, containing just the @code{struct GNUnet MessageHeader} | ||
987 | without any payload). Upon receiving the service's response, you | ||
988 | should call a callback provided to your helper function's API. | ||
989 | |||
990 | Exercise: Figure out where you can pass values to the | ||
991 | closures (@code{cls}). | ||
992 | |||
993 | @node Writing a user interface | ||
994 | @subsection Writing a user interface | ||
995 | |||
996 | Given a client library, all it takes to access a service now is to | ||
997 | combine calls to the client library with parsing command-line | ||
998 | options. | ||
999 | |||
1000 | Exercise: Call your client API from your @code{run()} method in your | ||
1001 | client application to send a request to the service. For example, | ||
1002 | send a 32-bit integer value based on a number given at the | ||
1003 | command-line to the service. | ||
1004 | |||
1005 | @node Writing a Service | ||
1006 | @section Writing a Service | ||
1007 | |||
1008 | Before you can test the client you've written so far, you'll | ||
1009 | need to also implement the corresponding service. | ||
1010 | |||
1011 | @menu | ||
1012 | * Code Placement:: | ||
1013 | * Starting a Service:: | ||
1014 | @end menu | ||
1015 | |||
1016 | @node Code Placement | ||
1017 | @subsection Code Placement | ||
1018 | |||
1019 | New services are placed in their own subdirectory under | ||
1020 | @file{gnunet/src}. This subdirectory should contain the API | ||
1021 | implementation file @file{SERVICE\_api.c}, the description of | ||
1022 | the client-service protocol @file{SERVICE.h} and P2P protocol | ||
1023 | @file{SERVICE\_protocol.h}, the implementation of the service itself | ||
1024 | @file{gnunet-service-SERVICE.h} and several files for tests, | ||
1025 | including test code and configuration files. | ||
1026 | |||
1027 | @node Starting a Service | ||
1028 | @subsection Starting a Service | ||
1029 | |||
1030 | The key API definition for creating a service is the | ||
1031 | @code{GNUNET\_SERVICE\_MAIN} macro: | ||
1032 | |||
1033 | @example | ||
1034 | @verbatiminclude tutorial-examples/007.c | ||
1035 | @end example | ||
1036 | |||
1037 | @noindent | ||
1038 | In addition to the service name and flags, the macro takes three | ||
1039 | functions, typically called @code{run}, @code{client\_connect\_cb} and | ||
1040 | @code{client\_disconnect\_cb} as well as an array of message handlers | ||
1041 | that will be called for incoming messages from clients. | ||
1042 | |||
1043 | A minimal version of the three central service funtions would look | ||
1044 | like this: | ||
1045 | |||
1046 | @example | ||
1047 | @verbatiminclude tutorial-examples/008.c | ||
1048 | @end example | ||
1049 | |||
1050 | @noindent | ||
1051 | Exercise: Write a stub service that processes no messages at all | ||
1052 | in your code. Create a default configuration for it, integrate it | ||
1053 | with the build system and start the service from | ||
1054 | @command{gnunet-service-arm} using @command{gnunet-arm -i NAME}. | ||
1055 | |||
1056 | Exercise: Figure out how to set the closure (@code{cls}) for handlers | ||
1057 | of a service. | ||
1058 | |||
1059 | Exercise: Figure out how to send messages from the service back to the | ||
1060 | client. | ||
1061 | |||
1062 | Each handler function in the service @b{must} eventually (possibly in some | ||
1063 | asynchronous continuation) call | ||
1064 | @code{GNUNET\_SERVICE\_client\_continue()}. Only after this call | ||
1065 | additional messages from the same client may | ||
1066 | be processed. This way, the service can throttle processing messages | ||
1067 | from the same client. | ||
1068 | |||
1069 | Exercise: Change the service to ``handle'' the message from your | ||
1070 | client (for now, by printing a message). What happens if you | ||
1071 | forget to call @code{GNUNET\_SERVICE\_client\_continue()}? | ||
1072 | |||
1073 | @node Interacting directly with other Peers using the CORE Service | ||
1074 | @section Interacting directly with other Peers using the CORE Service | ||
1075 | |||
1076 | FIXME: This section still needs to be updated to the lastest API! | ||
1077 | |||
1078 | One of the most important services in GNUnet is the @code{CORE} service | ||
1079 | managing connections between peers and handling encryption between peers. | ||
1080 | |||
1081 | One of the first things any service that extends the P2P protocol | ||
1082 | typically does is connect to the @code{CORE} service using: | ||
1083 | |||
1084 | @example | ||
1085 | @verbatiminclude tutorial-examples/009.c | ||
1086 | @end example | ||
1087 | |||
1088 | @menu | ||
1089 | * New P2P connections:: | ||
1090 | * Receiving P2P Messages:: | ||
1091 | * Sending P2P Messages:: | ||
1092 | * End of P2P connections:: | ||
1093 | @end menu | ||
1094 | |||
1095 | @node New P2P connections | ||
1096 | @subsection New P2P connections | ||
1097 | |||
1098 | Before any traffic with a different peer can be exchanged, the peer must | ||
1099 | be known to the service. This is notified by the @code{CORE} | ||
1100 | @code{connects} callback, which communicates the identity of the new | ||
1101 | peer to the service: | ||
1102 | |||
1103 | @example | ||
1104 | @verbatiminclude tutorial-examples/010.c | ||
1105 | @end example | ||
1106 | |||
1107 | @noindent | ||
1108 | Note that whatever you return from @code{connects} is given as the | ||
1109 | @code{cls} argument to the message handlers for messages from | ||
1110 | the respective peer. | ||
1111 | |||
1112 | Exercise: Create a service that connects to the @code{CORE}. Then | ||
1113 | start (and connect) two peers and print a message once your connect | ||
1114 | callback is invoked. | ||
1115 | |||
1116 | @node Receiving P2P Messages | ||
1117 | @subsection Receiving P2P Messages | ||
1118 | |||
1119 | To receive messages from @code{CORE}, you pass the desired | ||
1120 | @code{handlers} to the @code{GNUNET\_CORE\_connect()} function, | ||
1121 | just as we showed for services. | ||
1122 | |||
1123 | It is your responsibility to process messages fast enough or | ||
1124 | to implement flow control. If an application does not process | ||
1125 | CORE messages fast enough, CORE will randomly drop messages | ||
1126 | to not keep a very long queue in memory. | ||
1127 | |||
1128 | Exercise: Start one peer with a new service that has a message | ||
1129 | handler and start a second peer that only has your ``old'' service | ||
1130 | without message handlers. Which ``connect'' handlers are invoked when | ||
1131 | the two peers are connected? Why? | ||
1132 | |||
1133 | @node Sending P2P Messages | ||
1134 | @subsection Sending P2P Messages | ||
1135 | |||
1136 | You can transmit messages to other peers using the @code{mq} you were | ||
1137 | given during the @code{connect} callback. Note that the @code{mq} | ||
1138 | automatically is released upon @code{disconnect} and that you must | ||
1139 | not use it afterwards. | ||
1140 | |||
1141 | It is your responsibility to not over-fill the message queue, GNUnet | ||
1142 | will send the messages roughly in the order given as soon as possible. | ||
1143 | |||
1144 | Exercise: Write a service that upon connect sends messages as | ||
1145 | fast as possible to the other peer (the other peer should run a | ||
1146 | service that ``processes'' those messages). How fast is the | ||
1147 | transmission? Count using the STATISTICS service on both ends. Are | ||
1148 | messages lost? How can you transmit messages faster? What happens if | ||
1149 | you stop the peer that is receiving your messages? | ||
1150 | |||
1151 | @node End of P2P connections | ||
1152 | @subsection End of P2P connections | ||
1153 | |||
1154 | If a message handler returns @code{GNUNET\_SYSERR}, the remote | ||
1155 | peer shuts down or there is an unrecoverable network | ||
1156 | disconnection, CORE notifies the service that the peer disconnected. | ||
1157 | After this notification no more messages will be received from the | ||
1158 | peer and the service is no longer allowed to send messages to the peer. | ||
1159 | The disconnect callback looks like the following: | ||
1160 | |||
1161 | @example | ||
1162 | @verbatiminclude tutorial-examples/011.c | ||
1163 | @end example | ||
1164 | |||
1165 | @noindent | ||
1166 | Exercise: Fix your service to handle peer disconnects. | ||
1167 | |||
1168 | @node Storing peer-specific data using the PEERSTORE service | ||
1169 | @section Storing peer-specific data using the PEERSTORE service | ||
1170 | |||
1171 | GNUnet's PEERSTORE service offers a persistorage for arbitrary | ||
1172 | peer-specific data. Other GNUnet services can use the PEERSTORE | ||
1173 | to store, retrieve and monitor data records. Each data record | ||
1174 | stored with PEERSTORE contains the following fields: | ||
1175 | |||
1176 | @itemize | ||
1177 | @item subsystem: Name of the subsystem responsible for the record. | ||
1178 | @item peerid: Identity of the peer this record is related to. | ||
1179 | @item key: a key string identifying the record. | ||
1180 | @item value: binary record value. | ||
1181 | @item expiry: record expiry date. | ||
1182 | @end itemize | ||
1183 | |||
1184 | The first step is to start a connection to the PEERSTORE service: | ||
1185 | @example | ||
1186 | @verbatiminclude tutorial-examples/012.c | ||
1187 | @end example | ||
1188 | |||
1189 | The service handle @code{peerstore_handle} will be needed for | ||
1190 | all subsequent PEERSTORE operations. | ||
1191 | |||
1192 | @menu | ||
1193 | * Storing records:: | ||
1194 | * Retrieving records:: | ||
1195 | * Monitoring records:: | ||
1196 | * Disconnecting from PEERSTORE:: | ||
1197 | @end menu | ||
1198 | |||
1199 | @node Storing records | ||
1200 | @subsection Storing records | ||
1201 | |||
1202 | To store a new record, use the following function: | ||
1203 | |||
1204 | @example | ||
1205 | @verbatiminclude tutorial-examples/013.c | ||
1206 | @end example | ||
1207 | |||
1208 | @noindent | ||
1209 | The @code{options} parameter can either be | ||
1210 | @code{GNUNET_PEERSTORE_STOREOPTION_MULTIPLE} which means that multiple | ||
1211 | values can be stored under the same key combination | ||
1212 | (subsystem, peerid, key), or @code{GNUNET_PEERSTORE_STOREOPTION_REPLACE} | ||
1213 | which means that PEERSTORE will replace any existing values under the | ||
1214 | given key combination (subsystem, peerid, key) with the new given value. | ||
1215 | |||
1216 | The continuation function @code{cont} will be called after the store | ||
1217 | request is successfully sent to the PEERSTORE service. This does not | ||
1218 | guarantee that the record is successfully stored, only that it was | ||
1219 | received by the service. | ||
1220 | |||
1221 | The @code{GNUNET_PEERSTORE_store} function returns a handle to the store | ||
1222 | operation. This handle can be used to cancel the store operation only | ||
1223 | before the continuation function is called: | ||
1224 | |||
1225 | @example | ||
1226 | @verbatiminclude tutorial-examples/013.1.c | ||
1227 | @end example | ||
1228 | |||
1229 | @node Retrieving records | ||
1230 | @subsection Retrieving records | ||
1231 | |||
1232 | To retrieve stored records, use the following function: | ||
1233 | |||
1234 | @example | ||
1235 | @verbatiminclude tutorial-examples/014.c | ||
1236 | @end example | ||
1237 | |||
1238 | @noindent | ||
1239 | The values of @code{peer} and @code{key} can be @code{NULL}. This | ||
1240 | allows the iteration over values stored under any of the following | ||
1241 | key combinations: | ||
1242 | |||
1243 | @itemize | ||
1244 | @item (subsystem) | ||
1245 | @item (subsystem, peerid) | ||
1246 | @item (subsystem, key) | ||
1247 | @item (subsystem, peerid, key) | ||
1248 | @end itemize | ||
1249 | |||
1250 | The @code{callback} function will be called once with each retrieved | ||
1251 | record and once more with a @code{NULL} record to signal the end of | ||
1252 | results. | ||
1253 | |||
1254 | The @code{GNUNET_PEERSTORE_iterate} function returns a handle to the | ||
1255 | iterate operation. This handle can be used to cancel the iterate | ||
1256 | operation only before the callback function is called with a | ||
1257 | @code{NULL} record. | ||
1258 | |||
1259 | @node Monitoring records | ||
1260 | @subsection Monitoring records | ||
1261 | |||
1262 | PEERSTORE offers the functionality of monitoring for new records | ||
1263 | stored under a specific key combination (subsystem, peerid, key). | ||
1264 | To start the monitoring, use the following function: | ||
1265 | |||
1266 | @example | ||
1267 | @verbatiminclude tutorial-examples/015.c | ||
1268 | @end example | ||
1269 | |||
1270 | @noindent | ||
1271 | Whenever a new record is stored under the given key combination, | ||
1272 | the @code{callback} function will be called with this new | ||
1273 | record. This will continue until the connection to the PEERSTORE | ||
1274 | service is broken or the watch operation is canceled: | ||
1275 | |||
1276 | @example | ||
1277 | @verbatiminclude tutorial-examples/016.c | ||
1278 | @end example | ||
1279 | |||
1280 | @node Disconnecting from PEERSTORE | ||
1281 | @subsection Disconnecting from PEERSTORE | ||
1282 | |||
1283 | When the connection to the PEERSTORE service is no longer needed, | ||
1284 | disconnect using the following function: | ||
1285 | |||
1286 | @example | ||
1287 | @verbatiminclude tutorial-examples/017.c | ||
1288 | @end example | ||
1289 | |||
1290 | @noindent | ||
1291 | If the @code{sync_first} flag is set to @code{GNUNET_YES}, | ||
1292 | the API will delay the disconnection until all store requests | ||
1293 | are received by the PEERSTORE service. Otherwise, it will | ||
1294 | disconnect immediately. | ||
1295 | |||
1296 | @node Using the DHT | ||
1297 | @section Using the DHT | ||
1298 | |||
1299 | The DHT allows to store data so other peers in the P2P network can | ||
1300 | access it and retrieve data stored by any peers in the network. | ||
1301 | This section will explain how to use the DHT. Of course, the first | ||
1302 | thing to do is to connect to the DHT service: | ||
1303 | |||
1304 | @example | ||
1305 | @verbatiminclude tutorial-examples/018.c | ||
1306 | @end example | ||
1307 | |||
1308 | @noindent | ||
1309 | The second parameter indicates how many requests in parallel to expect. | ||
1310 | It is not a hard limit, but a good approximation will make the DHT more | ||
1311 | efficient. | ||
1312 | |||
1313 | @menu | ||
1314 | * Storing data in the DHT:: | ||
1315 | * Obtaining data from the DHT:: | ||
1316 | * Implementing a block plugin:: | ||
1317 | * Monitoring the DHT:: | ||
1318 | @end menu | ||
1319 | |||
1320 | @node Storing data in the DHT | ||
1321 | @subsection Storing data in the DHT | ||
1322 | Since the DHT is a dynamic environment (peers join and leave frequently) | ||
1323 | the data that we put in the DHT does not stay there indefinitely. It is | ||
1324 | important to ``refresh'' the data periodically by simply storing it | ||
1325 | again, in order to make sure other peers can access it. | ||
1326 | |||
1327 | The put API call offers a callback to signal that the PUT request has been | ||
1328 | sent. This does not guarantee that the data is accessible to others peers, | ||
1329 | or even that is has been stored, only that the service has requested to | ||
1330 | a neighboring peer the retransmission of the PUT request towards its final | ||
1331 | destination. Currently there is no feedback about whether or not the data | ||
1332 | has been sucessfully stored or where it has been stored. In order to | ||
1333 | improve the availablilty of the data and to compensate for possible | ||
1334 | errors, peers leaving and other unfavorable events, just make several | ||
1335 | PUT requests! | ||
1336 | |||
1337 | @example | ||
1338 | @verbatiminclude tutorial-examples/019.c | ||
1339 | @end example | ||
1340 | |||
1341 | @noindent | ||
1342 | Exercise: Store a value in the DHT periodically to make sure it | ||
1343 | is available over time. You might consider using the function | ||
1344 | @code{GNUNET\_SCHEDULER\_add\_delayed} and call | ||
1345 | @code{GNUNET\_DHT\_put} from inside a helper function. | ||
1346 | |||
1347 | @node Obtaining data from the DHT | ||
1348 | @subsection Obtaining data from the DHT | ||
1349 | |||
1350 | As we saw in the previous example, the DHT works in an asynchronous mode. | ||
1351 | Each request to the DHT is executed ``in the background'' and the API | ||
1352 | calls return immediately. In order to receive results from the DHT, the | ||
1353 | API provides a callback. Once started, the request runs in the service, | ||
1354 | the service will try to get as many results as possible (filtering out | ||
1355 | duplicates) until the timeout expires or we explicitly stop the request. | ||
1356 | It is possible to give a ``forever'' timeout with | ||
1357 | @code{GNUNET\_TIME\_UNIT\_FOREVER\_REL}. | ||
1358 | |||
1359 | If we give a route option @code{GNUNET\_DHT\_RO\_RECORD\_ROUTE} | ||
1360 | the callback will get a list of all the peers the data has travelled, | ||
1361 | both on the PUT path and on the GET path. | ||
1362 | |||
1363 | @example | ||
1364 | @verbatiminclude tutorial-examples/020.c | ||
1365 | @end example | ||
1366 | |||
1367 | @noindent | ||
1368 | Exercise: Store a value in the DHT and after a while retrieve it. | ||
1369 | Show the IDs of all the peers the requests have gone through. | ||
1370 | In order to convert a peer ID to a string, use the function | ||
1371 | @code{GNUNET\_i2s}. Pay attention to the route option parameters | ||
1372 | in both calls! | ||
1373 | |||
1374 | @node Implementing a block plugin | ||
1375 | @subsection Implementing a block plugin | ||
1376 | |||
1377 | In order to store data in the DHT, it is necessary to provide a block | ||
1378 | plugin. The DHT uses the block plugin to ensure that only well-formed | ||
1379 | requests and replies are transmitted over the network. | ||
1380 | |||
1381 | The block plugin should be put in a file @file{plugin\_block\_SERVICE.c} | ||
1382 | in the service's respective directory. The | ||
1383 | mandatory functions that need to be implemented for a block plugin are | ||
1384 | described in the following sections. | ||
1385 | |||
1386 | @menu | ||
1387 | * Validating requests and replies:: | ||
1388 | * Deriving a key from a reply:: | ||
1389 | * Initialization of the plugin:: | ||
1390 | * Shutdown of the plugin:: | ||
1391 | * Integration of the plugin with the build system:: | ||
1392 | @end menu | ||
1393 | |||
1394 | @node Validating requests and replies | ||
1395 | @subsubsection Validating requests and replies | ||
1396 | |||
1397 | The evaluate function should validate a reply or a request. It returns | ||
1398 | a @code{GNUNET\_BLOCK\_EvaluationResult}, which is an enumeration. All | ||
1399 | possible answers are in @file{gnunet\_block\_lib.h}. The function will | ||
1400 | be called with a @code{reply\_block} argument of @code{NULL} for | ||
1401 | requests. Note that depending on how @code{evaluate} is called, only | ||
1402 | some of the possible return values are valid. The specific meaning of | ||
1403 | the @code{xquery} argument is application-specific. Applications that | ||
1404 | do not use an extended query should check that the @code{xquery\_size} | ||
1405 | is zero. The block group is typically used to filter duplicate | ||
1406 | replies. | ||
1407 | |||
1408 | @example | ||
1409 | @verbatiminclude tutorial-examples/021.c | ||
1410 | @end example | ||
1411 | |||
1412 | @noindent | ||
1413 | Note that it is mandatory to detect duplicate replies in this function | ||
1414 | and return the respective status code. Duplicate detection is | ||
1415 | typically done using the Bloom filter block group provided by | ||
1416 | @file{libgnunetblockgroup.so}. Failure to do so may cause replies to | ||
1417 | circle in the network. | ||
1418 | |||
1419 | @node Deriving a key from a reply | ||
1420 | @subsubsection Deriving a key from a reply | ||
1421 | |||
1422 | The DHT can operate more efficiently if it is possible to derive a key | ||
1423 | from the value of the corresponding block. The @code{get\_key} | ||
1424 | function is used to obtain the key of a block --- for example, by | ||
1425 | means of hashing. If deriving the key is not possible, the function | ||
1426 | should simply return @code{GNUNET\_SYSERR} (the DHT will still work | ||
1427 | just fine with such blocks). | ||
1428 | |||
1429 | @example | ||
1430 | @verbatiminclude tutorial-examples/022.c | ||
1431 | @end example | ||
1432 | |||
1433 | @node Initialization of the plugin | ||
1434 | @subsubsection Initialization of the plugin | ||
1435 | |||
1436 | The plugin is realized as a shared C library. The library must export | ||
1437 | an initialization function which should initialize the plugin. The | ||
1438 | initialization function specifies what block types the plugin cares | ||
1439 | about and returns a struct with the functions that are to be used for | ||
1440 | validation and obtaining keys (the ones just defined above). | ||
1441 | |||
1442 | @example | ||
1443 | @verbatiminclude tutorial-examples/023.c | ||
1444 | @end example | ||
1445 | |||
1446 | @node Shutdown of the plugin | ||
1447 | @subsubsection Shutdown of the plugin | ||
1448 | |||
1449 | Following GNUnet's general plugin API concept, the plugin must | ||
1450 | export a second function for cleaning up. It usually does very | ||
1451 | little. | ||
1452 | |||
1453 | @example | ||
1454 | @verbatiminclude tutorial-examples/024.c | ||
1455 | @end example | ||
1456 | |||
1457 | @node Integration of the plugin with the build system | ||
1458 | @subsubsection Integration of the plugin with the build system | ||
1459 | |||
1460 | In order to compile the plugin, the @file{Makefile.am} file for the | ||
1461 | service SERVICE should contain a rule similar to this: | ||
1462 | @c Actually this is a Makefile not C. But the whole structure of examples | ||
1463 | @c must be improved. | ||
1464 | |||
1465 | @example | ||
1466 | @verbatiminclude tutorial-examples/025.c | ||
1467 | @end example | ||
1468 | |||
1469 | @noindent | ||
1470 | Exercise: Write a block plugin that accepts all queries | ||
1471 | and all replies but prints information about queries and replies | ||
1472 | when the respective validation hooks are called. | ||
1473 | |||
1474 | @node Monitoring the DHT | ||
1475 | @subsection Monitoring the DHT | ||
1476 | |||
1477 | It is possible to monitor the functioning of the local | ||
1478 | DHT service. When monitoring the DHT, the service will | ||
1479 | alert the monitoring program of any events, both started | ||
1480 | locally or received for routing from another peer. | ||
1481 | The are three different types of events possible: a | ||
1482 | GET request, a PUT request or a response (a reply to a GET). | ||
1483 | |||
1484 | Since the different events have different associated data, | ||
1485 | the API gets 3 different callbacks (one for each message type) | ||
1486 | and optional type and key parameters, to allow for filtering of | ||
1487 | messages. When an event happens, the appropiate callback is | ||
1488 | called with all the information about the event. | ||
1489 | |||
1490 | @example | ||
1491 | @verbatiminclude tutorial-examples/026.c | ||
1492 | @end example | ||
1493 | |||
1494 | @node Debugging with gnunet-arm | ||
1495 | @section Debugging with gnunet-arm | ||
1496 | |||
1497 | Even if services are managed by @command{gnunet-arm}, you can | ||
1498 | start them with @command{gdb} or @command{valgrind}. For | ||
1499 | example, you could add the following lines to your | ||
1500 | configuration file to start the DHT service in a @command{gdb} | ||
1501 | session in a fresh @command{xterm}: | ||
1502 | |||
1503 | @example | ||
1504 | [dht] | ||
1505 | PREFIX=xterm -e gdb --args | ||
1506 | @end example | ||
1507 | |||
1508 | @noindent | ||
1509 | Alternatively, you can stop a service that was started via | ||
1510 | ARM and run it manually: | ||
1511 | |||
1512 | @example | ||
1513 | $ gnunet-arm -k dht | ||
1514 | $ gdb --args gnunet-service-dht -L DEBUG | ||
1515 | $ valgrind gnunet-service-dht -L DEBUG | ||
1516 | @end example | ||
1517 | |||
1518 | @noindent | ||
1519 | Assuming other services are well-written, they will automatically | ||
1520 | re-integrate the restarted service with the peer. | ||
1521 | |||
1522 | GNUnet provides a powerful logging mechanism providing log | ||
1523 | levels @code{ERROR}, @code{WARNING}, @code{INFO} and @code{DEBUG}. | ||
1524 | The current log level is configured using the @code{$GNUNET_FORCE_LOG} | ||
1525 | environmental variable. The @code{DEBUG} level is only available if | ||
1526 | @command{--enable-logging=verbose} was used when running | ||
1527 | @command{configure}. More details about logging can be found under | ||
1528 | @uref{https://gnunet.org/logging}. | ||
1529 | |||
1530 | You should also probably enable the creation of core files, by setting | ||
1531 | @code{ulimit}, and echo'ing @code{1} into | ||
1532 | @file{/proc/sys/kernel/core\_uses\_pid}. Then you can investigate the | ||
1533 | core dumps with @command{gdb}, which is often the fastest method to | ||
1534 | find simple errors. | ||
1535 | |||
1536 | Exercise: Add a memory leak to your service and obtain a trace | ||
1537 | pointing to the leak using @command{valgrind} while running the service | ||
1538 | from @command{gnunet-service-arm}. | ||
1539 | |||
1540 | @bye | ||