aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--doc/gnunet-c-tutorial.pdfbin0 -> 329226 bytes
-rw-r--r--doc/gnunet-c-tutorial.tex1505
2 files changed, 1505 insertions, 0 deletions
diff --git a/doc/gnunet-c-tutorial.pdf b/doc/gnunet-c-tutorial.pdf
new file mode 100644
index 000000000..5636dde4c
--- /dev/null
+++ b/doc/gnunet-c-tutorial.pdf
Binary files differ
diff --git a/doc/gnunet-c-tutorial.tex b/doc/gnunet-c-tutorial.tex
new file mode 100644
index 000000000..73c376b0b
--- /dev/null
+++ b/doc/gnunet-c-tutorial.tex
@@ -0,0 +1,1505 @@
1\documentclass[10pt]{article}
2\usepackage[ansinew]{inputenc}
3\usepackage{makeidx,amsmath,amssymb,exscale,multicol,epsfig,graphics,verbatim,ulem}
4\usepackage{epsfig,geometry,url,listings, subcaption}
5\usepackage{boxedminipage}
6\usepackage[T1]{fontenc}%required
7\usepackage{textcomp}
8\geometry{headsep=3ex,hscale=0.9}
9\usepackage{hyperref}
10\hypersetup{pdftitle={GNUnet C Tutorial},
11 pdfsubject={GNUnet},
12 pdfauthor={Christian Grothoff <christian@grothoff.org>},
13 pdfkeywords={p2p,search,gnunet,tutorial}
14 %,pdfpagemode={FullScreen}
15 }
16
17
18\lstset{
19language=bash,
20basicstyle=\ttfamily,
21upquote=true,
22columns=fullflexible,
23literate={*}{{\char42}}1
24 {-}{{\char45}}1
25}
26
27\newcommand{\exercise}[1]{\noindent\begin{boxedminipage}{\textwidth}{\bf Exercise:} #1 \end{boxedminipage}}
28
29\begin{document}
30
31\begin{center}
32\large {A Tutorial for GNUnet 0.9.x (C version)}
33
34Christian Grothoff $\qquad$ Bart Polot $\qquad$ Matthias Wachs
35
36\today
37\end{center}
38This tutorials explains how to install GNUnet on a GNU/Linux system ond gives an introduction how
39GNUnet can be used to develop a Peer-to-Peer application. Detailed installation instructions for
40various operating systems and a detailed list of all dependencies can found on our website at
41\url{https://gnunet.org/installation}.
42
43\textbf{Please read this tutorial carefully since every single step is important and do not hesitate to contact the GNUnet team if you have any questions or problems! Check here how to contact the GNUnet team:
44\url{https://gnunet.org/contact_information}}
45
46
47\section{Installing GNUnet}
48First of all you have to install a current version of GNUnet. You can download a
49tarball of a stable version from GNU FTP mirrors or obtain the latest development
50version from our Subversion repository.
51
52Most of the time you should prefer to download the stable version since with the
53latest development version things can be broken, functionality can be changed or tests
54can fail. You should only use the development version if you know that you require a
55certain feature or a certain issue has been fixed since the last release.
56
57\subsection{Obtaining a stable version}
58You can download the latest stable version of GNUnet from GNU FTP mirrors:
59\begin{center}
60\url{ftp://ftp.gnu.org/gnu/gnunet/gnunet-0.9.5a.tar.gz}
61\end{center}
62You should also download the signature file and verify the integrity of the tarball.
63\begin{center}
64\url{ftp://ftp.gnu.org/gnu/gnunet/gnunet-0.9.5a.tar.gz.sig}
65\end{center}
66To verify the signature you should first import the GPG key used to sign the tarball
67\begin{lstlisting}
68$ gpg --keyserver keys.gnupg.net --recv-keys 48426C7E
69\end{lstlisting}
70And use this key to verify the tarball's signature
71\begin{lstlisting}
72$ gpg --verify gnunet-0.9.5a.tar.gz.sig gnunet-0.9.5a.tar.gz
73\end{lstlisting}
74After successfully verifying the integrity you can extract the tarball using
75\begin{lstlisting}
76$ tar xvzf gnunet-0.9.5a.tar.gz
77$ mv gnunet-0.9.5a gnunet # we will use the directory "gnunet" in the reminder of this document
78$ cd gnunet
79\end{lstlisting}
80
81\subsection{Installing Build Tool Chain and Dependencies}
82To successfully compile GNUnet you need the tools to build GNUnet and the required dependencies.
83Please have a look at \url{https://gnunet.org/dependencies} for a list of required dependencies
84and \url{https://gnunet.org/generic_installation} for specific instructions for your operating system.
85
86Please check the notes at the end of the configure process about required dependencies.
87
88For GNUNet bootstrapping support and the http(s) plugin you should install \texttt{libcurl}.
89For the filesharing service you should install at least one of the datastore backends \texttt{mysql},
90\texttt{sqlite} or \texttt{postgresql}.
91
92\subsection{Obtaining the latest version from Subversion}
93The latest development version can obtained from our Subversion (\textit{svn}) repository. To obtain
94the code you need Subversion installed and checkout the repository using:
95\lstset{language=bash}
96\begin{lstlisting}
97$ svn checkout https://gnunet.org/svn/gnunet
98\end{lstlisting}
99After cloning the repository you have to execute
100\lstset{language=bash}
101\begin{lstlisting}
102$ cd gnunet
103$ ./bootstrap
104\end{lstlisting}
105
106
107\subsection{Compiling and Installing GNUnet}
108\label{sub:install}
109Assuming all dependencies are installed, the following commands will compile and install GNUnet in your
110home directory. You can specify the directory where GNUnet will be installed by changing the \lstinline|--prefix| value when calling \lstinline|./configure|. If you do not specifiy a prefix, GNUnet is installed in the directory \lstinline|/usr/local|. When developing new applications you may want to enable
111verbose logging by adding \lstinline|--enable-logging=verbose|:
112
113\lstset{language=bash}
114\begin{lstlisting}
115$ ./configure --prefix=$HOME --enable-logging
116$ make
117$ make install
118\end{lstlisting}
119
120After installing GNUnet you have to set the \lstinline|GNUNET_PREFIX| environmental variable used by GNUnet to detect it's installation directory and add your GNUnet installation to your path environmental variable.
121This configuration is only valid for the current shell session, so you should add \lstinline|export GNUNET_PREFIX=$HOME| to your \lstinline|.bash_rc| or \lstinline|.profile| to be sure the environment variable is always set. In addition you have to create the \lstinline|.gnunet| directory in your home directory where GNUnet stores it's data and an empty GNUnet configuration file:
122
123\lstset{language=bash}
124\begin{lstlisting}
125$ export GNUNET_PREFIX=$HOME
126$ export PATH=$PATH:$GNUNET_PREFIX/bin
127$ echo export GNUNET_PREFIX=$HOME >> ~/.bashrc
128$ echo export PATH=$GNUNET_PREFIX/bin:$PATH >> ~/.bashrc
129$ mkdir ~/.gnunet/
130$ touch ~/.gnunet/gnunet.conf
131\end{lstlisting}
132% $
133
134\subsection{Common Issues - Check your GNUnet installation}
135You should check your installation to ensure that installing GNUnet was successful up to this point. You should be able to access GNUnet's binaries and run GNUnet's self check.
136\begin{lstlisting}
137$ which gnunet-arm
138\end{lstlisting}
139should return \lstinline|$GNUNET_PREFIX/bin/gnunet-arm|. It should be located in your GNUnet installation and the output should not be empty. If you see an output like:
140\begin{lstlisting}
141$ which gnunet-arm
142$
143\end{lstlisting}
144check your {\tt PATH} variable to ensure GNUnet's {\tt bin} directory is included.
145
146GNUnet provides tests for all of it's subcomponents. Run
147\begin{lstlisting}
148$ make check
149\end{lstlisting}
150to execute tests for all components. {\tt make check} traverses all subdirectories in {\tt src}.
151For every subdirectory you should get a message like this:
152
153\begin{lstlisting}
154make[2]: Entering directory `/home/mwachs/gnunet/contrib'
155PASS: test_gnunet_prefix
156=============
1571 test passed
158=============
159\end{lstlisting}
160
161If you see a message like this:
162
163\begin{lstlisting}
164Mar 12 16:57:56-642482 resolver-api-19449 ERROR Must specify `HOSTNAME' for `resolver' in configuration!
165Mar 12 16:57:56-642573 test_program-19449 ERROR Assertion failed at resolver_api.c:204.
166/bin/bash: line 5: 19449 Aborted (core dumped) ${dir}$tst
167FAIL: test_program
168\end{lstlisting}
169double check your {\tt GNUNET\_PREFIX} environmental variable and double check the steps performed in ~\ref{sub:install}
170
171\section{Background: GNUnet Architecture}
172GNUnet is organized in layers and services. Each service is composed of a
173main service implementation and a client library for other programs to use
174the service's functionality, described by an API. This approach is shown in
175figure~\ref{fig:service}. Some services provide an additional command line
176tool to enable the user to interact with the service.
177
178Very often it is other GNUnet services that will use these APIs to build the
179higher layers of GNUnet on top of the lower ones. Each layer expands or extends
180the functionality of the service below (for instance, to build a mesh on top of
181a DHT). See figure ~\ref{fig:interaction} for an illustration of this approach.
182
183\begin{figure}[!h]
184 \begin{center}
185% \begin{subfigure}
186 \begin{subfigure}[b]{0.3\textwidth}
187 \centering
188 \includegraphics[width=\textwidth]{figs/Service.pdf}
189 \caption{Service with API and network protocol}
190 \label{fig:service}
191 \end{subfigure}
192 ~~~~~~~~~~
193 \begin{subfigure}[b]{0.3\textwidth}
194 \centering
195 \includegraphics[width=\textwidth]{figs/System.pdf}
196 \caption{Service interaction}
197 \label{fig:interaction}
198 \end{subfigure}
199 \end{center}
200 \caption{GNUnet's layered system architecture}
201\end{figure}
202
203The main service implementation runs as a standalone process in the operating
204system and the client code runs as part of the client program, so crashes of a
205client do not affect the service process or other clients. The service and the
206clients communicate via a message protocol to be defined and implemented by
207the programmer.
208
209\section{First Steps with GNUnet}
210
211\subsection{Configure your peer}
212First of all we need to configure your peer. Each peer is started with a configuration containing settings for GNUnet itself and it's services. This configuration is based on the default configuration shipped with GNUnet and can be modified. The default configuration is located in the {\tt \$GNUNET\_PREFIX/share/gnunet/config.d} directory. When starting a peer, you can specify a customized configuration using the the {\tt$-c$} command line switch when starting the ARM service and all other services. When using a modified configuration the default values are loaded and only values specified in the configuration file will replace the default values.
213
214Since we want to start additional peers later, we need
215some modifications from the default configuration. We need to create a separate service home and a file containing our modifications for this peer:
216\begin{lstlisting}
217$ mkdir ~/gnunet1/
218$ touch peer1.conf
219\end{lstlisting}
220
221Now add the following lines to peer1.conf to use this directory. For simplified usage we want to prevent
222the peer to connect to the GNUnet network since this could lead to confusing output. This modifications will replace the default settings:
223\begin{lstlisting}
224$ [PATHS]
225$ SERVICEHOME = ~/gnunet1/ # Use this directory to store GNUnet data
226$ [hostlist]
227$ SERVERS = # prevent bootstrapping
228\end{lstlisting}
229
230\subsection{Start a peer}
231Each GNUnet instance (called peer) has an identity (\textit{peer ID}) based on a
232cryptographic public private key pair. The peer ID is the printable hash of the
233public key. So before starting the peer, you may want to just generate the peer's private
234key using the command
235\lstset{language=bash}
236\begin{lstlisting}
237$ gnunet-peerinfo -c ~/peer1.conf -s
238\end{lstlisting}
239You should see an output containing the peer ID similar to:
240\lstset{language=bash}
241\begin{lstlisting}
242I am peer `0PA02UVRKQTS2C .. JL5Q78F6H0B1ACPV1CJI59MEQUMQCC5G'.
243\end{lstlisting}
244
245GNUnet services are controlled by a master service the so called \textit{Automatic Restart Manager} (ARM).
246ARM starts, stops and even restarts services automatically or on demand when a client connects.
247You interact with the ARM service using the \lstinline|gnunet-arm| tool.
248GNUnet can then be started with \lstinline|gnunet-arm -s| and stopped with
249\lstinline|gnunet-arm -e|. An additional service not automatically started
250can be started using \lstinline|gnunet-arm -i <service name>| and stopped
251using \lstinline|gnunet-arm -k <servicename>|.
252
253\subsection{Monitor a peer}
254In this section, we will monitor the behaviour of our peer's DHT service with respect to a
255specific key. First we will start GNUnet and then start the DHT service and use the DHT monitor tool
256to monitor the PUT and GET commands we issue ussing the \lstinline|gnunet-dht-put| and
257\lstinline|gnunet-dht-get| command. Using the ``monitor'' line given below, you can observe the behavior of
258your own peer's DHT with respect to the specified KEY:
259
260\lstset{language=bash}
261\begin{lstlisting}
262$ gnunet-arm -c ~/peer1.conf -s # start gnunet with all default services
263$ gnunet-arm -c ~/peer1.conf -i dht # start DHT service
264$ cd ~/gnunet/src/dht;
265$ ./gnunet-dht-monitor -c ~/gnunet1/peer1.conf -k KEY
266\end{lstlisting}
267Now open a separate terminal and change again to the \lstinline|gnunet/src/dht| directory:
268\begin{lstlisting}
269$ cd ~/gnunet/src/dht
270$ ./gnunet-dht-put -c ~/peer1.conf -k KEY -d VALUE # put VALUE under KEY in the DHT
271$ ./gnunet/src/dht/gnunet-dht-get -k KEY # get key KEY from the DHT
272$ gnunet-statistics -c ~/peer1.conf # print statistics about current GNUnet state
273$ gnunet-statistics -c ~/peer1.conf -s dht # print statistics about DHT service
274\end{lstlisting}
275% $
276\subsection{Starting Two Peers by Hand}
277\subsubsection{Setup a second peer}
278We will now start a second peer on your machine.
279For the second peer, you will need to manually create a modified
280configuration file to avoid conflicts with ports and directories.
281A peers configuration file is by default located in {\tt ~/.gnunet/gnunet.conf}.
282This file is typically very short or event empty as only the differences to the
283defaults need to be specified. The defaults are located in
284many files in the {\tt \$GNUNET\_PREFIX/share/gnunet/config.d} directory.
285
286To configure the second peer, use the files {\tt
287 \$GNUNET\_PREFIX/share/gnunet/config.d} as a template for your main
288configuration file:
289%
290\lstset{language=bash}
291\begin{lstlisting}
292$ cat $GNUNET_PREFIX/share/gnunet/config.d/*.conf > peer2.conf
293\end{lstlisting}
294Now you have to edit {\tt peer2.conf} and change:
295\begin{itemize}
296 \itemsep0em
297 \item{\texttt{SERVICEHOME} under \texttt{PATHS}}
298 \item{Every value for ``\texttt{PORT}'' (add 10000) in any section (if \texttt{PORT} is enabled, may be disabled using "\#") }
299 \item{Every value for ``\texttt{UNIXPATH}'' in any section (e.g. by adding a "-p2" suffix)}
300\end{itemize}
301to a fresh, unique value. Make sure that the \texttt{PORT} numbers stay
302below 65536. From now on, whenever you interact with the second
303peer, you need to specify {\tt -c peer2.conf} as an additional
304command line argument.
305
306Now, generate the 2nd peer's private key:
307
308\lstset{language=bash}
309\begin{lstlisting}
310$ gnunet-peerinfo -s -c peer2.conf
311\end{lstlisting}
312% $
313
314This may take a while, generate entropy using your keyboard or mouse
315as needed. Also, make sure the output is different from the {\tt
316 gnunet-peerinfo} output for the first peer (otherwise you made an
317error in the configuration).
318
319\subsubsection{Start the second peer and connect the peers}
320Then, you can start a second peer using:
321\lstset{language=bash}
322\begin{lstlisting}
323$ gnunet-arm -c peer2.conf -s
324$ gnunet-arm -c peer2.conf -i dht
325$ ~/gnunet/src/dht/gnunet-dht-put -c peer2.conf -k KEY -d VALUE
326$ ~/gnunet/src/dht/gnunet-dht-get -c peer2.conf -k KEY
327\end{lstlisting}
328If you want the two peers to connect, you have multiple options:
329\begin{itemize}
330\itemsep0em
331 \item UDP neighbour discovery (automatic)
332 \item Setup a bootstrap server
333 \item Connect manually
334\end{itemize}
335To setup peer 1 as bootstrapping server change the configuration of the first one to be a hostlist server by adding the following lines to \texttt{peer1.conf} to enable bootstrapping server:
336 \begin{lstlisting}
337[hostlist]
338OPTIONS = -p
339\end{lstlisting}
340
341Then change {\tt peer2.conf} and replace the ``\texttt{SERVERS}'' line in the ``\texttt{[hostlist]}'' section with
342``\texttt{http://localhost:8080/}''. Restart both peers using:
343\begin{lstlisting}
344$ gnunet-arm -c peer1.conf -e # stop first peer
345$ gnunet-arm -c peer1.conf -s # start first peer
346$ gnunet-arm -c peer2.conf -s # start second peer
347\end{lstlisting}
348
349Note that if you start your peers without changing these settings, they
350will use the ``global'' hostlist servers of the GNUnet P2P network and
351likely connect to those peers. At that point, debugging might become
352tricky as you're going to be connected to many more peers and would
353likely observe traffic and behaviors that are not explicitly controlled
354by you.
355
356\subsubsection{How to connect manually}
357If you want to use the \texttt{peerinfo} tool to connect your peers, you should:
358\begin{itemize}
359\itemsep0em
360 \item{Remove {\tt hostlist} from {\tt DEFAULTSERVICES} (to not connect to the global GNUnet)}
361 \item{Start both peers running {\tt gnunet-arm -c peer1.conf -s} and {\tt gnunet-arm -c peer2.conf -s}}
362 \item{Get \texttt{HELLO} message of the first peer running {\tt gnunet-peerinfo -c peer1.conf -g}}
363 \item{Give the output to the second peer by running {\tt gnunet-peerinfo -c peer2.conf -p '<output>'}}
364\end{itemize}
365
366Check that they are connected using {\tt gnunet-core -c peer1.conf}, which should give you the other peer's
367peer identity:
368\begin{lstlisting}
369$ gnunet-core -c peer1.conf
370Peer `9TVUCS8P5A7ILLBGO6JSTSSN2B44H3D2MUIFJMLKAITC0I22UVFBFP1H8NRK2IA35VKAK16LLO0MFS7TAQ9M1KNBJ4NGCHP3JPVULDG'
371\end{lstlisting}
372
373\subsection{Starting Peers Using the Testbed Service}
374
375GNUnet's testbed service is used for testing scenarios where a number of peers
376are to be started. The testbed can manage peers on a single host or on multiple
377hosts in a distributed fashion. On a single affordable computer, it should be
378possible to run around 100 peers without drastically increasing the load on the
379system.
380
381The testbed service can be access through its API
382\texttt{include/gnunet\_testbed\_service.h}. The API provides many routines for
383managing a testbed. It also provides a helper function
384\texttt{GNUNET\_TESTBED\_test\_run()} to quickly setup a minimalistic testing
385environment on a single host.
386
387This function takes a configuration file which will be used as a template
388configuration for the peers. The testbed takes care of modifying relevant
389options in the peers' configuration such as SERVICEHOME, PORT, UNIXPATH to
390unique values so that peers run without running into conflicts. It also checks
391and assigns the ports in configurations only if they are free.
392
393Additionally, the testbed service also reads its options from the same
394configuration file. Various available options and details about them can be
395found in the testbed default configuration file \texttt{testbed/testbed.conf}.
396
397With the testbed API, a sample test case can be structured as follows:
398\lstset{language=c}
399\begin{lstlisting}
400/* Number of peers we want to start */
401#define NUM_PEERS 30
402
403struct GNUNET_TESTBED_Operation *dht_op;
404
405struct GNUNET_DHT_Handle *dht_handle;
406
407struct MyContext
408{
409 int ht_len;
410} ctxt;
411
412static void finish () /* Finish test case */
413{
414 if (NULL != dht_op)
415 {
416 GNUNET_TESTBED_operation_done (dht_op); /* calls the dht_da() for closing
417 down the connection */
418 dht_op = NULL;
419 }
420 result = GNUNET_OK;
421 GNUNET_SCHEDULER_shutdown (); /* Also kills the testbed */
422}
423
424
425static void
426service_connect_comp (void *cls,
427 struct GNUNET_TESTBED_Operation *op,
428 void *ca_result,
429 const char *emsg)
430{
431 /* Service to DHT successful; do something */
432}
433
434
435static void *
436dht_ca (void *cls, const struct GNUNET_CONFIGURATION_Handle *cfg)
437{
438 struct MyContext *ctxt = cls;
439
440 /* Use the provided configuration to connect to service */
441 dht_handle = GNUNET_DHT_connect (cfg, ctxt->ht_len);
442 return dht_handle;
443}
444
445
446static void
447dht_da (void *cls, void *op_result)
448{
449 struct MyContext *ctxt = cls;
450
451 /* Disconnect from DHT service */
452 GNUNET_DHT_disconnect ((struct GNUNET_DHT_Handle *) op_result);
453 ctxt->ht_len = 0;
454 dht_handle = NULL;
455}
456
457static void
458test_master (void *cls, unsigned int num_peers,
459 struct GNUNET_TESTBED_Peer **peers)
460{
461 /* Testbed is ready with peers running and connected in a pre-defined overlay
462 topology */
463
464 /* do something */
465 ctxt.ht_len = 10;
466
467 /* connect to a peers service */
468 dht_op = GNUNET_TESTBED_service_connect
469 (NULL, /* Closure for operation */
470 peers[0], /* The peer whose service to connect to */
471 "dht" /* The name of the service */
472 service_connect_comp, /* callback to call after a handle to service
473 is opened */
474 NULL, /* closure for the above callback */
475 dht_ca, /* callback to call with peer's configuration;
476 this should open the needed service connection */
477 dht_da, /* callback to be called when closing the
478 opened service connection */
479 &ctxt); /* closure for the above two callbacks */
480}
481
482
483int
484main (int argc, char **argv)
485{
486 int ret;
487
488 ret = GNUNET_TESTBED_test_run
489 ("awesome-test", /* test case name */
490 "template.conf", /* template configuration */
491 NUM_PEERS, /* number of peers to start */
492 0LL, /* Event mask - set to 0 for no event notifications */
493 NULL, /* Controller event callback */
494 NULL, /* Closure for controller event callback */
495 &test_master, /* continuation callback to be called when testbed setup is
496 complete */
497 NULL); /* Closure for the test_master callback */
498 if ( (GNUNET_OK != ret) || (GNUNET_OK != result) )
499 return 1;
500 return 0;
501}
502\end{lstlisting}
503
504All of testbed API peer management functions treat management actions as
505operations and return operation handles. It is expected that the operations
506begin immediately, but they may get delayed (to balance out load on the system).
507The program using the API then has to take care of marking the operation as
508``done'' so that its associated resources can be freed immediately and other
509waiting operations can be executed. Operations will be canceled if they are
510marked as ``done'' before their completion.
511
512An operation is treated as completed when it succeeds or fails. Completion of
513an operation is either conveyed as events through \textit{controller event
514 callback} or through respective operation completion callbacks. In functions
515which support completion notification through both controller event callback and
516operation completion callback, first the controller event callback will be
517called. If the operation is not marked as done in that callback or if the
518callback is given as NULL when creating the operation, the operation completion
519callback will be called. The API documentation shows which event are to be
520expected in the controller event notifications. It also documents any
521exceptional behaviours.
522
523Once the peers are started, test cases often need to connect some of the peers'
524services. Normally, opening a connect to a peer's service requires the peer's
525configuration. While using testbed, the testbed automatically generates
526per-peer configuration. Accessing those configurations directly through file
527system is discouraged as their locations are dynamically created and will be
528different among various runs of testbed. To make access to these configurations
529easy, testbed API provides the function
530\texttt{GNUNET\_TESTBED\_service\_connect()}. This function fetches the
531configuration of a given peer and calls the \textit{Connect Adapter}.
532In the example code, it is the \texttt{dht\_ca}. A connect adapter is expected
533to open the connection to the needed service by using the provided configuration
534and return the created service connection handle. Successful connection to the
535needed service is signaled through \texttt{service\_connect\_comp\_cb}.
536
537A dual to connect adapter is the \textit{Disconnect Adapter}. This callback is
538called after the connect adapter has been called when the operation from
539\texttt{GNUNET\_TESTBED\_service\_connect()} is marked as ``done''. It has to
540disconnect from the service with the provided service handle (\texttt{op\_result}).
541
542\exercise{Find out how many peers you can run on your system.}
543
544\exercise{Find out how to create connections from within {\tt run} and create a
545 2D torus topology. Then use the DHT API to store and retrieve values in the
546 network.}
547
548\section{Developing Applications}
549\subsection{gnunet-ext}
550To develop a new peer-to-peer application or to extend GNUnet we provide
551a template build system for writing GNUnet extensions in C. It can be
552obtained as follows:
553
554\lstset{language=bash}
555\begin{lstlisting}
556$ svn checkout https://gnunet.org/svn/gnunet-ext/
557$ cd gnunet-ext/
558$ ./bootstrap
559$ ./configure --prefix=$HOME --with-gnunet=$GNUNET_PREFIX
560$ make
561$ make install
562$ make check
563\end{lstlisting}
564% $
565
566The GNUnet ext template includes examples and a working buildsystem for a new GNUnet service.
567A common GNUnet service consists of the following parts which will be discussed in detail in the
568remainder of this document. The functionality of a GNUnet service is implemented in:
569
570\begin{itemize}
571\itemsep0em
572 \item the GNUnet service (\lstinline|gnunet-ext/src/ext/gnunet-service-ext.c|)
573 \item the client API (\lstinline|gnunet-ext/src/ext/ext_api.c|)
574 \item the client application using the service API (\lstinline|gnunet-ext/src/ext/gnunet-ext.c|)
575
576
577\end{itemize}
578
579The interfaces for these entities are defined in:
580\begin{itemize}
581\itemsep0em
582 \item client API interface (\lstinline|gnunet-ext/src/ext/ext.h|)
583 \item the service interface (\lstinline|gnunet-ext/src/include/gnunet_service_SERVICE.h|)
584 \item the P2P protocol (\lstinline|gnunet-ext/src/include/gnunet_protocols_ext.h|)
585\end{itemize}
586
587
588In addition the \texttt{ext} systems provides:
589\begin{itemize}
590\itemsep0em
591 \item a test testing the API (\lstinline|gnunet-ext/src/ext/test_ext_api.c|)
592 \item a configuration template for the service (\lstinline|gnunet-ext/src/ext/ext.conf.in|)
593\end{itemize}
594
595
596\subsection{Adapting the Template}
597
598The first step for writing any extension with a new service is to
599ensure that the {\tt ext.conf.in} file contains entries for the
600\texttt{UNIXPATH}, \texttt{PORT} and \texttt{BINARY} for the service in a section named after
601the service.
602
603If you want to adapt the template rename the {\tt ext.conf.in} to match your
604services name, you have to modify the \texttt{AC\_OUTPUT} section in {\tt configure.ac}
605in the \texttt{gnunet-ext} root.
606
607\section{Writing a Client Application}
608
609When writing any client application (for example, a command-line
610tool), the basic structure is to start with the {\tt
611 GNUNET\_PROGRAM\_run} function. This function will parse
612command-line options, setup the scheduler and then invoke the {\tt
613 run} function (with the remaining non-option arguments) and a handle
614to the parsed configuration (and the configuration file name that was
615used, which is typically not needed):
616
617\lstset{language=c}
618\begin{lstlisting}
619#include <gnunet/platform.h>
620#include <gnunet/gnunet_util_lib.h>
621
622static int ret;
623
624static void
625run (void *cls,
626 char *const *args,
627 const char *cfgfile,
628 const struct GNUNET_CONFIGURATION_Handle *cfg)
629{
630 /* main code here */
631 ret = 0;
632}
633
634int
635main (int argc, char *const *argv)
636{
637 static const struct GNUNET_GETOPT_CommandLineOption options[] = {
638 GNUNET_GETOPT_OPTION_END
639 };
640 return (GNUNET_OK ==
641 GNUNET_PROGRAM_run (argc,
642 argv,
643 "binary-name",
644 gettext_noop ("binary description text"),
645 options, &run, NULL)) ? ret : 1;
646}
647\end{lstlisting}
648
649\subsection{Handling command-line options}
650
651Options can then be added easily by adding global variables and
652expanding the {\tt options} array. For example, the following would
653add a string-option and a binary flag (defaulting to {\tt NULL} and
654{\tt GNUNET\_NO} respectively):
655
656\begin{lstlisting}
657static char *string_option;
658static int a_flag;
659
660// ...
661 static const struct GNUNET_GETOPT_CommandLineOption options[] = {
662 {'s', "name", "SOMESTRING",
663 gettext_noop ("text describing the string_option NAME"), 1,
664 &GNUNET_GETOPT_set_string, &string_option},
665 {'f', "flag", NULL,
666 gettext_noop ("text describing the flag option"), 0,
667 &GNUNET_GETOPT_set_one, &a_flag},
668 GNUNET_GETOPT_OPTION_END
669 };
670// ...
671\end{lstlisting}
672
673Issues such as displaying some helpful text describing options using
674the {\tt --help} argument and error handling are taken care of when
675using this approach. Other {\tt GNUNET\_GETOPT\_}-functions can be used
676to obtain integer value options, increment counters, etc. You can
677even write custom option parsers for special circumstances not covered
678by the available handlers.
679
680Inside the {\tt run} method, the program would perform the
681application-specific logic, which typically involves initializing and
682using some client library to interact with the service. The client
683library is supposed to implement the IPC whereas the service provides
684more persistent P2P functions.
685
686\exercise{Add a few command-line options and print them inside
687of {\tt run}. What happens if the user gives invalid arguments?}
688
689\subsection{Writing a Client Library}
690
691The first and most important step in writing a client library is to
692decide on an API for the library. Typical API calls include
693connecting to the service, performing application-specific requests
694and cleaning up. Many examples for such service APIs can be found
695in the {\tt gnunet/src/include/gnunet\_*\_service.h} files.
696
697Then, a client-service protocol needs to be designed. This typically
698involves defining various message formats in a header that will be
699included by both the service and the client library (but is otherwise
700not shared and hence located within the service's directory and not
701installed by {\tt make install}). Each message must start with a {\tt
702 struct GNUNET\_MessageHeader} and must be shorter than 64k. By
703convention, all fields in IPC (and P2P) messages must be in big-endian
704format (and thus should be read using {\tt ntohl} and similar
705functions and written using {\tt htonl} and similar functions).
706Unique message types must be defined for each message struct in the
707{\tt gnunet\_protocols.h} header (or an extension-specific include
708file).
709
710\subsubsection{Connecting to the Service}
711
712Before a client library can implement the application-specific protocol
713with the service, a connection must be created:
714
715\lstset{language=c}
716\begin{lstlisting}
717 struct GNUNET_CLIENT_Connection *client;
718 client = GNUNET_CLIENT_connect ("service-name", cfg);
719\end{lstlisting}
720
721As a result a {\tt GNUNET\_CLIENT\_Connection} handle is returned
722which has to used in later API calls related to this service.
723The complete client API can be found in {\tt gnunet\_client\_lib.h}
724
725\subsubsection{GNUnet Messages}
726
727In GNUnet, messages are always sent beginning with a {\tt struct GNUNET\_MessageHeader}
728in big endian format. This header defines the size and the type of the
729message, the payload follows after this header.
730
731\lstset{language=c}
732\begin{lstlisting}
733struct GNUNET_MessageHeader
734{
735
736 /**
737 * The length of the struct (in bytes, including the length field itself),
738 * in big-endian format.
739 */
740 uint16_t size GNUNET_PACKED;
741
742 /**
743 * The type of the message (GNUNET_MESSAGE_TYPE_XXXX), in big-endian format.
744 */
745 uint16_t type GNUNET_PACKED;
746
747};
748\end{lstlisting}
749
750Existing message types are defined in {\tt gnunet\_protocols.h}\\
751A common way to create a message is:
752
753\lstset{language=c}
754\begin{lstlisting}
755struct GNUNET_MessageHeader *msg =
756 GNUNET_malloc(payload_size + sizeof(struct GNUNET_MessageHeader));
757msg->size = htons(payload_size + sizeof(struct GNUNET_MessageHeader));
758msg->type = htons(GNUNET_MY_MESSAGE_TYPE);
759memcpy(&msg[1], &payload, payload_size);
760// use 'msg'
761\end{lstlisting}
762
763\exercise{Define a message struct that includes a 32-bit
764unsigned integer in addition to the standard GNUnet MessageHeader.
765Add a C struct and define a fresh protocol number for your message.}
766
767
768\subsubsection{Sending Requests to the Service}
769
770Any client-service protocol must start with the client sending the
771first message to the service, since services are only notified about
772(new) clients upon receiving a the first message.
773
774Clients can transmit messages to the service using the
775{\tt GNUNET\_CLIENT\_notify\_transmit\_ready} API:
776\lstset{language=c}
777\begin{lstlisting}
778static size_t
779transmit_cb (void *cls, size_t size, void *buf)
780{
781 // ...
782 if (NULL == buf) { handle_error(); return 0; }
783 GNUNET_assert (size >= msg_size);
784 memcpy (buf, my_msg, msg_size);
785 // ...
786 return msg_size;
787}
788
789// ...
790th = GNUNET_CLIENT_notify_transmit_ready (client,
791 msg_size,
792 timeout,
793 GNUNET_YES,
794 &transmit_cb, cls);
795// ...
796\end{lstlisting}
797
798The client-service protocoll calls {\tt GNUNET\_CLIENT\_notify\_transmit\_ready}
799to be notified when the client is ready to send data to the service.
800Besides other arguments, you have to pass the client returned
801from the {\tt connect} call, the message size and the callback function to
802call when the client is ready to send.
803
804Only a single transmission request can be queued per client at the
805same time using this API. The handle {\tt th} can be used to cancel
806the request if necessary (for example, during shutdown).
807
808When {\tt transmit\_cb} is called the message is copied in the buffer provided and
809the number of bytes copied into the buffer is returned. {\tt transmit\_cb}
810could also return 0 if for some reason no message
811could be constructed; this is not an error and the connection to the
812service will persist in this case.
813
814\exercise{Define a helper function to transmit a 32-bit
815unsigned integer (as payload) to a service using some given client
816handle.}
817
818
819\subsubsection{Receiving Replies from the Service}
820
821Clients can receive messages from the service using the
822{\tt GNUNET\_CLIENT\_receive} API:
823
824\lstset{language=c}
825\begin{lstlisting}
826/**
827 * Function called with messages from stats service.
828 *
829 * @param cls closure
830 * @param msg message received, NULL on timeout or fatal error
831 */
832static void
833receive_message (void *cls, const struct GNUNET_MessageHeader *msg)
834{
835 struct MyArg *arg = cls;
836
837 // process 'msg'
838}
839
840// ...
841 GNUNET_CLIENT_receive (client,
842 &receive_message,
843 arg,
844 timeout);
845// ...
846\end{lstlisting}
847
848It should be noted that this receive call only receives a single
849message. To receive additional messages, {\tt
850 GNUNET\_CLIENT\_receive} must be called again.
851
852\exercise{Expand your helper function to receive a
853response message (for example, containing just the GNUnet MessageHeader
854without any payload). Upon receiving the service's response, you should
855call a callback provided to your helper function's API. You'll need to
856define a new 'struct' to hold your local context (``closure'').}
857
858
859\subsection{Writing a user interface}
860
861Given a client library, all it takes to access a service now is to
862combine calls to the client library with parsing command-line
863options.
864
865\exercise{Call your client API from your {\tt run} method
866in your client application to send a request to the service.
867For example, send a 32-bit integer value based on a number given
868at the command-line to the service.}
869
870
871
872\section{Writing a Service}
873
874Before you can test the client you've written so far, you'll need to also
875implement the corresponding service.
876
877
878\subsection{Code Placement}
879
880New services are placed in their own subdirectory under {\tt gnunet/src}.
881This subdirectory should contain the API implementation file {\tt SERVICE\_api.c},
882the description of the client-service protocol {\tt SERVICE.h} and P2P protocol
883{\tt SERVICE\_protocol.h}, the implementation of the service itself
884{\tt gnunet-service-SERVICE.h} and several files for tests, including test code
885and configuration files.
886
887\subsection{Starting a Service}
888
889The key API definitions for starting services are:
890\lstset{language=C}
891\begin{lstlisting}
892typedef void (*GNUNET_SERVICE_Main) (void *cls,
893 struct GNUNET_SERVER_Handle *server,
894 const struct GNUNET_CONFIGURATION_Handle *cfg);
895int GNUNET_SERVICE_run (int argc,
896 char *const *argv,
897 const char *serviceName,
898 enum GNUNET_SERVICE_Options opt,
899 GNUNET_SERVICE_Main task,
900 void *task_cls);
901\end{lstlisting}
902
903Here is a starting point for your main function for your service:
904
905\lstset{language=c}
906\begin{lstlisting}
907static void my_main (void *cls,
908 struct GNUNET_SERVER_Handle *server,
909 const struct GNUNET_CONFIGURATION_Handle *cfg)
910{
911 /* do work */
912}
913
914int main (int argc, char *const*argv)
915{
916 if (GNUNET_OK !=
917 GNUNET_SERVICE_run (argc, argv, "my",
918 GNUNET_SERVICE_OPTION_NONE,
919 &my_main, NULL);
920 return 1;
921 return 0;
922}
923\end{lstlisting}
924
925\exercise{Write a stub service that processes no messages at all
926in your code. Create a default configuration for it, integrate it
927with the build system and start the service from {\tt
928 gnunet-service-arm} using {\tt gnunet-arm -i NAME}.}
929
930
931\subsection{Receiving Requests from Clients}
932
933Inside of the {\tt my\_main} method, a service typically registers for
934the various message types from clients that it supports by providing
935a handler function, the message type itself and possibly a fixed
936message size (or 0 for variable-size messages):
937
938\lstset{language=c}
939\begin{lstlisting}
940static void
941handle_set (void *cls,
942 struct GNUNET_SERVER_Client *client,
943 const struct GNUNET_MessageHeader *message)
944{
945 GNUNET_SERVER_receive_done (client, GNUNET_OK);
946}
947static void
948handle_get (void *cls,
949 struct GNUNET_SERVER_Client *client,
950 const struct GNUNET_MessageHeader *message)
951{
952 GNUNET_SERVER_receive_done (client, GNUNET_OK);
953}
954
955static void my_main (void *cls,
956 struct GNUNET_SERVER_Handle *server,
957 const struct GNUNET_CONFIGURATION_Handle *cfg)
958{
959 static const struct GNUNET_SERVER_MessageHandler handlers[] = {
960 {&handle_set, NULL, GNUNET_MESSAGE_TYPE_MYNAME_SET, 0},
961 {&handle_get, NULL, GNUNET_MESSAGE_TYPE_MYNAME_GET, 0},
962 {NULL, NULL, 0, 0}
963 };
964 GNUNET_SERVER_add_handlers (server, handlers);
965 /* do more setup work */
966}
967\end{lstlisting}
968
969Each handler function {\bf must} eventually (possibly in some
970asynchronous continuation) call {\tt GNUNET\_SERVER\_receive\_done}.
971Only after this call additional messages from the same client may
972be processed. This way, the service can throttle processing messages
973from the same client. By passing {\tt GNUNET\_SYSERR}, the service
974can close the connection to the client, indicating an error.
975
976Services must check that client requests are well-formed and must not
977crash on protocol violations by the clients. Similarly, client
978libraries must check replies from servers and should gracefully report
979errors via their API.
980
981
982\exercise{Change the service to ``handle'' the message from your
983client (for now, by printing a message). What happens if you
984forget to call {\tt GNUNET\_SERVER\_receive\_done}?}
985
986
987\subsection{Responding to Clients}
988
989Servers can send messages to clients using the
990{\tt GNUNET\_SERVER\_notify\_transmit\_ready} API:
991
992\lstset{language=c}
993\begin{lstlisting}
994static size_t
995transmit_cb (void *cls, size_t size, void *buf)
996{
997 // ...
998 if (NULL == buf) { handle_error(); return 0; }
999 GNUNET_assert (size >= msg_size);
1000 memcpy (buf, my_msg, msg_size);
1001 // ...
1002 return msg_size;
1003}
1004
1005// ...
1006struct GNUNET_SERVER_TransmitHandle *th;
1007th = GNUNET_SERVER_notify_transmit_ready (client,
1008 msg_size,
1009 timeout,
1010 &transmit_cb, cls);
1011// ...
1012\end{lstlisting}
1013
1014Only a single transmission request can be queued per client
1015at the same time using this API.
1016Additional APIs for sending messages to clients can be found
1017in the {\tt gnunet\_server\_lib.h} header.
1018
1019
1020\exercise{Change the service respond to the request from your
1021client. Make sure you handle malformed messages in both directions.}
1022
1023
1024\section{Interacting directly with other Peers using the CORE Service}
1025
1026One of the most important services in GNUnet is the \texttt{CORE} service
1027managing connections between peers and handling encryption between peers.
1028
1029One of the first things any service that extends the P2P protocol typically does
1030is connect to the \texttt{CORE} service using:
1031
1032\lstset{language=C}
1033\begin{lstlisting}
1034#include <gnunet/gnunet_core_service.h>
1035
1036struct GNUNET_CORE_Handle *
1037GNUNET_CORE_connect (const struct GNUNET_CONFIGURATION_Handle *cfg,
1038 void *cls,
1039 GNUNET_CORE_StartupCallback init,
1040 GNUNET_CORE_ConnectEventHandler connects,
1041 GNUNET_CORE_DisconnectEventHandler disconnects,
1042 GNUNET_CORE_MessageCallback inbound_notify,
1043 int inbound_hdr_only,
1044 GNUNET_CORE_MessageCallback outbound_notify,
1045 int outbound_hdr_only,
1046 const struct GNUNET_CORE_MessageHandler *handlers);
1047\end{lstlisting}
1048
1049\subsection{New P2P connections}
1050
1051Before any traffic with a different peer can be exchanged, the peer must be
1052known to the service. This is notified by the \texttt{CORE} {\tt connects} callback,
1053which communicates the identity of the new peer to the service:
1054
1055\lstset{language=C}
1056\begin{lstlisting}
1057void
1058connects (void *cls,
1059 const struct GNUNET_PeerIdentity * peer)
1060{
1061 /* Save identity for later use */
1062 /* Optional: start sending messages to peer */
1063}
1064\end{lstlisting}
1065
1066\exercise{Create a service that connects to the \texttt{CORE}. Then
1067start (and connect) two peers and print a message once your connect
1068callback is invoked.}
1069
1070\subsection{Receiving P2P Messages}
1071
1072To receive messages from \texttt{CORE}, services register a set of handlers
1073(parameter {\tt *handlers} in the \lstinline|GNUNET_CORE_connect| call that are called by \texttt{CORE}
1074when a suitable message arrives.
1075
1076\lstset{language=c}
1077\begin{lstlisting}
1078static int
1079callback_function_for_type_one(void *cls,
1080 const struct GNUNET_PeerIdentity *peer,
1081 const struct GNUNET_MessageHeader *message)
1082{
1083 /* Do stuff */
1084 return GNUNET_OK; /* or GNUNET_SYSERR to close the connection */
1085}
1086
1087/**
1088 * Functions to handle messages from core
1089 */
1090static struct GNUNET_CORE_MessageHandler core_handlers[] = {
1091 {&callback_function_for_type_one, GNUNET_MESSAGE_TYPE_MYSERVICE_TYPE_ONE, 0},
1092 /* more handlers*/
1093 {NULL, 0, 0}
1094};
1095\end{lstlisting}
1096
1097\exercise{Start one peer with a new service that has a message
1098handler and start a second peer that only has your ``old'' service
1099without message handlers. Which ``connect'' handlers are invoked when
1100the two peers are connected? Why?}
1101
1102
1103\subsection{Sending P2P Messages}
1104
1105In response to events (connect, disconnect, inbound messages,
1106timing, etc.) services can then use this API to transmit messages:
1107
1108\lstset{language=C}
1109\begin{lstlisting}
1110typedef size_t
1111(*GNUNET_CONNECTION_TransmitReadyNotify) (void *cls,
1112 size_t size,
1113 void *buf)
1114{
1115 /* Fill "*buf" with up to "size" bytes, must start with GNUNET_MessageHeader */
1116 return n; /* Total size of the message put in "*buf" */
1117}
1118
1119struct GNUNET_CORE_TransmitHandle *
1120GNUNET_CORE_notify_transmit_ready (struct GNUNET_CORE_Handle *handle,
1121 int cork, uint32_t priority,
1122 struct GNUNET_TIME_Relative maxdelay,
1123 const struct GNUNET_PeerIdentity *target,
1124 size_t notify_size,
1125 GNUNET_CONNECTION_TransmitReadyNotify notify,
1126 void *notify_cls);
1127\end{lstlisting}
1128
1129\exercise{Write a service that upon connect sends messages as
1130fast as possible to the other peer (the other peer should run a
1131service that ``processes'' those messages). How fast is the
1132transmission? Count using the STATISTICS service on both ends. Are
1133messages lost? How can you transmit messages faster? What happens if
1134you stop the peer that is receiving your messages?}
1135
1136
1137\subsection{End of P2P connections}
1138
1139If a message handler returns {\tt GNUNET\_SYSERR}, the remote peer shuts down or
1140there is an unrecoverable network disconnection, CORE notifies the service that
1141the peer disconnected. After this notification no more messages will be received
1142from the peer and the service is no longer allowed to send messages to the peer.
1143The disconnect callback looks like the following:
1144
1145\lstset{language=C}
1146\begin{lstlisting}
1147void
1148disconnects (void *cls,
1149 const struct GNUNET_PeerIdentity * peer)
1150{
1151 /* Remove peer's identity from known peers */
1152 /* Make sure no messages are sent to peer from now on */
1153}
1154\end{lstlisting}
1155
1156\exercise{Fix your service to handle peer disconnects.}
1157
1158\section{Using the DHT}
1159The DHT allows to store data so other peers in the P2P network can
1160access it and retrieve data stored by any peers in the network.
1161This section will explain how to use the DHT. Of course, the first
1162thing to do is to connect to the DHT service:
1163\lstset{language=C}
1164\begin{lstlisting}
1165dht_handle = GNUNET_DHT_connect (cfg, parallel_requests);
1166\end{lstlisting}
1167The second parameter indicates how many requests in parallel to expect.
1168It is not a hard limit, but a good approximation will make the DHT more
1169efficiently.
1170
1171\subsection{Storing data in the DHT}
1172Since the DHT is a dynamic environment (peers join a leave frequently)
1173the data that we put in the DHT does not stay there indefinitely. It is
1174important to ``refresh'' the data periodically by simply storing it again,
1175in order to make sure other peers can access it.
1176
1177The put API call offers a callback to signal that the PUT request has been
1178sent. This does not guarantee that the data is accessible to others peers,
1179or even that is has been stored, only that the service has requested to
1180a neighboring peer the retransmission of the PUT request towards its final
1181destination. Currently there is no feedback about whether or not the data
1182has been sucessfully stored or where it has been stored. In order to improve
1183the availablilty of the data and to compensate for possible errors, peers leaving
1184and other unfavorable events, just make several PUT requests!
1185
1186\lstset{language=C}
1187\begin{lstlisting}
1188void
1189message_sent_cont (void *cls, const struct GNUNET_SCHEDULER_TaskContext *tc)
1190{
1191 /* Request has left local node */
1192}
1193
1194struct GNUNET_DHT_PutHandle *
1195GNUNET_DHT_put (struct GNUNET_DHT_Handle *handle,
1196 const struct GNUNET_HashCode * key,
1197 uint32_t desired_replication_level,
1198 enum GNUNET_DHT_RouteOption options, /* Route options, see next call */
1199 enum GNUNET_BLOCK_Type type, size_t size, const void *data,
1200 struct GNUNET_TIME_Absolute exp, /* When does the data expire? */
1201 struct GNUNET_TIME_Relative timeout, /* How long to try to send the request */
1202 GNUNET_DHT_PutContinuation cont,
1203 void *cont_cls)
1204\end{lstlisting}
1205
1206\exercise{Store a value in the DHT periodically to make sure it is available
1207over time. You might consider using the function GNUNET\_SCHEDULER\_add\_delayed and
1208call GNUNET\_DHT\_put from inside a helper function.}
1209
1210
1211\subsection{Obtaining data from the DHT}
1212As we saw in the previous example, the DHT works in an asynchronous mode.
1213Each request to the DHT is executed ``in the background'' and the API
1214calls return immediately. In order to receive results from the DHT, the
1215API provides a callback. Once started, the request runs in the service,
1216the service will try to get as many results as possible (filtering out
1217duplicates) until the timeout expires or we explicitly stop the request.
1218It is possible to give a ``forever'' timeout with
1219{\tt GNUNET\_TIME\_UNIT\_FOREVER\_REL}.
1220
1221If we give a route option {\tt GNUNET\_DHT\_RO\_RECORD\_ROUTE} the callback
1222will get a list of all the peers the data has travelled, both on the PUT
1223path and on the GET path.
1224\lstset{language=C}
1225\begin{lstlisting}
1226static void
1227get_result_iterator (void *cls, struct GNUNET_TIME_Absolute expiration,
1228 const struct GNUNET_HashCode * key,
1229 const struct GNUNET_PeerIdentity *get_path,
1230 unsigned int get_path_length,
1231 const struct GNUNET_PeerIdentity *put_path,
1232 unsigned int put_path_length,
1233 enum GNUNET_BLOCK_Type type, size_t size, const void *data)
1234{
1235 /* Do stuff with the data and/or route */
1236 /* Optionally: */
1237 GNUNET_DHT_get_stop (get_handle);
1238}
1239
1240get_handle =
1241 GNUNET_DHT_get_start (dht_handle,
1242 block_type,
1243 &key,
1244 replication,
1245 GNUNET_DHT_RO_NONE, /* Route options */
1246 NULL, /* xquery: not used here */
1247 0, /* xquery size */
1248 &get_result_iterator,
1249 cls)
1250\end{lstlisting}
1251
1252\exercise{Store a value in the DHT and after a while retrieve it. Show the IDs of all
1253the peers the requests have gone through. In order to convert a peer ID to a string, use
1254the function GNUNET\_i2s. Pay attention to the route option parameters in both calls!}
1255
1256\subsection{Implementing a block plugin}
1257
1258In order to store data in the DHT, it is necessary to provide a block
1259plugin. The DHT uses the block plugin to ensure that only well-formed
1260requests and replies are transmitted over the network.
1261
1262The block plugin should be put in a file {\tt
1263 plugin\_block\_SERVICE.c} in the service's respective directory. The
1264mandatory functions that need to be implemented for a block plugin are
1265described in the following sections.
1266
1267\subsubsection{Validating requests and replies}
1268
1269The evaluate function should validate a reply or a request. It returns
1270a {\tt GNUNET\_BLOCK\_EvaluationResult}, which is an enumeration. All
1271possible answers are in {\tt gnunet\_block\_lib.h}. The function will
1272be called with a {\tt reply\_block} argument of {\tt NULL} for
1273requests. Note that depending on how {\tt evaluate} is called, only
1274some of the possible return values are valid. The specific meaning of
1275the {\tt xquery} argument is application-specific. Applications that
1276do not use an extended query should check that the {\tt xquery\_size}
1277is zero. The Bloom filter is typically used to filter duplicate
1278replies.
1279
1280\lstset{language=C}
1281\begin{lstlisting}
1282static enum GNUNET_BLOCK_EvaluationResult
1283block_plugin_SERVICE_evaluate (void *cls,
1284 enum GNUNET_BLOCK_Type type,
1285 const GNUNET_HashCode * query,
1286 struct GNUNET_CONTAINER_BloomFilter **bf,
1287 int32_t bf_mutator,
1288 const void *xquery,
1289 size_t xquery_size,
1290 const void *reply_block,
1291 size_t reply_block_size)
1292{
1293 /* Verify type, block and bloomfilter */
1294}
1295\end{lstlisting}
1296
1297Note that it is mandatory to detect duplicate replies in this
1298function and return the respective status code. Duplicate
1299detection should be done by setting the respective bits in
1300the Bloom filter {\tt bf}. Failure to do so may cause replies
1301to circle in the network.
1302
1303\subsubsection{Deriving a key from a reply}
1304
1305The DHT can operate more efficiently if it is possible to derive a key
1306from the value of the corresponding block. The {\tt get\_key}
1307function is used to obtain the key of a block --- for example, by
1308means of hashing. If deriving the key is not possible, the function
1309should simply return {\tt GNUNET\_SYSERR} (the DHT will still work
1310just fine with such blocks).
1311
1312\lstset{language=C}
1313\begin{lstlisting}
1314static int
1315block_plugin_SERVICE_get_key (void *cls, enum GNUNET_BLOCK_Type type,
1316 const void *block, size_t block_size,
1317 GNUNET_HashCode * key)
1318{
1319 /* Store the key in the key argument, return GNUNET_OK on success. */
1320}
1321\end{lstlisting}
1322
1323\subsubsection{Initialization of the plugin}
1324
1325The plugin is realized as a shared C library. The library must export
1326an initialization function which should initialize the plugin. The
1327initialization function specifies what block types the plugin cares
1328about and returns a struct with the functions that are to be used for
1329validation and obtaining keys (the ones just defined above).
1330
1331\lstset{language=C}
1332\begin{lstlisting}
1333void *
1334libgnunet_plugin_block_SERVICE_init (void *cls)
1335{
1336 static enum GNUNET_BLOCK_Type types[] =
1337 {
1338 GNUNET_BLOCK_TYPE_SERVICE_BLOCKYPE, /* list of blocks we care about, from gnunet_block_lib.h */
1339 GNUNET_BLOCK_TYPE_ANY /* end of list */
1340 };
1341 struct GNUNET_BLOCK_PluginFunctions *api;
1342
1343 api = GNUNET_malloc (sizeof (struct GNUNET_BLOCK_PluginFunctions));
1344 api->evaluate = &block_plugin_SERICE_evaluate;
1345 api->get_key = &block_plugin_SERVICE_get_key;
1346 api->types = types;
1347 return api;
1348}
1349\end{lstlisting}
1350
1351\subsubsection{Shutdown of the plugin}
1352
1353Following GNUnet's general plugin API concept, the plugin must
1354export a second function for cleaning up. It usually does very
1355little.
1356
1357\lstset{language=C}
1358\begin{lstlisting}
1359void *
1360libgnunet_plugin_block_SERVICE_done (void *cls)
1361{
1362 struct GNUNET_TRANSPORT_PluginFunctions *api = cls;
1363
1364 GNUNET_free (api);
1365 return NULL;
1366}
1367\end{lstlisting}
1368
1369
1370\subsubsection{Integration of the plugin with the build system}
1371
1372In order to compile the plugin, the {\tt Makefile.am} file for the
1373service should contain a rule similar to this:
1374
1375\lstset{language=make}
1376\begin{lstlisting}
1377plugin_LTLIBRARIES = \
1378 libgnunet_plugin_block_SERVICE.la
1379libgnunet_plugin_block_SERVICE_la_SOURCES = \
1380 plugin_block_SERVICE.c
1381libgnunet_plugin_block_SERVICE_la_LIBADD = \
1382 $(top_builddir)/src/hello/libgnunethello.la \
1383 $(top_builddir)/src/block/libgnunetblock.la \
1384 $(top_builddir)/src/util/libgnunetutil.la
1385libgnunet_plugin_block_SERVICE_la_LDFLAGS = \
1386 $(GN_PLUGIN_LDFLAGS)
1387libgnunet_plugin_block_SERVICE_la_DEPENDENCIES = \
1388 $(top_builddir)/src/block/libgnunetblock.la
1389\end{lstlisting}
1390% $
1391
1392
1393\exercise{Write a block plugin that accepts all queries
1394and all replies but prints information about queries and replies
1395when the respective validation hooks are called.}
1396
1397
1398
1399\subsection{Monitoring the DHT}
1400It is possible to monitor the functioning of the local DHT service. When monitoring
1401the DHT, the service will alert the monitoring program of any events,
1402both started locally or received for routing from another peer. The are three different
1403types of events possible: a GET request, a PUT request or a response (a reply to
1404a GET).
1405
1406Since the different events have different associated data, the API gets 3
1407different callbacks (one for each message type) and optional type and key parameters,
1408to allow for filtering of messages. When an event happens, the appropiate callback
1409is called with all the information about the event.
1410\lstset{language=C}
1411\begin{lstlisting}
1412void
1413get_callback (void *cls,
1414 enum GNUNET_DHT_RouteOption options,
1415 enum GNUNET_BLOCK_Type type,
1416 uint32_t hop_count,
1417 uint32_t desired_replication_level,
1418 unsigned int path_length,
1419 const struct GNUNET_PeerIdentity *path,
1420 const struct GNUNET_HashCode * key)
1421{
1422}
1423
1424void
1425get_resp_callback (void *cls,
1426 enum GNUNET_BLOCK_Type type,
1427 const struct GNUNET_PeerIdentity *get_path,
1428 unsigned int get_path_length,
1429 const struct GNUNET_PeerIdentity *put_path,
1430 unsigned int put_path_length,
1431 struct GNUNET_TIME_Absolute exp,
1432 const struct GNUNET_HashCode * key,
1433 const void *data,
1434 size_t size)
1435{
1436}
1437
1438void
1439put_callback (void *cls,
1440 enum GNUNET_DHT_RouteOption options,
1441 enum GNUNET_BLOCK_Type type,
1442 uint32_t hop_count,
1443 uint32_t desired_replication_level,
1444 unsigned int path_length,
1445 const struct GNUNET_PeerIdentity *path,
1446 struct GNUNET_TIME_Absolute exp,
1447 const struct GNUNET_HashCode * key,
1448 const void *data,
1449 size_t size)
1450{
1451}
1452
1453monitor_handle = GNUNET_DHT_monitor_start (dht_handle,
1454 block_type, /* GNUNET_BLOCK_TYPE_ANY for all */
1455 key, /* NULL for all */
1456 &get_callback,
1457 &get_resp_callback,
1458 &put_callback,
1459 cls);
1460\end{lstlisting}
1461
1462
1463\section{Debugging with {\tt gnunet-arm}}
1464
1465Even if services are managed by {\tt gnunet-arm}, you can start them with
1466{\tt gdb} or {\tt valgrind}. For example, you could add the following lines
1467to your configuration file to start the DHT service in a {\tt gdb} session in a
1468fresh {\tt xterm}:
1469
1470\begin{verbatim}
1471[dht]
1472PREFIX=xterm -e gdb --args
1473\end{verbatim}
1474
1475Alternatively, you can stop a service that was started via ARM and run it manually:
1476
1477\lstset{language=bash}
1478\begin{lstlisting}
1479$ gnunet-arm -k dht
1480$ gdb --args gnunet-service-dht -L DEBUG
1481$ valgrind gnunet-service-dht -L DEBUG
1482\end{lstlisting}
1483% $
1484
1485Assuming other services are well-written, they will automatically re-integrate the
1486restarted service with the peer.
1487
1488GNUnet provides a powerful logging mechanism providing log levels \texttt{ERROR},
1489\texttt{WARNING}, \texttt{INFO} and \texttt{DEBUG}. The current log level is
1490configured using the \lstinline|$GNUNET_FORCE_LOG| environmental variable.
1491The \texttt{DEBUG} level is only available if \lstinline|--enable-logging=verbose| was used when
1492running \texttt{configure}. More details about logging can be found under
1493\url{https://gnunet.org/logging}.
1494
1495You should also probably enable the creation of core files, by setting
1496{\tt ulimit}, and echo'ing 1 into {\tt /proc/sys/kernel/core\_uses\_pid}.
1497Then you can investigate the core dumps with {\tt gdb}, which is often
1498the fastest method to find simple errors.
1499
1500\exercise{Add a memory leak to your service and obtain a trace
1501pointing to the leak using {\tt valgrind} while running the service
1502from {\tt gnunet-service-arm}.}
1503
1504
1505\end{document}