diff options
author | ng0 <ng0@infotropique.org> | 2017-10-26 12:28:28 +0000 |
---|---|---|
committer | ng0 <ng0@infotropique.org> | 2017-10-26 12:28:28 +0000 |
commit | e2f0870adf4baad5dfefaefd92c122ee9e1e0e5a (patch) | |
tree | 9374b281f7f450e9bd6c73ace85bb6d627fdd9d9 /doc/documentation/chapters | |
parent | 17e755c71521be9a84f0f3b46d1de6962298f733 (diff) | |
download | gnunet-e2f0870adf4baad5dfefaefd92c122ee9e1e0e5a.tar.gz gnunet-e2f0870adf4baad5dfefaefd92c122ee9e1e0e5a.zip |
documentation
Diffstat (limited to 'doc/documentation/chapters')
-rw-r--r-- | doc/documentation/chapters/developer.texi | 169 |
1 files changed, 95 insertions, 74 deletions
diff --git a/doc/documentation/chapters/developer.texi b/doc/documentation/chapters/developer.texi index 996474359..9459068a9 100644 --- a/doc/documentation/chapters/developer.texi +++ b/doc/documentation/chapters/developer.texi | |||
@@ -4,11 +4,14 @@ | |||
4 | 4 | ||
5 | This book is intended to be an introduction for programmers that want to | 5 | This book is intended to be an introduction for programmers that want to |
6 | extend the GNUnet framework. GNUnet is more than a simple peer-to-peer | 6 | extend the GNUnet framework. GNUnet is more than a simple peer-to-peer |
7 | application. For developers, GNUnet is: | 7 | application. |
8 | |||
9 | For developers, GNUnet is: | ||
8 | 10 | ||
9 | @itemize @bullet | 11 | @itemize @bullet |
10 | @item Free software under the GNU General Public License, with a community | 12 | @item developed by a community that believes in the GNU philosophy |
11 | that believes in the GNU philosophy | 13 | @item Free Software (Free as in Freedom), licensed under the |
14 | @xref{GNU General Public License} | ||
12 | @item A set of standards, including coding conventions and | 15 | @item A set of standards, including coding conventions and |
13 | architectural rules | 16 | architectural rules |
14 | @item A set of layered protocols, both specifying the communication | 17 | @item A set of layered protocols, both specifying the communication |
@@ -20,17 +23,19 @@ writing extensions | |||
20 | 23 | ||
21 | In particular, the architecture specifies that a peer consists of many | 24 | In particular, the architecture specifies that a peer consists of many |
22 | processes communicating via protocols. Processes can be written in almost | 25 | processes communicating via protocols. Processes can be written in almost |
23 | any language. C and Java APIs exist for accessing existing services and | 26 | any language. |
24 | for writing extensions. It is possible to write extensions in other | 27 | C and Java APIs exist for accessing existing services and for writing |
25 | languages by implementing the necessary IPC protocols. | 28 | extensions. It is possible to write extensions in other languages by |
29 | implementing the necessary IPC protocols. | ||
26 | 30 | ||
27 | GNUnet can be extended and improved along many possible dimensions, and | 31 | GNUnet can be extended and improved along many possible dimensions, and |
28 | anyone interested in Free Software and Freedom-enhancing Networking is | 32 | anyone interested in Free Software and Freedom-enhancing Networking is |
29 | welcome to join the effort. This Developer Handbook attempts to provide | 33 | welcome to join the effort. This Developer Handbook attempts to provide |
30 | an initial introduction to some of the key design choices and central | 34 | an initial introduction to some of the key design choices and central |
31 | components of the system. This part of the GNUNet documentation | 35 | components of the system. |
32 | is far from complete, and we welcome informed contributions, | 36 | This part of the GNUNet documentation is far from complete, |
33 | be it in the form of new chapters or insightful comments. | 37 | and we welcome informed contributions, be it in the form of |
38 | new chapters, sections or insightful comments. | ||
34 | 39 | ||
35 | @menu | 40 | @menu |
36 | * Developer Introduction:: | 41 | * Developer Introduction:: |
@@ -92,11 +97,13 @@ following links: | |||
92 | @item GNUnet Java tutorial | 97 | @item GNUnet Java tutorial |
93 | @end itemize | 98 | @end itemize |
94 | 99 | ||
95 | In addition to this book, the GNUnet server contains various resources for | 100 | In addition to the GNUnet Reference Documentation you are reading, |
96 | GNUnet developers. They are all conveniently reachable via the "Developer" | 101 | the GNUnet server contains various resources for GNUnet |
102 | developers and those who aspire to become regular contributors. | ||
103 | They are all conveniently reachable via the "Developer" | ||
97 | entry in the navigation menu. Some additional tools (such as static | 104 | entry in the navigation menu. Some additional tools (such as static |
98 | analysis reports) require a special developer access to perform certain | 105 | analysis reports) require a special developer access to perform certain |
99 | operations. If you feel you need access, you should contact | 106 | operations. If you want (or require) access, you should contact |
100 | @uref{http://grothoff.org/christian/, Christian Grothoff}, | 107 | @uref{http://grothoff.org/christian/, Christian Grothoff}, |
101 | GNUnet's maintainer. | 108 | GNUnet's maintainer. |
102 | 109 | ||
@@ -104,20 +111,26 @@ The public subsystems on the GNUnet server that help developers are: | |||
104 | 111 | ||
105 | @itemize @bullet | 112 | @itemize @bullet |
106 | 113 | ||
107 | @item The Version control system (git) keeps our code and enables | 114 | @item The version control system (git) keeps our code and enables |
108 | distributed development. | 115 | distributed development. |
116 | It is pubclicly accessible at @uref{https://gnunet.org/git/}. | ||
109 | Only developers with write access can commit code, everyone else is | 117 | Only developers with write access can commit code, everyone else is |
110 | encouraged to submit patches to the | 118 | encouraged to submit patches to the |
111 | @uref{https://lists.gnu.org/mailman/listinfo/gnunet-developers, GNUnet-developers mailinglist} | 119 | @uref{https://lists.gnu.org/mailman/listinfo/gnunet-developers, GNUnet-developers mailinglist}. |
112 | . | 120 | |
113 | 121 | @item The bugtracking system (Mantis). | |
114 | @item The GNUnet bugtracking system (Mantis) is used to track | 122 | We use it to track feature requests, open bug reports and their |
115 | feature requests, open bug reports and their resolutions. | 123 | resolutions. |
116 | Anyone can report bugs, only developers can claim to have fixed them. | 124 | It can be accessed at @uref{https://gnunet.org/bugs/}. |
117 | 125 | Anyone can report bugs, but only developers can claim to have fixed them. | |
118 | @item A site installation of the CI system "Buildbot" is used to check | 126 | |
119 | GNUnet builds automatically on a range of platforms. | 127 | @item Our site installation of the |
120 | Builds are triggered automatically after 30 minutes of no changes to Git. | 128 | CI@footnote{Continuous Integration} system "@code{Buildbot}" is used |
129 | to check GNUnet builds automatically on a range of platforms. | ||
130 | The web interface of this CI is exposed at | ||
131 | @uref{https://gnunet.org/buildbot/}. | ||
132 | Builds are triggered automatically 30 minutes after the last commit to | ||
133 | our repository was made. | ||
121 | 134 | ||
122 | @item The current quality of our automated test suite is assessed using | 135 | @item The current quality of our automated test suite is assessed using |
123 | Code coverage analysis. This analysis is run daily; however the webpage | 136 | Code coverage analysis. This analysis is run daily; however the webpage |
@@ -163,34 +176,41 @@ GNUnet sub-projects in order of likely relevance are currently: | |||
163 | 176 | ||
164 | @table @asis | 177 | @table @asis |
165 | 178 | ||
166 | @item gnunet | 179 | @item @command{gnunet} |
167 | Core of the P2P framework, including file-sharing, VPN and | 180 | Core of the P2P framework, including file-sharing, VPN and |
168 | chat applications; this is what the developer handbook covers mostly | 181 | chat applications; this is what the Developer Handbook covers mostly |
169 | @item gnunet-gtk Gtk+-based user interfaces, including gnunet-fs-gtk | 182 | @item @command{gnunet-gtk} |
170 | (file-sharing), gnunet-statistics-gtk (statistics over time), | 183 | Gtk+-based user interfaces, including: |
171 | gnunet-peerinfo-gtk (information about current connections and known | 184 | |
172 | peers), gnunet-chat-gtk (chat GUI) and gnunet-setup (setup tool for | 185 | @itemize @bullet |
173 | "everything") | 186 | @item @command{gnunet-fs-gtk} (file-sharing), |
174 | @item gnunet-fuse | 187 | @item @command{gnunet-statistics-gtk} (statistics over time), |
188 | @item @command{gnunet-peerinfo-gtk} | ||
189 | (information about current connections and known peers), | ||
190 | @item @command{gnunet-chat-gtk} (chat GUI) and | ||
191 | @item @command{gnunet-setup} (setup tool for "everything") | ||
192 | @end itemize | ||
193 | |||
194 | @item @command{gnunet-fuse} | ||
175 | Mounting directories shared via GNUnet's file-sharing | 195 | Mounting directories shared via GNUnet's file-sharing |
176 | on Linux | 196 | on GNU/Linux distributions |
177 | @item gnunet-update | 197 | @item @command{gnunet-update} |
178 | Installation and update tool | 198 | Installation and update tool |
179 | @item gnunet-ext | 199 | @item @command{gnunet-ext} |
180 | Template for starting 'external' GNUnet projects | 200 | Template for starting 'external' GNUnet projects |
181 | @item gnunet-java | 201 | @item @command{gnunet-java} |
182 | Java APIs for writing GNUnet services and applications | 202 | Java APIs for writing GNUnet services and applications |
183 | @c ** FIXME: Point to new website repository once we have it: | 203 | @c ** FIXME: Point to new website repository once we have it: |
184 | @c ** @item svn/gnunet-www/ Code and media helping drive the GNUnet | 204 | @c ** @item svn/gnunet-www/ Code and media helping drive the GNUnet |
185 | @c website | 205 | @c website |
186 | @item eclectic | 206 | @item @command{eclectic} |
187 | Code to run GNUnet nodes on testbeds for research, development, | 207 | Code to run GNUnet nodes on testbeds for research, development, |
188 | testing and evaluation | 208 | testing and evaluation |
189 | @c ** FIXME: Solve the status and location of gnunet-qt | 209 | @c ** FIXME: Solve the status and location of gnunet-qt |
190 | @item gnunet-qt | 210 | @item @command{gnunet-qt} |
191 | Qt-based GNUnet GUI (dead?) | 211 | Qt-based GNUnet GUI (is it depreacated?) |
192 | @item gnunet-cocoa | 212 | @item @command{gnunet-cocoa} |
193 | cocoa-based GNUnet GUI (dead?) | 213 | cocoa-based GNUnet GUI (is it depreacated?) |
194 | 214 | ||
195 | @end table | 215 | @end table |
196 | 216 | ||
@@ -198,19 +218,19 @@ We are also working on various supporting libraries and tools: | |||
198 | @c ** FIXME: What about gauger, and what about libmwmodem? | 218 | @c ** FIXME: What about gauger, and what about libmwmodem? |
199 | 219 | ||
200 | @table @asis | 220 | @table @asis |
201 | @item libextractor | 221 | @item @command{libextractor} |
202 | GNU libextractor (meta data extraction) | 222 | GNU libextractor (meta data extraction) |
203 | @item libmicrohttpd | 223 | @item @command{libmicrohttpd} |
204 | GNU libmicrohttpd (embedded HTTP(S) server library) | 224 | GNU libmicrohttpd (embedded HTTP(S) server library) |
205 | @item gauger | 225 | @item @command{gauger} |
206 | Tool for performance regression analysis | 226 | Tool for performance regression analysis |
207 | @item monkey | 227 | @item @command{monkey} |
208 | Tool for automated debugging of distributed systems | 228 | Tool for automated debugging of distributed systems |
209 | @item libmwmodem | 229 | @item @command{libmwmodem} |
210 | Library for accessing satellite connection quality | 230 | Library for accessing satellite connection quality |
211 | reports | 231 | reports |
212 | @item libgnurl | 232 | @item @command{libgnurl} |
213 | gnURL (feature restricted variant of cURL/libcurl) | 233 | gnURL (feature-restricted variant of cURL/libcurl) |
214 | @end table | 234 | @end table |
215 | 235 | ||
216 | Finally, there are various external projects (see links for a list of | 236 | Finally, there are various external projects (see links for a list of |
@@ -247,7 +267,7 @@ type defines a particular format and how that binary format is to be | |||
247 | linked to a hash code (the key for the DHT and for databases). The block | 267 | linked to a hash code (the key for the DHT and for databases). The block |
248 | library is a wapper around block plugins which provide the necessary | 268 | library is a wapper around block plugins which provide the necessary |
249 | functions for each block type. | 269 | functions for each block type. |
250 | @item @file{statistics/} | 270 | @item @file{statistics/} --- statistics service |
251 | The statistics service enables associating | 271 | The statistics service enables associating |
252 | values (of type uint64_t) with a componenet name and a string. The main | 272 | values (of type uint64_t) with a componenet name and a string. The main |
253 | uses is debugging (counting events), performance tracking and user | 273 | uses is debugging (counting events), performance tracking and user |
@@ -257,7 +277,7 @@ The automatic-restart-manager (ARM) service | |||
257 | is the GNUnet master service. Its role is to start gnunet-services, to | 277 | is the GNUnet master service. Its role is to start gnunet-services, to |
258 | re-start them when they crashed and finally to shut down the system when | 278 | re-start them when they crashed and finally to shut down the system when |
259 | requested. | 279 | requested. |
260 | @item @file{peerinfo/} | 280 | @item @file{peerinfo/} --- peerinfo service |
261 | The peerinfo service keeps track of which peers are known | 281 | The peerinfo service keeps track of which peers are known |
262 | to the local peer and also tracks the validated addresses for each peer | 282 | to the local peer and also tracks the validated addresses for each peer |
263 | (in the form of a HELLO message) for each of those peers. The peer is not | 283 | (in the form of a HELLO message) for each of those peers. The peer is not |
@@ -269,17 +289,17 @@ The datacache library provides (temporary) block storage for the DHT. | |||
269 | Existing plugins can store blocks in Sqlite, Postgres or MySQL databases. | 289 | Existing plugins can store blocks in Sqlite, Postgres or MySQL databases. |
270 | All data stored in the cache is lost when the peer is stopped or | 290 | All data stored in the cache is lost when the peer is stopped or |
271 | restarted (datacache uses temporary tables). | 291 | restarted (datacache uses temporary tables). |
272 | @item @file{datastore/} | 292 | @item @file{datastore/} --- datastore service |
273 | The datastore service stores file-sharing blocks in | 293 | The datastore service stores file-sharing blocks in |
274 | databases for extended periods of time. In contrast to the datacache, data | 294 | databases for extended periods of time. In contrast to the datacache, data |
275 | is not lost when peers restart. However, quota restrictions may still | 295 | is not lost when peers restart. However, quota restrictions may still |
276 | cause old, expired or low-priority data to be eventually discarded. | 296 | cause old, expired or low-priority data to be eventually discarded. |
277 | Existing plugins can store blocks in Sqlite, Postgres or MySQL databases. | 297 | Existing plugins can store blocks in Sqlite, Postgres or MySQL databases. |
278 | @item @file{template/} | 298 | @item @file{template/} --- service template |
279 | Template for writing a new service. Does nothing. | 299 | Template for writing a new service. Does nothing. |
280 | @item @file{ats/} --- Automatic Transport Selection | 300 | @item @file{ats/} --- Automatic Transport Selection |
281 | The automatic transport | 301 | The automatic transport selection (ATS) service |
282 | selection (ATS) service is responsible for deciding which address (i.e. | 302 | is responsible for deciding which address (i.e. |
283 | which transport plugin) should be used for communication with other peers, | 303 | which transport plugin) should be used for communication with other peers, |
284 | and at what bandwidth. | 304 | and at what bandwidth. |
285 | @item @file{nat/} --- libgnunetnat | 305 | @item @file{nat/} --- libgnunetnat |
@@ -295,14 +315,14 @@ transfer unit (MTU) for packets. The fragmentation library can be used to | |||
295 | break larger packets into chunks of at most 1k and transmit the resulting | 315 | break larger packets into chunks of at most 1k and transmit the resulting |
296 | fragments reliabily (with acknowledgement, retransmission, timeouts, | 316 | fragments reliabily (with acknowledgement, retransmission, timeouts, |
297 | etc.). | 317 | etc.). |
298 | @item @file{transport/} | 318 | @item @file{transport/} --- transport service |
299 | The transport service is responsible for managing the | 319 | The transport service is responsible for managing the |
300 | basic P2P communication. It uses plugins to support P2P communication | 320 | basic P2P communication. It uses plugins to support P2P communication |
301 | over TCP, UDP, HTTP, HTTPS and other protocols.The transport service | 321 | over TCP, UDP, HTTP, HTTPS and other protocols.The transport service |
302 | validates peer addresses, enforces bandwidth restrictions, limits the | 322 | validates peer addresses, enforces bandwidth restrictions, limits the |
303 | total number of connections and enforces connectivity restrictions (i.e. | 323 | total number of connections and enforces connectivity restrictions (i.e. |
304 | friends-only). | 324 | friends-only). |
305 | @item @file{peerinfo-tool/} | 325 | @item @file{peerinfo-tool/} --- gnunet-peerinfo |
306 | This directory contains the gnunet-peerinfo binary which can be used to | 326 | This directory contains the gnunet-peerinfo binary which can be used to |
307 | inspect the peers and HELLOs known to the peerinfo service. | 327 | inspect the peers and HELLOs known to the peerinfo service. |
308 | @item @file{core/} | 328 | @item @file{core/} |
@@ -315,7 +335,7 @@ for writing testcases. | |||
315 | It also supports automatic generation of configurations for peers | 335 | It also supports automatic generation of configurations for peers |
316 | ensuring that the ports and paths are disjoint. libgnunettesting is also | 336 | ensuring that the ports and paths are disjoint. libgnunettesting is also |
317 | the foundation for the testbed service | 337 | the foundation for the testbed service |
318 | @item @file{testbed/} | 338 | @item @file{testbed/} --- testbed service |
319 | The testbed service is used for creating small or large scale deployments | 339 | The testbed service is used for creating small or large scale deployments |
320 | of GNUnet peers for evaluation of protocols. | 340 | of GNUnet peers for evaluation of protocols. |
321 | It facilitates peer depolyments on multiple | 341 | It facilitates peer depolyments on multiple |
@@ -329,13 +349,13 @@ P2P network. | |||
329 | The distributed hash table (DHT) service provides a | 349 | The distributed hash table (DHT) service provides a |
330 | distributed implementation of a hash table to store blocks under hash | 350 | distributed implementation of a hash table to store blocks under hash |
331 | keys in the P2P network. | 351 | keys in the P2P network. |
332 | @item @file{hostlist/} | 352 | @item @file{hostlist/} --- hostlist service |
333 | The hostlist service allows learning about | 353 | The hostlist service allows learning about |
334 | other peers in the network by downloading HELLO messages from an HTTP | 354 | other peers in the network by downloading HELLO messages from an HTTP |
335 | server, can be configured to run such an HTTP server and also implements | 355 | server, can be configured to run such an HTTP server and also implements |
336 | a P2P protocol to advertise and automatically learn about other peers | 356 | a P2P protocol to advertise and automatically learn about other peers |
337 | that offer a public hostlist server. | 357 | that offer a public hostlist server. |
338 | @item @file{topology/} | 358 | @item @file{topology/} --- topology service |
339 | The topology service is responsible for | 359 | The topology service is responsible for |
340 | maintaining the mesh topology. It tries to maintain connections to friends | 360 | maintaining the mesh topology. It tries to maintain connections to friends |
341 | (depending on the configuration) and also tries to ensure that the peer | 361 | (depending on the configuration) and also tries to ensure that the peer |
@@ -349,7 +369,7 @@ connections are permitted (for friend-to-friend networking) | |||
349 | The file-sharing (FS) service implements GNUnet's | 369 | The file-sharing (FS) service implements GNUnet's |
350 | file-sharing application. Both anonymous file-sharing (using gap) and | 370 | file-sharing application. Both anonymous file-sharing (using gap) and |
351 | non-anonymous file-sharing (using dht) are supported. | 371 | non-anonymous file-sharing (using dht) are supported. |
352 | @item @file{cadet/} | 372 | @item @file{cadet/} --- cadet service |
353 | The CADET service provides a general-purpose routing abstraction to create | 373 | The CADET service provides a general-purpose routing abstraction to create |
354 | end-to-end encrypted tunnels in mesh networks. We wrote a paper | 374 | end-to-end encrypted tunnels in mesh networks. We wrote a paper |
355 | documenting key aspects of the design. | 375 | documenting key aspects of the design. |
@@ -368,7 +388,7 @@ Service that allows intercepting and modifying DNS requests of | |||
368 | the local machine. Currently used for IPv4-IPv6 protocol translation | 388 | the local machine. Currently used for IPv4-IPv6 protocol translation |
369 | (DNS-ALG) as implemented by "pt/" and for the GNUnet naming system. The | 389 | (DNS-ALG) as implemented by "pt/" and for the GNUnet naming system. The |
370 | service can also be configured to offer an exit service for DNS traffic. | 390 | service can also be configured to offer an exit service for DNS traffic. |
371 | @item @file{vpn/} | 391 | @item @file{vpn/} --- VPN service |
372 | The virtual public network (VPN) service provides a virtual | 392 | The virtual public network (VPN) service provides a virtual |
373 | tunnel interface (VTUN) for IP routing over GNUnet. | 393 | tunnel interface (VTUN) for IP routing over GNUnet. |
374 | Needs some other peers to run an "exit" service to work. | 394 | Needs some other peers to run an "exit" service to work. |
@@ -741,6 +761,7 @@ libgnunet_plugin_transport_tcp) | |||
741 | @node Coding style | 761 | @node Coding style |
742 | @subsection Coding style | 762 | @subsection Coding style |
743 | 763 | ||
764 | @c XXX: Adjust examples to GNU Standards! | ||
744 | @itemize @bullet | 765 | @itemize @bullet |
745 | @item We follow the GNU Coding Standards (@pxref{Top, The GNU Coding Standards,, standards, The GNU Coding Standards}); | 766 | @item We follow the GNU Coding Standards (@pxref{Top, The GNU Coding Standards,, standards, The GNU Coding Standards}); |
746 | @item Indentation is done with spaces, two per level, no tabs; | 767 | @item Indentation is done with spaces, two per level, no tabs; |
@@ -1346,31 +1367,31 @@ shell program. For e.g: | |||
1346 | 1367 | ||
1347 | @example | 1368 | @example |
1348 | export GNUNET_TESTBED_RSH_CMD="ssh -o BatchMode=yes \ | 1369 | export GNUNET_TESTBED_RSH_CMD="ssh -o BatchMode=yes \ |
1349 | -o NoHostAuthenticationForLocalhost=yes %h"@ | 1370 | -o NoHostAuthenticationForLocalhost=yes %h" |
1350 | @end example | 1371 | @end example |
1351 | 1372 | ||
1352 | Substitutions are allowed int the above command string also allows for | 1373 | Substitutions are allowed in the command string above, |
1353 | substitions. through placemarks which begin with a `%'. At present the | 1374 | this allows for substitutions through placemarks which begin with a `%'. |
1354 | following substitutions are supported | 1375 | At present the following substitutions are supported |
1355 | 1376 | ||
1356 | @itemize @bullet | 1377 | @itemize @bullet |
1357 | @item | 1378 | @item %h: hostname |
1358 | %h: hostname | 1379 | @item %u: username |
1359 | @item | 1380 | @item %p: port |
1360 | %u: username | ||
1361 | @item | ||
1362 | %p: port | ||
1363 | @end itemize | 1381 | @end itemize |
1364 | 1382 | ||
1365 | Note that the substitution placemark is replaced only when the | 1383 | Note that the substitution placemark is replaced only when the |
1366 | corresponding field is available and only once. Specifying | 1384 | corresponding field is available and only once. Specifying |
1385 | |||
1367 | @example | 1386 | @example |
1368 | %u@atchar{}%h | 1387 | %u@atchar{}%h |
1369 | @end example | 1388 | @end example |
1370 | doesn't work either. | 1389 | |
1371 | If you want to user username substitutions for SSH | 1390 | doesn't work either. If you want to user username substitutions for |
1372 | use the argument @code{-l} before the username substitution. | 1391 | @command{SSH}, use the argument @code{-l} before the |
1373 | For exmaple: | 1392 | username substitution. |
1393 | |||
1394 | For example: | ||
1374 | @example | 1395 | @example |
1375 | ssh -l %u -p %p %h | 1396 | ssh -l %u -p %p %h |
1376 | @end example | 1397 | @end example |