aboutsummaryrefslogtreecommitdiff
path: root/doc/documentation/chapters
diff options
context:
space:
mode:
authorng0 <ng0@infotropique.org>2017-10-26 12:28:28 +0000
committerng0 <ng0@infotropique.org>2017-10-26 12:28:28 +0000
commite2f0870adf4baad5dfefaefd92c122ee9e1e0e5a (patch)
tree9374b281f7f450e9bd6c73ace85bb6d627fdd9d9 /doc/documentation/chapters
parent17e755c71521be9a84f0f3b46d1de6962298f733 (diff)
downloadgnunet-e2f0870adf4baad5dfefaefd92c122ee9e1e0e5a.tar.gz
gnunet-e2f0870adf4baad5dfefaefd92c122ee9e1e0e5a.zip
documentation
Diffstat (limited to 'doc/documentation/chapters')
-rw-r--r--doc/documentation/chapters/developer.texi169
1 files changed, 95 insertions, 74 deletions
diff --git a/doc/documentation/chapters/developer.texi b/doc/documentation/chapters/developer.texi
index 996474359..9459068a9 100644
--- a/doc/documentation/chapters/developer.texi
+++ b/doc/documentation/chapters/developer.texi
@@ -4,11 +4,14 @@
4 4
5This book is intended to be an introduction for programmers that want to 5This book is intended to be an introduction for programmers that want to
6extend the GNUnet framework. GNUnet is more than a simple peer-to-peer 6extend the GNUnet framework. GNUnet is more than a simple peer-to-peer
7application. For developers, GNUnet is: 7application.
8
9For developers, GNUnet is:
8 10
9@itemize @bullet 11@itemize @bullet
10@item Free software under the GNU General Public License, with a community 12@item developed by a community that believes in the GNU philosophy
11that believes in the GNU philosophy 13@item Free Software (Free as in Freedom), licensed under the
14@xref{GNU General Public License}
12@item A set of standards, including coding conventions and 15@item A set of standards, including coding conventions and
13architectural rules 16architectural rules
14@item A set of layered protocols, both specifying the communication 17@item A set of layered protocols, both specifying the communication
@@ -20,17 +23,19 @@ writing extensions
20 23
21In particular, the architecture specifies that a peer consists of many 24In particular, the architecture specifies that a peer consists of many
22processes communicating via protocols. Processes can be written in almost 25processes communicating via protocols. Processes can be written in almost
23any language. C and Java APIs exist for accessing existing services and 26any language.
24for writing extensions. It is possible to write extensions in other 27C and Java APIs exist for accessing existing services and for writing
25languages by implementing the necessary IPC protocols. 28extensions. It is possible to write extensions in other languages by
29implementing the necessary IPC protocols.
26 30
27GNUnet can be extended and improved along many possible dimensions, and 31GNUnet can be extended and improved along many possible dimensions, and
28anyone interested in Free Software and Freedom-enhancing Networking is 32anyone interested in Free Software and Freedom-enhancing Networking is
29welcome to join the effort. This Developer Handbook attempts to provide 33welcome to join the effort. This Developer Handbook attempts to provide
30an initial introduction to some of the key design choices and central 34an initial introduction to some of the key design choices and central
31components of the system. This part of the GNUNet documentation 35components of the system.
32is far from complete, and we welcome informed contributions, 36This part of the GNUNet documentation is far from complete,
33be it in the form of new chapters or insightful comments. 37and we welcome informed contributions, be it in the form of
38new chapters, sections or insightful comments.
34 39
35@menu 40@menu
36* Developer Introduction:: 41* Developer Introduction::
@@ -92,11 +97,13 @@ following links:
92@item GNUnet Java tutorial 97@item GNUnet Java tutorial
93@end itemize 98@end itemize
94 99
95In addition to this book, the GNUnet server contains various resources for 100In addition to the GNUnet Reference Documentation you are reading,
96GNUnet developers. They are all conveniently reachable via the "Developer" 101the GNUnet server contains various resources for GNUnet
102developers and those who aspire to become regular contributors.
103They are all conveniently reachable via the "Developer"
97entry in the navigation menu. Some additional tools (such as static 104entry in the navigation menu. Some additional tools (such as static
98analysis reports) require a special developer access to perform certain 105analysis reports) require a special developer access to perform certain
99operations. If you feel you need access, you should contact 106operations. If you want (or require) access, you should contact
100@uref{http://grothoff.org/christian/, Christian Grothoff}, 107@uref{http://grothoff.org/christian/, Christian Grothoff},
101GNUnet's maintainer. 108GNUnet's maintainer.
102 109
@@ -104,20 +111,26 @@ The public subsystems on the GNUnet server that help developers are:
104 111
105@itemize @bullet 112@itemize @bullet
106 113
107@item The Version control system (git) keeps our code and enables 114@item The version control system (git) keeps our code and enables
108distributed development. 115distributed development.
116It is pubclicly accessible at @uref{https://gnunet.org/git/}.
109Only developers with write access can commit code, everyone else is 117Only developers with write access can commit code, everyone else is
110encouraged to submit patches to the 118encouraged to submit patches to the
111@uref{https://lists.gnu.org/mailman/listinfo/gnunet-developers, GNUnet-developers mailinglist} 119@uref{https://lists.gnu.org/mailman/listinfo/gnunet-developers, GNUnet-developers mailinglist}.
112. 120
113 121@item The bugtracking system (Mantis).
114@item The GNUnet bugtracking system (Mantis) is used to track 122We use it to track feature requests, open bug reports and their
115feature requests, open bug reports and their resolutions. 123resolutions.
116Anyone can report bugs, only developers can claim to have fixed them. 124It can be accessed at @uref{https://gnunet.org/bugs/}.
117 125Anyone can report bugs, but only developers can claim to have fixed them.
118@item A site installation of the CI system "Buildbot" is used to check 126
119GNUnet builds automatically on a range of platforms. 127@item Our site installation of the
120Builds are triggered automatically after 30 minutes of no changes to Git. 128CI@footnote{Continuous Integration} system "@code{Buildbot}" is used
129to check GNUnet builds automatically on a range of platforms.
130The web interface of this CI is exposed at
131@uref{https://gnunet.org/buildbot/}.
132Builds are triggered automatically 30 minutes after the last commit to
133our repository was made.
121 134
122@item The current quality of our automated test suite is assessed using 135@item The current quality of our automated test suite is assessed using
123Code coverage analysis. This analysis is run daily; however the webpage 136Code coverage analysis. This analysis is run daily; however the webpage
@@ -163,34 +176,41 @@ GNUnet sub-projects in order of likely relevance are currently:
163 176
164@table @asis 177@table @asis
165 178
166@item gnunet 179@item @command{gnunet}
167Core of the P2P framework, including file-sharing, VPN and 180Core of the P2P framework, including file-sharing, VPN and
168chat applications; this is what the developer handbook covers mostly 181chat applications; this is what the Developer Handbook covers mostly
169@item gnunet-gtk Gtk+-based user interfaces, including gnunet-fs-gtk 182@item @command{gnunet-gtk}
170(file-sharing), gnunet-statistics-gtk (statistics over time), 183Gtk+-based user interfaces, including:
171gnunet-peerinfo-gtk (information about current connections and known 184
172peers), gnunet-chat-gtk (chat GUI) and gnunet-setup (setup tool for 185@itemize @bullet
173"everything") 186@item @command{gnunet-fs-gtk} (file-sharing),
174@item gnunet-fuse 187@item @command{gnunet-statistics-gtk} (statistics over time),
188@item @command{gnunet-peerinfo-gtk}
189(information about current connections and known peers),
190@item @command{gnunet-chat-gtk} (chat GUI) and
191@item @command{gnunet-setup} (setup tool for "everything")
192@end itemize
193
194@item @command{gnunet-fuse}
175Mounting directories shared via GNUnet's file-sharing 195Mounting directories shared via GNUnet's file-sharing
176on Linux 196on GNU/Linux distributions
177@item gnunet-update 197@item @command{gnunet-update}
178Installation and update tool 198Installation and update tool
179@item gnunet-ext 199@item @command{gnunet-ext}
180Template for starting 'external' GNUnet projects 200Template for starting 'external' GNUnet projects
181@item gnunet-java 201@item @command{gnunet-java}
182Java APIs for writing GNUnet services and applications 202Java APIs for writing GNUnet services and applications
183@c ** FIXME: Point to new website repository once we have it: 203@c ** FIXME: Point to new website repository once we have it:
184@c ** @item svn/gnunet-www/ Code and media helping drive the GNUnet 204@c ** @item svn/gnunet-www/ Code and media helping drive the GNUnet
185@c website 205@c website
186@item eclectic 206@item @command{eclectic}
187Code to run GNUnet nodes on testbeds for research, development, 207Code to run GNUnet nodes on testbeds for research, development,
188testing and evaluation 208testing and evaluation
189@c ** FIXME: Solve the status and location of gnunet-qt 209@c ** FIXME: Solve the status and location of gnunet-qt
190@item gnunet-qt 210@item @command{gnunet-qt}
191Qt-based GNUnet GUI (dead?) 211Qt-based GNUnet GUI (is it depreacated?)
192@item gnunet-cocoa 212@item @command{gnunet-cocoa}
193cocoa-based GNUnet GUI (dead?) 213cocoa-based GNUnet GUI (is it depreacated?)
194 214
195@end table 215@end table
196 216
@@ -198,19 +218,19 @@ We are also working on various supporting libraries and tools:
198@c ** FIXME: What about gauger, and what about libmwmodem? 218@c ** FIXME: What about gauger, and what about libmwmodem?
199 219
200@table @asis 220@table @asis
201@item libextractor 221@item @command{libextractor}
202GNU libextractor (meta data extraction) 222GNU libextractor (meta data extraction)
203@item libmicrohttpd 223@item @command{libmicrohttpd}
204GNU libmicrohttpd (embedded HTTP(S) server library) 224GNU libmicrohttpd (embedded HTTP(S) server library)
205@item gauger 225@item @command{gauger}
206Tool for performance regression analysis 226Tool for performance regression analysis
207@item monkey 227@item @command{monkey}
208Tool for automated debugging of distributed systems 228Tool for automated debugging of distributed systems
209@item libmwmodem 229@item @command{libmwmodem}
210Library for accessing satellite connection quality 230Library for accessing satellite connection quality
211reports 231reports
212@item libgnurl 232@item @command{libgnurl}
213gnURL (feature restricted variant of cURL/libcurl) 233gnURL (feature-restricted variant of cURL/libcurl)
214@end table 234@end table
215 235
216Finally, there are various external projects (see links for a list of 236Finally, there are various external projects (see links for a list of
@@ -247,7 +267,7 @@ type defines a particular format and how that binary format is to be
247linked to a hash code (the key for the DHT and for databases). The block 267linked to a hash code (the key for the DHT and for databases). The block
248library is a wapper around block plugins which provide the necessary 268library is a wapper around block plugins which provide the necessary
249functions for each block type. 269functions for each block type.
250@item @file{statistics/} 270@item @file{statistics/} --- statistics service
251The statistics service enables associating 271The statistics service enables associating
252values (of type uint64_t) with a componenet name and a string. The main 272values (of type uint64_t) with a componenet name and a string. The main
253uses is debugging (counting events), performance tracking and user 273uses is debugging (counting events), performance tracking and user
@@ -257,7 +277,7 @@ The automatic-restart-manager (ARM) service
257is the GNUnet master service. Its role is to start gnunet-services, to 277is the GNUnet master service. Its role is to start gnunet-services, to
258re-start them when they crashed and finally to shut down the system when 278re-start them when they crashed and finally to shut down the system when
259requested. 279requested.
260@item @file{peerinfo/} 280@item @file{peerinfo/} --- peerinfo service
261The peerinfo service keeps track of which peers are known 281The peerinfo service keeps track of which peers are known
262to the local peer and also tracks the validated addresses for each peer 282to the local peer and also tracks the validated addresses for each peer
263(in the form of a HELLO message) for each of those peers. The peer is not 283(in the form of a HELLO message) for each of those peers. The peer is not
@@ -269,17 +289,17 @@ The datacache library provides (temporary) block storage for the DHT.
269Existing plugins can store blocks in Sqlite, Postgres or MySQL databases. 289Existing plugins can store blocks in Sqlite, Postgres or MySQL databases.
270All data stored in the cache is lost when the peer is stopped or 290All data stored in the cache is lost when the peer is stopped or
271restarted (datacache uses temporary tables). 291restarted (datacache uses temporary tables).
272@item @file{datastore/} 292@item @file{datastore/} --- datastore service
273The datastore service stores file-sharing blocks in 293The datastore service stores file-sharing blocks in
274databases for extended periods of time. In contrast to the datacache, data 294databases for extended periods of time. In contrast to the datacache, data
275is not lost when peers restart. However, quota restrictions may still 295is not lost when peers restart. However, quota restrictions may still
276cause old, expired or low-priority data to be eventually discarded. 296cause old, expired or low-priority data to be eventually discarded.
277Existing plugins can store blocks in Sqlite, Postgres or MySQL databases. 297Existing plugins can store blocks in Sqlite, Postgres or MySQL databases.
278@item @file{template/} 298@item @file{template/} --- service template
279Template for writing a new service. Does nothing. 299Template for writing a new service. Does nothing.
280@item @file{ats/} --- Automatic Transport Selection 300@item @file{ats/} --- Automatic Transport Selection
281The automatic transport 301The automatic transport selection (ATS) service
282selection (ATS) service is responsible for deciding which address (i.e. 302is responsible for deciding which address (i.e.
283which transport plugin) should be used for communication with other peers, 303which transport plugin) should be used for communication with other peers,
284and at what bandwidth. 304and at what bandwidth.
285@item @file{nat/} --- libgnunetnat 305@item @file{nat/} --- libgnunetnat
@@ -295,14 +315,14 @@ transfer unit (MTU) for packets. The fragmentation library can be used to
295break larger packets into chunks of at most 1k and transmit the resulting 315break larger packets into chunks of at most 1k and transmit the resulting
296fragments reliabily (with acknowledgement, retransmission, timeouts, 316fragments reliabily (with acknowledgement, retransmission, timeouts,
297etc.). 317etc.).
298@item @file{transport/} 318@item @file{transport/} --- transport service
299The transport service is responsible for managing the 319The transport service is responsible for managing the
300basic P2P communication. It uses plugins to support P2P communication 320basic P2P communication. It uses plugins to support P2P communication
301over TCP, UDP, HTTP, HTTPS and other protocols.The transport service 321over TCP, UDP, HTTP, HTTPS and other protocols.The transport service
302validates peer addresses, enforces bandwidth restrictions, limits the 322validates peer addresses, enforces bandwidth restrictions, limits the
303total number of connections and enforces connectivity restrictions (i.e. 323total number of connections and enforces connectivity restrictions (i.e.
304friends-only). 324friends-only).
305@item @file{peerinfo-tool/} 325@item @file{peerinfo-tool/} --- gnunet-peerinfo
306This directory contains the gnunet-peerinfo binary which can be used to 326This directory contains the gnunet-peerinfo binary which can be used to
307inspect the peers and HELLOs known to the peerinfo service. 327inspect the peers and HELLOs known to the peerinfo service.
308@item @file{core/} 328@item @file{core/}
@@ -315,7 +335,7 @@ for writing testcases.
315It also supports automatic generation of configurations for peers 335It also supports automatic generation of configurations for peers
316ensuring that the ports and paths are disjoint. libgnunettesting is also 336ensuring that the ports and paths are disjoint. libgnunettesting is also
317the foundation for the testbed service 337the foundation for the testbed service
318@item @file{testbed/} 338@item @file{testbed/} --- testbed service
319The testbed service is used for creating small or large scale deployments 339The testbed service is used for creating small or large scale deployments
320of GNUnet peers for evaluation of protocols. 340of GNUnet peers for evaluation of protocols.
321It facilitates peer depolyments on multiple 341It facilitates peer depolyments on multiple
@@ -329,13 +349,13 @@ P2P network.
329The distributed hash table (DHT) service provides a 349The distributed hash table (DHT) service provides a
330distributed implementation of a hash table to store blocks under hash 350distributed implementation of a hash table to store blocks under hash
331keys in the P2P network. 351keys in the P2P network.
332@item @file{hostlist/} 352@item @file{hostlist/} --- hostlist service
333The hostlist service allows learning about 353The hostlist service allows learning about
334other peers in the network by downloading HELLO messages from an HTTP 354other peers in the network by downloading HELLO messages from an HTTP
335server, can be configured to run such an HTTP server and also implements 355server, can be configured to run such an HTTP server and also implements
336a P2P protocol to advertise and automatically learn about other peers 356a P2P protocol to advertise and automatically learn about other peers
337that offer a public hostlist server. 357that offer a public hostlist server.
338@item @file{topology/} 358@item @file{topology/} --- topology service
339The topology service is responsible for 359The topology service is responsible for
340maintaining the mesh topology. It tries to maintain connections to friends 360maintaining the mesh topology. It tries to maintain connections to friends
341(depending on the configuration) and also tries to ensure that the peer 361(depending on the configuration) and also tries to ensure that the peer
@@ -349,7 +369,7 @@ connections are permitted (for friend-to-friend networking)
349The file-sharing (FS) service implements GNUnet's 369The file-sharing (FS) service implements GNUnet's
350file-sharing application. Both anonymous file-sharing (using gap) and 370file-sharing application. Both anonymous file-sharing (using gap) and
351non-anonymous file-sharing (using dht) are supported. 371non-anonymous file-sharing (using dht) are supported.
352@item @file{cadet/} 372@item @file{cadet/} --- cadet service
353The CADET service provides a general-purpose routing abstraction to create 373The CADET service provides a general-purpose routing abstraction to create
354end-to-end encrypted tunnels in mesh networks. We wrote a paper 374end-to-end encrypted tunnels in mesh networks. We wrote a paper
355documenting key aspects of the design. 375documenting key aspects of the design.
@@ -368,7 +388,7 @@ Service that allows intercepting and modifying DNS requests of
368the local machine. Currently used for IPv4-IPv6 protocol translation 388the local machine. Currently used for IPv4-IPv6 protocol translation
369(DNS-ALG) as implemented by "pt/" and for the GNUnet naming system. The 389(DNS-ALG) as implemented by "pt/" and for the GNUnet naming system. The
370service can also be configured to offer an exit service for DNS traffic. 390service can also be configured to offer an exit service for DNS traffic.
371@item @file{vpn/} 391@item @file{vpn/} --- VPN service
372The virtual public network (VPN) service provides a virtual 392The virtual public network (VPN) service provides a virtual
373tunnel interface (VTUN) for IP routing over GNUnet. 393tunnel interface (VTUN) for IP routing over GNUnet.
374Needs some other peers to run an "exit" service to work. 394Needs some other peers to run an "exit" service to work.
@@ -741,6 +761,7 @@ libgnunet_plugin_transport_tcp)
741@node Coding style 761@node Coding style
742@subsection Coding style 762@subsection Coding style
743 763
764@c XXX: Adjust examples to GNU Standards!
744@itemize @bullet 765@itemize @bullet
745@item We follow the GNU Coding Standards (@pxref{Top, The GNU Coding Standards,, standards, The GNU Coding Standards}); 766@item We follow the GNU Coding Standards (@pxref{Top, The GNU Coding Standards,, standards, The GNU Coding Standards});
746@item Indentation is done with spaces, two per level, no tabs; 767@item Indentation is done with spaces, two per level, no tabs;
@@ -1346,31 +1367,31 @@ shell program. For e.g:
1346 1367
1347@example 1368@example
1348export GNUNET_TESTBED_RSH_CMD="ssh -o BatchMode=yes \ 1369export GNUNET_TESTBED_RSH_CMD="ssh -o BatchMode=yes \
1349-o NoHostAuthenticationForLocalhost=yes %h"@ 1370-o NoHostAuthenticationForLocalhost=yes %h"
1350@end example 1371@end example
1351 1372
1352Substitutions are allowed int the above command string also allows for 1373Substitutions are allowed in the command string above,
1353substitions. through placemarks which begin with a `%'. At present the 1374this allows for substitutions through placemarks which begin with a `%'.
1354following substitutions are supported 1375At present the following substitutions are supported
1355 1376
1356@itemize @bullet 1377@itemize @bullet
1357@item 1378@item %h: hostname
1358%h: hostname 1379@item %u: username
1359@item 1380@item %p: port
1360%u: username
1361@item
1362%p: port
1363@end itemize 1381@end itemize
1364 1382
1365Note that the substitution placemark is replaced only when the 1383Note that the substitution placemark is replaced only when the
1366corresponding field is available and only once. Specifying 1384corresponding field is available and only once. Specifying
1385
1367@example 1386@example
1368%u@atchar{}%h 1387%u@atchar{}%h
1369@end example 1388@end example
1370doesn't work either. 1389
1371If you want to user username substitutions for SSH 1390doesn't work either. If you want to user username substitutions for
1372use the argument @code{-l} before the username substitution. 1391@command{SSH}, use the argument @code{-l} before the
1373For exmaple: 1392username substitution.
1393
1394For example:
1374@example 1395@example
1375ssh -l %u -p %p %h 1396ssh -l %u -p %p %h
1376@end example 1397@end example