aboutsummaryrefslogtreecommitdiff
path: root/RATIONALE
diff options
context:
space:
mode:
Diffstat (limited to 'RATIONALE')
-rw-r--r--RATIONALE45
1 files changed, 24 insertions, 21 deletions
diff --git a/RATIONALE b/RATIONALE
index b37e1a2fe..d68e244c6 100644
--- a/RATIONALE
+++ b/RATIONALE
@@ -1,12 +1,13 @@
1This document is a summary of why we're moving to GNUnet NG and what 1This document is a summary of the changes made to GNUnet for version
2this major redesign tries to address. 20.9.x (from 0.8.x) and what this major redesign tries to address.
3 3
4First of all, the redesign does not (intentionally) change anything 4First of all, the redesign does not (intentionally) change anything
5fundamental about the application-level protocols or how files are 5fundamental about the application-level protocols or how files are
6encoded and shared. However, it is not protocol-compatible due to 6encoded and shared. However, it is not protocol-compatible due to
7other changes that do not relate to the essence of the application 7other changes that do not relate to the essence of the application
8protocols. 8protocols. This choice was made since productive development and
9 9readable code were considered more important than compatibility at
10this point.
10 11
11The redesign tries to address the following major problem groups 12The redesign tries to address the following major problem groups
12describing isssues that apply more or less to all GNUnet versions 13describing isssues that apply more or less to all GNUnet versions
@@ -26,10 +27,10 @@ PROBLEM GROUP 1 (scalability):
26 mutexes and almost 1000 lines of lock/unlock operations. 27 mutexes and almost 1000 lines of lock/unlock operations.
27 It is challenging for even good programmers to program or 28 It is challenging for even good programmers to program or
28 maintain good multi-threaded code with this complexity. 29 maintain good multi-threaded code with this complexity.
29 The excessive locking essentially prevents GNUnet from 30 The excessive locking essentially prevents GNUnet 0.8 from
30 actually doing much in parallel on multicores. 31 actually doing much in parallel on multicores.
31* Despite efforts like Freeway, it was virtually 32* Despite efforts like Freeway, it was virtually
32 impossible to contribute code to GNUnet that was not 33 impossible to contribute code to GNUnet 0.8 that was not
33 writen in C/C++. 34 writen in C/C++.
34* Changes to the configuration almost always required restarts 35* Changes to the configuration almost always required restarts
35 of gnunetd; the existence of change-notifications does not 36 of gnunetd; the existence of change-notifications does not
@@ -44,11 +45,11 @@ PROBLEM GROUP 1 (scalability):
44 days, result in really nasty and hard-to-find crashes. 45 days, result in really nasty and hard-to-find crashes.
45* structs of function pointers in service APIs were 46* structs of function pointers in service APIs were
46 needlessly adding complexity, especially since in 47 needlessly adding complexity, especially since in
47 most cases there was no polymorphism 48 most cases there was no actual polymorphism
48 49
49SOLUTION: 50SOLUTION:
50* Use multiple, lously-coupled processes and one big select 51* Use multiple, lously-coupled processes and one big select
51 loop in each (supported by a powerful library to eliminate 52 loop in each (supported by a powerful util library to eliminate
52 code duplication for each process). 53 code duplication for each process).
53* Eliminate all threads, manage the processes with a 54* Eliminate all threads, manage the processes with a
54 master-process (gnunet-arm, for automatic restart manager) 55 master-process (gnunet-arm, for automatic restart manager)
@@ -65,13 +66,15 @@ SOLUTION:
65 => Process priorities can be used to schedule the CPU better 66 => Process priorities can be used to schedule the CPU better
66 Note that we can not just use one process with a big 67 Note that we can not just use one process with a big
67 select loop because we have blocking operations (and the 68 select loop because we have blocking operations (and the
68 blocking is outside of our control, thanks MySQL, 69 blocking is outside of our control, thanks to MySQL,
69 sqlite, gethostbyaddr, etc.). So in order to perform 70 sqlite, gethostbyaddr, etc.). So in order to perform
70 reasonably well, we need some construct for parallel 71 reasonably well, we need some construct for parallel
71 execution. 72 execution.
72 73
73 RULE: If your service contains blocking functions, it 74 RULE: If your service contains blocking functions, it
74 MUST be a process by itself. 75 MUST be a process by itself. If your service
76 is sufficiently complex, you MAY choose to make
77 it a separate process.
75* Eliminate structs with function pointers for service APIs; 78* Eliminate structs with function pointers for service APIs;
76 instead, provide a library (still ending in _service.h) API 79 instead, provide a library (still ending in _service.h) API
77 that transmits the requests nicely to the respective 80 that transmits the requests nicely to the respective
@@ -121,6 +124,8 @@ SOLUTION:
121 thing given the potential for bugs. 124 thing given the potential for bugs.
122* There is no more TIME API function to do anything 125* There is no more TIME API function to do anything
123 with 32-bit seconds 126 with 32-bit seconds
127* There is now a bandwidth API to handle
128 non-trivial bandwidth utilization calculations
124 129
125 130
126PROBLEM GROUP 3 (statistics): 131PROBLEM GROUP 3 (statistics):
@@ -237,18 +242,15 @@ PROBLEM GROUP 6 (FS-APIs):
237* FILENAME metadata is killed by ECRS/FSUI to avoid 242* FILENAME metadata is killed by ECRS/FSUI to avoid
238 exposing HOME, but what if the user set it manually? 243 exposing HOME, but what if the user set it manually?
239 244
240SOLUTION (draft, not done yet, details missing...): 245SOLUTION:
241* Eliminate threads from FS-APIs 246* Eliminate threads from FS-APIs
242 => Open question: how to best write the APIs to 247* Incrementally store FS-state always also on disk using many
243 allow integration with diverse event loops 248 small files instead of one big file
244 of GUI libraries?
245* Store FS-state always also on disk
246 => Open question: how to do this without
247 compromising state/scalability?
248* Have API to manipulate sharing tree before 249* Have API to manipulate sharing tree before
249 upload; have auto-construction modify FILENAME 250 upload; have auto-construction modify FILENAME
250 but allow user-modifications afterwards 251 but allow user-modifications afterwards
251 252
253
252PROBLEM GROUP 7 (User experience): 254PROBLEM GROUP 7 (User experience):
253* Searches often do not return a sufficient / significant number of 255* Searches often do not return a sufficient / significant number of
254 results 256 results
@@ -281,11 +283,12 @@ SUMMARY:
281 - network traffic monitors (goal: eliminate) 283 - network traffic monitors (goal: eliminate)
282 - IPC semaphores (goal: d-bus? / eliminate?) 284 - IPC semaphores (goal: d-bus? / eliminate?)
283 - second timers 285 - second timers
284 - DNS lookup (goal: have async service; issue: still need synchronous
285 resolution in places, current code may not be portable)
286 => code shrunk from 61 files to 34, 22k LOC to 15k LOC, 470k to 330k (with symbols)
287* New features in util: 286* New features in util:
288 - scheduler 287 - scheduler
289 - service and program boot-strap code 288 - service and program boot-strap code
289 - bandwidth and time APIs
290 - buffered IO API
291 - HKDF implementation (crypto)
290* Major changes in util: 292* Major changes in util:
291 - more expressive server (replaces selector) 293 - more expressive server (replaces selector)
294 - DNS lookup replaced by async service