diff options
Diffstat (limited to 'RATIONALE')
-rw-r--r-- | RATIONALE | 45 |
1 files changed, 24 insertions, 21 deletions
@@ -1,12 +1,13 @@ | |||
1 | This document is a summary of why we're moving to GNUnet NG and what | 1 | This document is a summary of the changes made to GNUnet for version |
2 | this major redesign tries to address. | 2 | 0.9.x (from 0.8.x) and what this major redesign tries to address. |
3 | 3 | ||
4 | First of all, the redesign does not (intentionally) change anything | 4 | First of all, the redesign does not (intentionally) change anything |
5 | fundamental about the application-level protocols or how files are | 5 | fundamental about the application-level protocols or how files are |
6 | encoded and shared. However, it is not protocol-compatible due to | 6 | encoded and shared. However, it is not protocol-compatible due to |
7 | other changes that do not relate to the essence of the application | 7 | other changes that do not relate to the essence of the application |
8 | protocols. | 8 | protocols. This choice was made since productive development and |
9 | 9 | readable code were considered more important than compatibility at | |
10 | this point. | ||
10 | 11 | ||
11 | The redesign tries to address the following major problem groups | 12 | The redesign tries to address the following major problem groups |
12 | describing isssues that apply more or less to all GNUnet versions | 13 | describing isssues that apply more or less to all GNUnet versions |
@@ -26,10 +27,10 @@ PROBLEM GROUP 1 (scalability): | |||
26 | mutexes and almost 1000 lines of lock/unlock operations. | 27 | mutexes and almost 1000 lines of lock/unlock operations. |
27 | It is challenging for even good programmers to program or | 28 | It is challenging for even good programmers to program or |
28 | maintain good multi-threaded code with this complexity. | 29 | maintain good multi-threaded code with this complexity. |
29 | The excessive locking essentially prevents GNUnet from | 30 | The excessive locking essentially prevents GNUnet 0.8 from |
30 | actually doing much in parallel on multicores. | 31 | actually doing much in parallel on multicores. |
31 | * Despite efforts like Freeway, it was virtually | 32 | * Despite efforts like Freeway, it was virtually |
32 | impossible to contribute code to GNUnet that was not | 33 | impossible to contribute code to GNUnet 0.8 that was not |
33 | writen in C/C++. | 34 | writen in C/C++. |
34 | * Changes to the configuration almost always required restarts | 35 | * Changes to the configuration almost always required restarts |
35 | of gnunetd; the existence of change-notifications does not | 36 | of gnunetd; the existence of change-notifications does not |
@@ -44,11 +45,11 @@ PROBLEM GROUP 1 (scalability): | |||
44 | days, result in really nasty and hard-to-find crashes. | 45 | days, result in really nasty and hard-to-find crashes. |
45 | * structs of function pointers in service APIs were | 46 | * structs of function pointers in service APIs were |
46 | needlessly adding complexity, especially since in | 47 | needlessly adding complexity, especially since in |
47 | most cases there was no polymorphism | 48 | most cases there was no actual polymorphism |
48 | 49 | ||
49 | SOLUTION: | 50 | SOLUTION: |
50 | * Use multiple, lously-coupled processes and one big select | 51 | * Use multiple, lously-coupled processes and one big select |
51 | loop in each (supported by a powerful library to eliminate | 52 | loop in each (supported by a powerful util library to eliminate |
52 | code duplication for each process). | 53 | code duplication for each process). |
53 | * Eliminate all threads, manage the processes with a | 54 | * Eliminate all threads, manage the processes with a |
54 | master-process (gnunet-arm, for automatic restart manager) | 55 | master-process (gnunet-arm, for automatic restart manager) |
@@ -65,13 +66,15 @@ SOLUTION: | |||
65 | => Process priorities can be used to schedule the CPU better | 66 | => Process priorities can be used to schedule the CPU better |
66 | Note that we can not just use one process with a big | 67 | Note that we can not just use one process with a big |
67 | select loop because we have blocking operations (and the | 68 | select loop because we have blocking operations (and the |
68 | blocking is outside of our control, thanks MySQL, | 69 | blocking is outside of our control, thanks to MySQL, |
69 | sqlite, gethostbyaddr, etc.). So in order to perform | 70 | sqlite, gethostbyaddr, etc.). So in order to perform |
70 | reasonably well, we need some construct for parallel | 71 | reasonably well, we need some construct for parallel |
71 | execution. | 72 | execution. |
72 | 73 | ||
73 | RULE: If your service contains blocking functions, it | 74 | RULE: If your service contains blocking functions, it |
74 | MUST be a process by itself. | 75 | MUST be a process by itself. If your service |
76 | is sufficiently complex, you MAY choose to make | ||
77 | it a separate process. | ||
75 | * Eliminate structs with function pointers for service APIs; | 78 | * Eliminate structs with function pointers for service APIs; |
76 | instead, provide a library (still ending in _service.h) API | 79 | instead, provide a library (still ending in _service.h) API |
77 | that transmits the requests nicely to the respective | 80 | that transmits the requests nicely to the respective |
@@ -121,6 +124,8 @@ SOLUTION: | |||
121 | thing given the potential for bugs. | 124 | thing given the potential for bugs. |
122 | * There is no more TIME API function to do anything | 125 | * There is no more TIME API function to do anything |
123 | with 32-bit seconds | 126 | with 32-bit seconds |
127 | * There is now a bandwidth API to handle | ||
128 | non-trivial bandwidth utilization calculations | ||
124 | 129 | ||
125 | 130 | ||
126 | PROBLEM GROUP 3 (statistics): | 131 | PROBLEM GROUP 3 (statistics): |
@@ -237,18 +242,15 @@ PROBLEM GROUP 6 (FS-APIs): | |||
237 | * FILENAME metadata is killed by ECRS/FSUI to avoid | 242 | * FILENAME metadata is killed by ECRS/FSUI to avoid |
238 | exposing HOME, but what if the user set it manually? | 243 | exposing HOME, but what if the user set it manually? |
239 | 244 | ||
240 | SOLUTION (draft, not done yet, details missing...): | 245 | SOLUTION: |
241 | * Eliminate threads from FS-APIs | 246 | * Eliminate threads from FS-APIs |
242 | => Open question: how to best write the APIs to | 247 | * Incrementally store FS-state always also on disk using many |
243 | allow integration with diverse event loops | 248 | small files instead of one big file |
244 | of GUI libraries? | ||
245 | * Store FS-state always also on disk | ||
246 | => Open question: how to do this without | ||
247 | compromising state/scalability? | ||
248 | * Have API to manipulate sharing tree before | 249 | * Have API to manipulate sharing tree before |
249 | upload; have auto-construction modify FILENAME | 250 | upload; have auto-construction modify FILENAME |
250 | but allow user-modifications afterwards | 251 | but allow user-modifications afterwards |
251 | 252 | ||
253 | |||
252 | PROBLEM GROUP 7 (User experience): | 254 | PROBLEM GROUP 7 (User experience): |
253 | * Searches often do not return a sufficient / significant number of | 255 | * Searches often do not return a sufficient / significant number of |
254 | results | 256 | results |
@@ -281,11 +283,12 @@ SUMMARY: | |||
281 | - network traffic monitors (goal: eliminate) | 283 | - network traffic monitors (goal: eliminate) |
282 | - IPC semaphores (goal: d-bus? / eliminate?) | 284 | - IPC semaphores (goal: d-bus? / eliminate?) |
283 | - second timers | 285 | - second timers |
284 | - DNS lookup (goal: have async service; issue: still need synchronous | ||
285 | resolution in places, current code may not be portable) | ||
286 | => code shrunk from 61 files to 34, 22k LOC to 15k LOC, 470k to 330k (with symbols) | ||
287 | * New features in util: | 286 | * New features in util: |
288 | - scheduler | 287 | - scheduler |
289 | - service and program boot-strap code | 288 | - service and program boot-strap code |
289 | - bandwidth and time APIs | ||
290 | - buffered IO API | ||
291 | - HKDF implementation (crypto) | ||
290 | * Major changes in util: | 292 | * Major changes in util: |
291 | - more expressive server (replaces selector) | 293 | - more expressive server (replaces selector) |
294 | - DNS lookup replaced by async service | ||