7 files changed, 208 insertions, 103 deletions

diff --git a/doc/Makefile.am b/doc/Makefile.am
index 28db606c5..1070974d1 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -7,4 +7,4 @@ if !DOCUMENTATION
 endif
 
 EXTRA_DIST =                                                    \
-        outdated-and-old-installation-instructions.txt          
+        outdated-and-old-installation-instructions.txt
diff --git a/doc/documentation/.gitignore b/doc/documentation/.gitignore
index 2e914d9c9..f490c3412 100644
--- a/doc/documentation/.gitignore
+++ b/doc/documentation/.gitignore
@@ -1,2 +1,9 @@
 stamp-1
 version2.texi
+manual
+*.fn
+*.fns
+*.ky
+*.pg
+*.tp
+*.vr
diff --git a/doc/documentation/README.txt b/doc/documentation/README.txt
deleted file mode 100644
index 9e76394c3..000000000
--- a/doc/documentation/README.txt
+++ /dev/null
@@ -1,63 +0,0 @@
-* Completion Levels:
-
-** chapters/philosophy: around 100% fixed after initial export.
-
-* What's left to do
-
-- Which Texlive modules are needed? Decrease the size.
-  - distro specific, or can we set requirements?
-- Update the content of gnunet documentation.
-- XXX: images are only generated for the html documentation
-  with gendoc.sh … FIXME!
-- XXX: png,dot, and svg images MUST be converted to eps by the
-  build system. Right now they aren't, as a result: No images.
-
-* How to use (hack) on this
-
-** with guix
-
-Adjust accordingly, ie read the Guix Documentation:
-setenv GUIX_PACKAGE_PATH "gnunet/contrib/packages/guix/packages"
-guix environment gnunet-doc
-and
-guix build -f contrib/packages/guix/gnunet-doc.scm
-
-** without guix
-
-You need to have Texinfo and Texlive in your path.
-sh bootstrap
-./configure --enable-documentation
-cd doc
-make (format you want)
-
-for example: make html, make info, make pdf
-
-* structure (relations)
-
-** gnunet.texi
- -> chapters/developer.texi
- -> chapters/installation.texi
- -> chapters/philosophy.texi
- -> chapters/user.texi
- -> chapters/vocabulary.texi
- -> images/*
- -> gpl-3.0.texi
- -> fdl-1.3.texi
-
-** gnunet-c-tutorial.texi
- -> figs/Service.pdf
- -> figs/System.pdf
- -> tutorial-examples/*.c
- -> gpl-3.0.texi
- -> fdl-1.3.texi
-
-- gnunet-c-tutorial-v1.pdf: original LaTeX "gnunet-c-tutorial.pdf".
-- man folder: the man pages.
-- doxygen folder
-- outdated-and-old-installation-instructions.txt: self described within the file.
-
-
-Use `gendocs', add to the manual/ directory of the web site.
-
-  $ cd doc
-  $ gendocs.sh gnunet "GNUnet 0.10.X Reference Manual"
diff --git a/doc/documentation/TODO b/doc/documentation/TODO
new file mode 100644
index 000000000..fa1ce7a23
--- /dev/null
+++ b/doc/documentation/TODO
@@ -0,0 +1,106 @@
+-*- mode: org -*-
+
+TODO - or: the Documentation Masterplan.
+
+To some extent this duplicates the Mantis tickets on this topic.
+
+* Motivation
+My motivation is to read into good documentations and create a self-contained collection of books,
+which can be understood without expecting too much background knowledge in every topic.
+** User Handbook:
+The content of the User book should be mostly concerned with our current and future graphical (gtk
+as well as terminal user interface) applications. After reading Preface and maybe Philosophy, the
+person reading the User Handbook should understand with the least possible strugle the application
+they intend to use. Examples should be given and concepts explained.
+** Installation Handbook:
+As seen with requests on the mailinglist, we will have to pick up people where they are, similar
+to the User Handbook. People already used to compiling and installing should have the chance to
+skip to the end, everyone else should: have step-by-step instructions, which will either already
+include OS specific notes or will be followed by OS specific instructions. It is up for discussion
+if configuring GNUnet is 'User Handbook' or 'Installation Handbook', the current mixture in
+the Installation Handbook is not good.
+** Contributors Handbook:
+This chapter could either be reduced to a couple of sections following the theme of 'contributing
+to GNUnet' or the chapter could be extended. If we extend it, we should explain a range of topics
+that can be useful for contributors. It can be understood as a recommended reading in addition to
+the Developer Handbook then, and the Developer Handbook could simply become a better commented
+reference for the code-base.
+** Developer Handbook:
+As outlined in the last sentences, the Developer Handbook could be reduced to the necessary bits
+with enough comments to be understood without reading into the papers published over the years.
+
+
+* DONE 1. Drupal books export to LaTeX.
+* DONE 2. LaTeX conversion to Texinfo.
+* DONE 3. (initial) Fixup of syntax errors introduced in conversion chain.
+* TODO 4. Update content.
+* TODO 5. Create API Reference or similar
+* TODO 6. Create Concept Index
+* TODO 7. Create Procedure Index
+* TODO 8. Create Type Index
+* TODO 9. Create Functions Index
+* TODO 10. Properly address concerns and criticism people had/have on the old and current documentation.
+* TODO 11. Reorder structure
+* TODO more TODO.
+
+
+* Status Progress / Completion Levels
+
+** chapters/philosophy: around 100% fixed after initial export.
+
+* System Integration Tasks
+
+* Which Texlive modules are needed for every output we generate?
+* Generate the images from their dot sources.
+
+* How to use (hack) on this
+
+This section will find its way into the documentation sooner or later.
+Potentially outdated or inaccurate bits.
+
+** with guix
+
+Adjust accordingly, ie read the Guix Documentation:
+guix environment gnunet-doc
+and
+guix build -f contrib/packages/guix/gnunet-doc.scm
+
+** without guix
+
+You need to have Texinfo and Texlive in your path.
+sh bootstrap
+./configure --enable-documentation
+cd doc
+make (format you want)
+
+for example: make html, make info, make pdf
+
+* structure (relations) (old!)
+
+** gnunet.texi
+ -> chapters/developer.texi
+ -> chapters/installation.texi
+ -> chapters/philosophy.texi
+ -> chapters/user.texi
+ -> chapters/vocabulary.texi
+ -> images/*
+ -> gpl-3.0.texi
+ -> fdl-1.3.texi
+
+** gnunet-c-tutorial.texi
+ -> figs/Service.pdf
+ -> figs/System.pdf
+ -> tutorial-examples/*.c
+ -> gpl-3.0.texi
+ -> fdl-1.3.texi
+
+- gnunet-c-tutorial-v1.pdf: original LaTeX "gnunet-c-tutorial.pdf".
+- man folder: the man pages.
+- doxygen folder
+- outdated-and-old-installation-instructions.txt: self described within the file.
+
+
+Use `gendocs', add to the manual/ directory of the web site.
+
+  $ cd doc
+  $ gendocs.sh gnunet "GNUnet 0.10.X Reference Manual"
diff --git a/doc/documentation/chapters/installation.texi b/doc/documentation/chapters/installation.texi
index f2042033e..665f980be 100644
--- a/doc/documentation/chapters/installation.texi
+++ b/doc/documentation/chapters/installation.texi
@@ -19,6 +19,7 @@ it in the form of new chapters or insightful comments.
 * Build instructions for Debian 7.5::
 * Installing GNUnet from Git on Ubuntu 14.4::
 * Build instructions for Debian 8::
+* Build instructions for macOS::
 @c * Build instructions for OpenBSD 6.2::
 * Outdated build instructions for previous revisions::
 @c * Portable GNUnet::
@@ -1472,6 +1473,59 @@ with the default Sqlite database. Sqlite is usually fine for most
 applications, but MySQL can offer better performance and Postgres better
 resillience.
 
+@node Build instructions for macOS
+@section Build instructions for macOS
+@c FIXME: I -> we
+
+These are the installation guidelines for macOS.
+They were tested on macOS High Sierra.
+
+@menu
+* Installing dependencies::
+* Compile from Source::
+@end menu
+
+@node Installing dependencies
+@subsection Installing dependencies
+
+First, install XCode in the newest version.
+See https://developer.apple.com/xcode/.
+
+Install Homebrew (https://brew.sh) and then install the dependencies listed above.
+If a dependency does not exists in brew, you need to compile it from source.
+
+@example
+# brew install <dependency>
+@end example
+
+@node Compile from Source
+@subsection Compile from Source
+
+Before you start building GNUnet, you need to setup your environment.
+This means that you have to make sure the proper tools are used in the build process.
+For example, after installing texinfo you need to make sure the new texinfo is actually used:
+
+@example
+# echo 'export PATH="/usr/local/opt/texinfo/bin:$PATH"' >> ~/.bash_profile 
+@end example
+
+Note: brew tells you the appropriate command when executing
+
+@example
+# brew info texinfo
+@end example
+
+This may also be necessary for the gettext package.
+
+Before you start compiling, you need to make sure gcc is used and not the clang compile of your macOS system.
+On my system, gcc was actually ``gcc-7'' and gcc pointed to the clang compiler.
+
+@example
+# export CC=gcc-7
+@end example
+
+After this the standard compile instructions apply.
+
 @c @node Build instructions for OpenBSD 6.2
 @c @section Build instructions for OpenBSD 6.2
 
diff --git a/doc/documentation/gnunet-c-tutorial.texi b/doc/documentation/gnunet-c-tutorial.texi
index 7eafa9ea9..0e2adaee7 100644
--- a/doc/documentation/gnunet-c-tutorial.texi
+++ b/doc/documentation/gnunet-c-tutorial.texi
@@ -10,7 +10,7 @@
 @include version2.texi
 
 @copying
-Copyright @copyright{} 2001-2017 GNUnet e.V.
+Copyright @copyright{} 2001-2018 GNUnet e.V.
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -68,9 +68,10 @@ dependencies can be found on our website at
 Reference Documentation (GNUnet Handbook).
 
 Please read this tutorial carefully since every single step is
-important and do not hesitate to contact the GNUnet team if you have
-any questions or problems! Check here how to contact the GNUnet
-team: @uref{https://gnunet.org/contact_information}
+important, and do not hesitate to contact the GNUnet team if you have
+any questions or problems! Visit this link in your webbrowser to learn
+how to contact the GNUnet team:
+@uref{https://gnunet.org/contact_information}
 
 @menu
 
@@ -151,7 +152,7 @@ $ gpg --verify-files gnunet-@value{VERSION}.tar.gz.sig
 
 @noindent
 If this command fails because you do not have the required public key,
-then you need to run this command to import it:
+then you need to run the following command to import it:
 
 @example
 $ gpg --keyserver keys.gnupg.net --recv-keys 48426C7E
@@ -167,19 +168,22 @@ revoked}. You will get an error message stating that
 The next release of GNUnet will have a valid signature
 again. We are sorry for the inconvenience this causes.
 Another possible source you could use is our
-"gnunet" git repository which has mandatory signed commits
-by every developer.
+"gnunet" git repository which, since the change from SVN to git in 2016,
+has mandatory signed commits by every developer.
 
-Now you can extract the tarball and rename the resulting
-directory to @file{gnunet} which we will be using in the
-remainder of this document.
+After verifying the signature you can extract the tarball.
+The resulting directory will be renamed to @file{gnunet}, which we will
+be using in the remainder of this document to refer to the
+root of the source directory.
 
 @example
 $ tar xvzf gnunet-@value{VERSION}.tar.gz
 $ mv gnunet-@value{VERSION} gnunet
-$ cd gnunet
 @end example
 
+@c FIXME: This can be irritating for the reader - First we say git should
+@c be avoid unless it is really required, and then we write this
+@c paragraph:
 @noindent
 However, please note that stable versions can be very outdated.
 As a developer you are @b{strongly} encouraged to use the version
@@ -192,32 +196,40 @@ To successfully compile GNUnet, you need the tools to build GNUnet and
 the required dependencies. Please take a look at the
 GNUnet Reference Documentation
 (@pxref{Dependencies, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation})
-for a list of required dependencies
-and
+for a list of required dependencies and
 (@pxref{Generic installation instructions, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation})
 read its Installation chapter for specific instructions for
-your operating system.
+your Operating System.
 Please check the notes at the end of the configure process about
 required dependencies.
 
 For GNUnet bootstrapping support and the HTTP(S) plugin you should
 install @uref{https://gnunet.org/gnurl, libgnurl}.
 For the filesharing service you should install at least one of the
-datastore backends. MySQL, SQlite and PostgreSQL are supported.
+datastore backends (MySQL, SQlite and PostgreSQL are supported).
 
 @node Obtaining the latest version from Git
 @section Obtaining the latest version from Git
 
-The latest development version can obtained from our Git repository.
-To obtain the code you need to have @code{Git} installed, which is
-required for obtaining the repository via:
+The latest development version can be obtained from our Git repository.
+To get the code you need to have @code{Git} installed. Usually your
+Operating System package manager should provide a suitable distribution
+of git (otherwise check out Guix or Nix). If you are using an Operating
+System based on Debian's apt:
+
+@example
+$ sudo apt-get install git
+@end example
+
+This is required for obtaining the repository, which is achieved with
+the following command:
 
 @example
 $ git clone https://gnunet.org/git/gnunet
 @end example
 
 @noindent
-After cloning the repository you have to execute the @file{bootstrap}
+After cloning the repository, you have to execute the @file{bootstrap}
 script in the new directory:
 
 @example
@@ -275,6 +287,7 @@ you do not specifiy a prefix, GNUnet is installed in the directory
 to enable verbose logging by adding @code{--enable-logging=verbose}:
 
 @example
+$ export PREFIX=$HOME
 $ ./configure --prefix=$PREFIX --enable-logging
 $ make
 $ make install
@@ -303,11 +316,14 @@ binaries and run GNUnet's self check.
 
 @example
 $ which gnunet-arm
+$PREFIX/bin/gnunet-arm
 @end example
 
 @noindent
-should return $PREFIX/bin/gnunet-arm. It should be located in your
+should return $PREFIX/bin/gnunet-arm (where $PREFIX is the location
+you  have set earlier). It should be located in your
 GNUnet installation and the output should not be empty.
+
 If you see an output like:
 
 @example
@@ -318,9 +334,11 @@ $ which gnunet-arm
 check your PATH variable to ensure GNUnet's @file{bin} directory is
 included.
 
-GNUnet provides tests for all of its subcomponents. Run
+GNUnet provides tests for all of its subcomponents. Assuming you have
+successfully built GNUnet, run
 
 @example
+$ cd gnunet
 $ make check
 @end example
 
@@ -387,7 +405,7 @@ a mesh on top of a DHT).
 @c \end{figure}
 
 The main service implementation runs as a standalone process in the
-operating system and the client code runs as part of the client program,
+Operating System and the client code runs as part of the client program,
 so crashes of a client do not affect the service process or other clients.
 The service and the clients communicate via a message protocol to be
 defined and implemented by the programmer.
diff --git a/doc/hacks.el b/doc/hacks.el
deleted file mode 100644
index 9f271b3af..000000000
--- a/doc/hacks.el
+++ /dev/null
@@ -1,17 +0,0 @@
-;;;; hacks.el --- a few functions to help me work on the manual
-;;;; Jim Blandy <jimb@red-bean.com> --- October 1998
-;;;; -- imported from https://git.savannah.gnu.org/cgit/guile.git/tree/doc/hacks.el
-
-(defun jh-exemplify-region (start end)
-  (interactive "r")
-  (save-excursion
-    (save-restriction
-      (narrow-to-region start end)
-
-      ;; Texinfo doesn't handle tabs well.
-      (untabify (point-min) (point-max))
-
-      ;; Quote any characters special to texinfo.
-      (goto-char (point-min))
-      (while (re-search-forward "[{}@]" nil t)
-        (replace-match "@\\&")))))