path: root/doc/network-structures.tm

<TeXmacs|2.1>

<project|scheme-gnunet.tm>

<style|tmmanual>

<\body>
  <index|network structure><index|netstruct>The modules <scm|(gnu gnunet
  netstruct procedural)> and <scm|(gnu gnunet netstruct syntactic)> can be
  used for formatting messages to be sent over the network or to a
  service.<space|1em>The macros <scm|define-type><index|define-type> and
  <scm|structure/packed><index|structure/packed> can be used to define new
  structures, like this:

  <\scm-code>
    (define-type /:msg:nse:estimate/example

    \ \ (structure/packed

    \ \ \ (properties '((message-symbol msg:nse:estimate)))

    \ \ \ (synopsis "Network size estimate")

    \ \ \ (documentation "Some explanation")

    \ \ \ (field (header /:message-header))

    \ \ \ (field (size-estimate ieee-double/big)

    \ \ \ \ \ \ \ \ \ \ (synopsis "Timestamp for when the estimate was
    made"))))
  </scm-code>

  This example is taken from the <scm|(gnu gnunet nse struct)> module and
  oversimplified.<space|1em>All its components will gradually
  explained.<space|1em>First, what actually is a network
  structure?<space|1em>This question is ambigious, because \<#2018\>network
  structure\<#2019\> can refer to either the <with|font-shape|italic|value>
  or the <with|font-shape|italic|type><index|network type><label|network
  types>.<space|1em>The <with|font-shape|italic|value> is a sequence of
  octets, i.e., a sequence of numbers in the closed range
  0\U255.<space|1em>The <with|font-shape|italic|type> describes how the
  <with|font-shape|italic|value> is structured.

  As an example, consider figure <reference|networkstructex>.<space|1em>There,
  the value is <scm|0 12 1 65 64 51 238 123 71 27 58 149> and the type is
  <scm|/:msg:nse:estimate/example>.

  <big-figure|<tree|<scm|/:msg:nse:estimate/example>|<tree|<scm|header>
  <scm|/:message-header>|<tree|<scm|size> <scm|u16/big>|0
  12>|<tree|<scm|type> <scm|u16/big>|1 65>>|<tree|<scm|size-estimate>
  <scm|ieee-double/big>|64 51 238 123 71 27 58 149>>|A network structure,
  both <with|font-shape|italic|value> and
  <with|font-shape|italic|type>.<label|networkstructex>>

  This value has a <with|font-shape|italic|header> with a
  <with|font-shape|italic|size> \<#2018\>0 0 0 12\<#2019\> of type
  <scm|u32/big>, so the <with|font-shape|italic|size> is 12.<space|1em>The
  <with|font-shape|italic|header> also has a <with|font-shape|italic|type>
  \<#2018\>0 0 1 65\<#2019\> of type <scm|u32/big>, so the type is
  <math|256\<times\>1+65=321>.<space|1em>The value also has a
  <with|font-shape|italic|size estimate> of type
  <scm|ieee-double/big>.<space|1em>The octets <scm|64 51 238 123 71 27 58
  149>, \ interpreted as an IEEE double, form the number
  <math|19.93\<ldots\>>.

  <section|Documentation>

  A network structure can optionally have embedded
  documentation.<space|1em>More specifically, network structures can
  optionally have the <with|font-shape|italic|synopsis><index|synopsis>,
  <with|font-shape|italic|documentation><index|documentation> and
  <with|font-shape|italic|properties><index|properties> set.<space|1em>The
  <with|font-shape|italic|synopsis> is a short description of what the
  network structure represents, typically a one-liner.<space|1em>The
  <with|font-shape|italic|documentation> can be more detailed, explaining how
  the structure works, can be used, is used and should be used.<space|1em>The
  <with|font-shape|italic|properties> form a free-formed association list.

  The synopsis, documentation and properties can be set on structured created
  with <scm|structure/packed> and individual fields and can be accessed with
  the procedures <scm|documentation>, <scm|synopsis> and <scm|properties>.

  The following properties are defined in Scheme-GNUnet:

  <\description>
    <\item*>
      <\code>
        message-symbol
      </code>
    </item*>

    <index|message-symbol>For <with|font-shape|italic|message structures>,
    this is a list of the names of the <with|font-shape|italic|message types>
    (see <reference|sec:message type> Message type database) this structure
    can be used for \U most of the time, there's only a single such message
    type, but sometimes a single structure can be used for multiple message
    types.

    <item*|c-type>

    <label|c-type>The value is the name of the equivalent type in the C
    implementation, if any, as a symbol. This can be useful if you know the
    name of the C type but not the name of the Scheme type: in that case, you
    can do <shell|git grep -F GNUNET_MessageHeader> in a git checkout of the
    source code of Scheme-GNUnet to discover that Scheme-GNUnet's name for
    <cpp|GNUNET_MessageHeader> is <scm|/:message-header>.
  </description>

  <todo|TODO: it would be nice to use the properties and documentation to
  automatically create a form of documentation, with some cross-references to
  the C code>

  <section|Reading and writing>

  The procedures <scm|read%><index|read%>, <scm|set%!><index|set%!>,
  <scm|sizeof><index|sizeof> and <scm|select><index|select> from <scm|(gnu
  gnunet netstruct procedural)> or the like-named macros from <scm|(gnu
  gnunet netstruct syntactic)> can be used for reading and writing network
  structures.<space|1em>The macros from <scm|syntactic> behave like the
  procedures from <scm|procedural> with some optimisations at expansion
  time.<space|1em>The procedures will be demonstrated with the
  <scm|/:msg:nse:estimate/example> network structure defined previously.

  First, create a memory slice with <scm|make-slice/read-write> from
  <scm|(gnu gnunet utils bv-slice)>.<space|1em>The required size can be
  determinded with <scm|(sizeof /:msg:nse:estimate/example
  '())>.<space|1em>The role of <scm|'()> will be explained later.

  <\scm-code>
    (import (gnu gnunet netstruct procedural)

    \ \ \ \ \ \ \ \ (gnu gnunet netstruct syntactic)

    \ \ \ \ \ \ \ \ (gnu gnunet utils bv-slice)

    \ \ \ \ \ \ \ \ (gnu gnunet util struct))

    \;

    (define-type /:msg:nse:estimate/example

    \ \ (structure/packed

    \ \ \ (field (header /:message-header))

    \ \ \ (field (size-estimate ieee-double/big))))

    \;

    (define message

    \ \ (make-slice/read-write (sizeof /:msg:nse:estimate/example '())))
  </scm-code>

  The fields of <scm|message> can be set with <scm|(set%! netstruct '(field
  ...) slice value)>.<space|1em>The following code sets all the fields:

  <\scm-code>
    ;; This example will be simplified later!

    (set%! /:msg:nse:estimate/example '(header size) message

    \ \ \ \ \ \ \ (sizeof /:msg:nse:estimate/example '()))

    (set%! /:msg:nse:estimate/example '(header type) message 165)

    (set%! /:msg:nse:estimate/example '(size-estimate) message 19.2)
  </scm-code>

  Repeating the message type and slice can be repetitive, so <scm|(gnu gnunet
  netstruct syntactic)> has a macro <scm|construct> to avoid this:

  <\explain>
    <scm|(construct <var|type> #:tail (<var|tail> <var|space>) <var|clause>
    ...)><index|construct>
  <|explain>
    The keyword argument <scm|#:tail (<var|tail> <var|space>)> is optional \U
    <var|space> defaults to <scm|0>.

    Return a fresh read-write bytevector slice holding a <var|type> and
    <var|space> additional bytes. The slice is initialised by sequentally
    running <var|clause> <text-dots>. In the syntactic environment of the
    clauses, when present, <var|tail> is bound to a read-write bytevector
    slice for the <var|space> additional bytes, and in the environment the
    following macros are available:

    <\description>
      <item*|<scm|(%sizeof)><index|%sizeof>>The total size of the bytevector
      size, i.e. the size of the network structure <var|type> plus
      <var|space>.

      <item*|<scm|(=\<gtr\>! <var|field> <var|value>)><index|=\<gtr\>!>>Assign
      <var|value> to the field <var|field>. This is to be understood as a
      literal list, not as a procedure invocation \U neither the list nor
      <var|field> <text-dots> is evaluated.

      <item*|<scm|(=\<gtr\>slice! <var|field>
      <var|slice>)><index|=\<gtr\>slice!>>Copy the slice <var|slice> into the
      field <var|field>, where <scm|(field ...)> is interpreted the same way
      as for <scm|=\<gtr\>!>.
    </description>

    <var|type> can currently be evaluated multiple times. These macros
    <scm|%sizeof>, <scm|=\<gtr\>!><space|1em>and <scm|=\<gtr\>slice!> can
    only be used inside a <scm|construct> form; in other contexts an
    exception is raised.
  </explain>

  Using this macro, the previous example can be simplified to:

  <\scm-code>
    (import [...]) (define-type [...]) ; copy from previous example

    (define message

    \ \ (construct /:msg:nse:estimate/example

    \ \ \ \ (=\<gtr\>! (header size) (%sizeof))

    \ \ \ \ (=\<gtr\>! (header type) 165)

    \ \ \ \ (=\<gtr\>! (size-estimate) 19.2)))
  </scm-code>

  The size of an individual field can be determined with <scm|(sizeof
  netstruct '(field ...))>.<space|1em>For example, the following code
  determines the size of the \<#2018\>size\<#2019\> field in the header:

  <\scm-code>
    (sizeof /:msg:nse:estimate/example '(header size)) ; 12
  </scm-code>

  The fields can also be read:

  <\scm-code>
    ;; This example is simplified later!

    (read% /:msg:nse:estimate/example '(header size) message) ; 12

    (read% /:msg:nse:estimate/example '(header type) message) ; 165

    (read% /:msg:nse:estimate/example '(size-estimate) message) ; 19.2
  </scm-code>

  Repeating the message type and the slice can be repetitive, so <scm|(gnu
  gnunet netstruct syntactic)> has a macro to avoid that:

  <\explain>
    <scm|(analyse <var|type> <var|message> <var|body>
    <text-dots>)><index|analyse>
  <|explain>
    Expand to <scm|<var|body ...>> \ in a context where the syntax parameters
    <scm|r%> and <scm|s%> (from <scm|(gnu gnunet netstruct syntactic)>) are
    bound to macros with the following interface:

    <\description>
      <item*|<scm|(r% <var|field> ...)><index|r%>>Read the field
      <scm|(<var|field> ...)> of <var|message>. This is to be understood as a
      literal list, not as a procedure invocation \U neither the list nor
      <var|field> <text-dots> is evaluated.

      <item*|<scm|(s% <var|field> ...)><index|s%>>Select the field
      <scm|(field ...)> of <var|message>, where <scm|(field ...)> is
      interpreted the same way as for <scm|r%>.
    </description>

    <var|type> and <var|message> can currently be evaluated multiple times.
    These macros <scm|r%> and <scm|s%> can only be used inside a
    <scm|analyse> or <scm|define-analyser> construct; in other contexts an
    exception is raised.
  </explain>

  Using this macro, the previous example can be simplified to: '

  <\scm-code>
    ;; 'pk' is for printing the value

    (analyse /:msg:nse:estimate/example message

    \ \ \ \ \ \ \ \ \ (pk (r% header size)) ; 12

    \ \ \ \ \ \ \ \ \ (pk (r% header type)) ; 165

    \ \ \ \ \ \ \ \ \ (pk (r% size-estimate))) ; 19.2
  </scm-code>

  For very simple 'analysis' procedures, the macro
  <scm|define-analyser><index|define-analyser> can be useful:

  <\scm-code>
    ;; 'pk' is for printing the value

    (define-analyser analyse-example /:msg:nse:estimate/example

    \ \ "Put a docstring here"

    \ \ (pk (r% header size)) ; 12

    \ \ (pk (r% header type)) ; 165

    \ \ (pk (r% size-estimate)))

    (analyse-example [...])
  </scm-code>

  <section|Primitive types>

  There are a number of pre-defined types.<space|1em>First, there is
  <scm|u8>, a single octet that is interpreted as an integer in the closed
  range <math|<around*|[|0,255|]>>.<space|1em>There are also types
  <scm|uN/endian> for <math|N\<in\><around*|{|16,32,64|}>> and
  <math|endian\<in\><around*|{|little,big|}>>, which interprets <math|N/8>
  octets as integers in the closed range <math|<around*|[|0,2<rsup|N>-1|]>>.<space|1em>The
  types <scm|ieee-double/big> and <scm|ieee-double/little> are 8 octets long
  and represent floating-point numbers in IEEE 754 format
  (\<#2018\>binary64\<#2019\>).

  <section|Packing>

  In contrast to C structures, Scheme-GNUnet network structures are always
  packed \V there are no \<#2018\>gaps\<#2019\> between fields.
</body>

<\initial>
  <\collection>
    <associate|page-medium|paper>
    <associate|save-aux|false>
  </collection>
</initial>