commit 8dbbdd77feaa5180b81e7682d2fa9c201c9a4f98
parent 6418a0b3b5a0f985933ab3e1789777d5fa3d5f1f
Author: Christian Grothoff <christian@grothoff.org>
Date: Tue, 28 Jul 2026 22:47:16 +0200
add tutorial for #5657
Diffstat:
9 files changed, 2441 insertions(+), 5 deletions(-)
diff --git a/doc/Makefile.am b/doc/Makefile.am
@@ -22,6 +22,7 @@ microhttpd_TEXINFOS = \
gpl-2.0.texi \
fdl-1.3.texi
microhttpd_tutorial_TEXINFOS = \
+ chapters/asyncresponses.inc \
chapters/basicauthentication.inc \
chapters/bibliography.inc \
chapters/callbackresponse.inc \
diff --git a/doc/chapters/asyncresponses.inc b/doc/chapters/asyncresponses.inc
@@ -0,0 +1,501 @@
+The previous chapter left one question open. A content reader that
+has no data ready may return @code{0}, but doing so turns the daemon
+into a busy loop, and the honest answer---suspend the connection---was
+only sketched there. This chapter builds the thing itself: a server
+whose slow work happens in threads of its own, whose connections are
+parked while that work is under way, and which is driven by an
+external @code{select()} loop so that the application stays in charge
+of its own event handling.
+
+The combination is a common one. Most requests are cheap and can be
+answered on the spot; a few of them start a report, a backup or a
+query that runs for seconds. Handling the slow ones inline would
+block every other client for as long as they take. Giving every
+connection its own thread---@code{MHD_USE_THREAD_PER_CONNECTION}---is
+no answer either, both because it scales badly and because suspending
+is not available in that mode at all. What is wanted is a single
+event loop that never blocks, plus a worker per slow job.
+
+
+@heading Who may call what
+
+Almost every difficulty in this design comes from one rule, so it is
+worth stating before any code:
+
+@table @asis
+@item @code{MHD_queue_response()}
+Must be called from the thread that runs the daemon, from inside the
+access handler. A worker thread must never queue a response.
+
+@item @code{MHD_suspend_connection()}
+May only be called from the access handler or from a content reader
+callback---that is, from @emph{MHD}'s own thread, while @emph{MHD} is
+calling into the application.
+
+@item @code{MHD_resume_connection()}
+May be called from any thread at any time, and this is the exception
+that makes the whole design work. It is the only @emph{MHD} function
+a worker thread ever needs.
+@end table
+
+@noindent
+So the split is: the worker computes and, when it has something,
+resumes. Everything that touches the connection or the response
+happens on the daemon's thread, where @emph{MHD} calls the application
+back. The worker and the daemon share exactly one object, a
+per-request context, and the only thing the worker does with the
+connection handle it holds is pass it to
+@code{MHD_resume_connection()}.
+
+The daemon has to be started with @code{MHD_ALLOW_SUSPEND_RESUME}; both
+suspend and resume call @code{MHD_PANIC()} and abort the process
+otherwise. Note also that suspending does not work with a thread pool:
+@code{MHD_resume_connection()} asserts that the daemon has none.
+
+
+@heading The event loop
+
+Leaving out @code{MHD_USE_INTERNAL_POLLING_THREAD} puts @emph{MHD}
+into external polling mode: it starts no thread of its own and does
+nothing until the application asks it to. The loop is the usual one,
+and the example adds its own descriptor to the very same sets---there
+is nothing special about @emph{MHD}'s:
+
+@verbatim
+ FD_ZERO (&rs); FD_ZERO (&ws); FD_ZERO (&es);
+ max = 0;
+ if (MHD_YES != MHD_get_fdset (daemon, &rs, &ws, &es, &max))
+ break; /* fatal internal error */
+ FD_SET (wake_pipe[0], &rs); /* our own wakeup pipe */
+ if (max < wake_pipe[0])
+ max = wake_pipe[0];
+
+ if (MHD_YES == MHD_get_timeout64 (daemon, &mhd_timeout))
+ { ...fill tv...; tvp = &tv; }
+ else
+ tvp = NULL; /* nothing to wait for */
+
+ select ((int) max + 1, &rs, &ws, &es, tvp);
+ MHD_run_from_select (daemon, &rs, &ws, &es);
+@end verbatim
+@noindent
+
+The interesting branch is the one that sets @code{tvp} to @code{NULL}.
+@code{MHD_get_timeout64()} returns @code{MHD_NO} when @emph{MHD} has
+nothing whatsoever to wait for, and with every connection suspended
+that is precisely the situation: suspended connections do not time
+out, so there is no deadline to wake up for. The loop then blocks in
+@code{select()} indefinitely. Whether the server works at all depends
+on something waking it up.
+
+
+@heading Waking the loop up
+
+That something is @emph{MHD}'s inter-thread communication channel.
+@code{MHD_ALLOW_SUSPEND_RESUME} is defined as
+@code{8192 | MHD_USE_ITC}, so requesting suspend support requests the
+ITC as well; @code{MHD_get_fdset()} puts the reading end of that
+channel into the read set, ahead of everything else; and
+@code{MHD_resume_connection()} writes a byte to it. A worker thread
+that resumes a connection therefore breaks the main thread out of
+@code{select()}, even though the two share nothing but the daemon
+handle.
+
+This deserves emphasis because the header file is misleading about it:
+the comment on @code{MHD_USE_ITC} says the flag is ignored with
+external polling. That is true for the flag in isolation, but not for
+the channel: @code{MHD_ALLOW_SUSPEND_RESUME} creates it regardless of
+the polling mode, and in external mode it is the application's own
+@code{select()} that watches it. Without that, a resumed connection
+would sit untouched until some unrelated event happened to wake the
+loop.
+
+There is no race to guard against here. If the worker resumes in the
+window between @code{MHD_get_fdset()} and @code{select()}, the byte is
+already in the channel when @code{select()} is entered, and
+@code{select()} returns immediately---the channel is level triggered,
+so a wakeup cannot be missed. The same property makes it safe for a
+worker to resume a connection that has not finished suspending yet:
+@emph{MHD} remembers the request and cancels the pending suspension
+instead of parking the connection. Neither side needs to know what
+the other is doing.
+
+
+@heading Answering in one piece
+
+The simplest of the two patterns is for an answer that is only useful
+as a whole---a generated report, the result of a query. Nothing can be
+sent before it is finished, so the connection is suspended in the
+access handler and the response is queued when @emph{MHD} comes back:
+
+@verbatim
+static enum MHD_Result
+handle_slow (struct MHD_Connection *connection, void **req_cls)
+{
+ struct Job *job = *req_cls;
+
+ if (NULL == job)
+ {
+ /* First call for this request: start the work and ask MHD to
+ come back to us. */
+ job = job_create (connection, 0);
+ if (NULL == job)
+ return MHD_NO;
+ if (MHD_NO == job_start (job))
+ { job_unref (job); job_unref (job); return MHD_NO; }
+ *req_cls = job;
+ return MHD_YES;
+ }
+ pthread_mutex_lock (&job->lock);
+ if (0 == job->finished)
+ {
+ job->suspended = 1;
+ MHD_suspend_connection (connection);
+ pthread_mutex_unlock (&job->lock);
+ return MHD_YES;
+ }
+ ...copy the result out from under the lock...
+ pthread_mutex_unlock (&job->lock);
+ ...MHD_create_response_from_buffer_copy() and MHD_queue_response()...
+}
+@end verbatim
+@noindent
+
+The control flow is worth following, because it is not obvious that it
+terminates. @emph{MHD} calls the access handler for a request more
+than once; the first call is the one where @code{*req_cls} is still
+@code{NULL}, and returning @code{MHD_YES} without queueing anything
+tells @emph{MHD} to carry on. On the second call the worker is
+running but not finished, so the connection is suspended and the
+handler returns again. When the worker resumes the connection,
+@emph{MHD} calls the handler a third time---on its own thread, with
+the same @code{*req_cls}---and this time @code{finished} is set and the
+response is queued.
+
+That third call is the answer to the question this pattern exists for.
+The worker never queues anything; it merely makes the daemon call the
+application back at a moment when the answer is ready.
+
+
+@heading Answering as it is produced
+
+The second pattern is the more useful one when the answer is long, and
+it is what makes a browser show results as they appear rather than
+after a long blank pause. Here the response is queued
+@emph{immediately}, with a content reader and an unknown size, so the
+headers go out at once and the body follows as the worker fills it in:
+
+@verbatim
+ job = job_create (connection, 1);
+ response = MHD_create_response_from_callback (MHD_SIZE_UNKNOWN,
+ IO_BLOCK_SIZE,
+ &stream_reader,
+ job,
+ &stream_done);
+ ...
+ MHD_add_response_header (response, MHD_HTTP_HEADER_CONTENT_TYPE,
+ "text/event-stream");
+ ret = MHD_queue_response (connection, MHD_HTTP_OK, response);
+ MHD_destroy_response (response);
+@end verbatim
+@noindent
+
+Because the size is unknown, @emph{MHD} uses chunked transfer encoding,
+and every piece the content reader hands back is framed as its own
+chunk and goes out on the wire straight away. The content reader is
+where suspending happens:
+
+@verbatim
+static ssize_t
+stream_reader (void *cls, uint64_t pos, char *buf, size_t max)
+{
+ struct Job *job = cls;
+ size_t ready;
+
+ pthread_mutex_lock (&job->lock);
+ if (job->off == job->fill)
+ {
+ job->off = 0;
+ job->fill = 0;
+ if (0 != job->finished)
+ {
+ pthread_mutex_unlock (&job->lock);
+ return MHD_CONTENT_READER_END_OF_STREAM;
+ }
+ /* Nothing to send yet. Take the connection out of the event
+ loop; the worker will put it back in. */
+ job->suspended = 1;
+ MHD_suspend_connection (job->connection);
+ pthread_mutex_unlock (&job->lock);
+ return 0;
+ }
+ ready = job->fill - job->off;
+ if (ready > max)
+ ready = max;
+ memcpy (buf, &job->payload[job->off], ready);
+ job->off += ready;
+ pthread_mutex_unlock (&job->lock);
+ return (ssize_t) ready;
+}
+@end verbatim
+@noindent
+
+The return value is still @code{0}, exactly as in the busy-waiting
+version the previous chapter warned about. The difference is the line
+above it. Suspending removes the connection from the event loop, so
+@emph{MHD} has no reason to come back until the worker says so, and
+the process goes to sleep. The example ships with a test that measures
+this: streaming a document that takes one second to produce costs the
+server no measurable CPU time at all while the connection is suspended,
+and very nearly a full second---one core, saturated---when the
+@code{MHD_suspend_connection()} call is removed and only the
+@code{return 0} is left. The bytes that arrive are identical in both
+cases, which is what makes this such an easy mistake to keep.
+
+
+@heading The shared state
+
+Everything the two threads share sits in one structure per request:
+
+@verbatim
+struct Job
+{
+ pthread_mutex_t lock;
+ pthread_cond_t cond;
+ struct MHD_Connection *connection;
+ char payload[MAX_PAYLOAD];
+ size_t fill;
+ size_t off;
+ unsigned int rc;
+ int finished;
+ int suspended;
+ int abandoned;
+ ...
+};
+@end verbatim
+@noindent
+
+There is no registry of pending requests involved in answering them,
+and no lookup: the pointer travels in @code{*req_cls} for the first
+pattern and in the content reader's @code{crc_cls} for the second, and
+both the daemon and the worker are handed it directly. A shared list
+of all jobs is still useful, but only for shutting down in an orderly
+fashion, and it is protected by a lock of its own that is always taken
+before any job's lock, never after.
+
+The structure is reference counted, with one reference held by
+@emph{MHD} and one by the worker. The @emph{MHD} side lets go in the
+free callback of the response for a streamed answer, and in the
+@code{MHD_OPTION_NOTIFY_COMPLETED} callback for the other pattern; the
+worker lets go when it returns. Dropping the last reference does not
+free the job, though, because the worker still has to be joined and
+only the main thread may do that. The main loop therefore ends with a
+sweep that joins and frees whatever has no references left.
+
+
+@heading When the client goes away
+
+This is the part that is easy to get wrong, because it does not happen
+during development and does happen constantly in production. A client
+that closes the connection halfway through a download leaves a worker
+running with a pointer to a connection @emph{MHD} is about to free.
+
+The two callbacks that tell the application a request is over---the
+response's free callback and @code{MHD_OPTION_NOTIFY_COMPLETED}---are
+therefore where the worker is cut loose:
+
+@verbatim
+static void
+job_detach (struct Job *job)
+{
+ pthread_mutex_lock (&job->lock);
+ job->abandoned = 1;
+ job->connection = NULL;
+ pthread_cond_signal (&job->cond);
+ pthread_mutex_unlock (&job->lock);
+ job_unref (job);
+}
+@end verbatim
+@noindent
+
+and the worker checks that flag before every resume:
+
+@verbatim
+static void
+job_resume_locked (struct Job *job)
+{
+ if ( (0 == job->suspended) || (0 != job->abandoned) )
+ return;
+ job->suspended = 0;
+ MHD_resume_connection (job->connection);
+}
+@end verbatim
+@noindent
+
+Note that this function is called with the job's lock @emph{held}, and
+calls into @emph{MHD} without releasing it. That is deliberate and it
+is what makes the pattern safe. @emph{MHD} destroys the connection
+only after the completion callback has returned, and that callback
+cannot run while the worker holds the lock; so for as long as the
+worker is inside @code{MHD_resume_connection()}, the connection it
+passes is guaranteed to still exist. Dropping the lock first and
+resuming afterwards would open exactly the window this is meant to
+close.
+
+The reverse order never occurs: @emph{MHD} invokes the free and
+completion callbacks without holding the internal locks that
+@code{MHD_resume_connection()} acquires, so there is no way for the
+two to deadlock.
+
+The condition variable in the same function is a convenience rather
+than a necessity. Signalling it wakes a worker that is asleep between
+steps, so that the work stops within microseconds of the client
+disappearing instead of running to completion for nobody.
+
+
+@heading Shutting down
+
+Stopping a daemon that still has suspended connections is an API
+violation, and a suspended connection is one that only a worker can
+release. The order is therefore fixed:
+
+@enumerate
+@item
+Stop handing out new jobs.
+
+@item
+Tell every worker to give up, and @emph{join} them. After this point
+no connection can be suspended any more: the workers resume before
+they exit, and the content reader only ever suspends while a worker is
+running.
+
+@item
+Keep running the event loop until the connections have drained, so
+that the last bytes actually reach the clients.
+
+@item
+@code{MHD_stop_daemon()}.
+@end enumerate
+
+@noindent
+Skipping the join in step two is the tempting shortcut, and it is the
+one that leaves a connection parked forever with nothing left alive to
+wake it.
+
+
+@heading Example code
+
+The complete program is available as @code{asyncresponse.c}. It runs
+one external @code{select()} loop and serves four URLs:
+
+@table @code
+@item /
+A small page whose JavaScript opens the stream below and appends a row
+for every event as it arrives. The rows visibly trickle in, one per
+step, which is the most direct demonstration of what the chapter is
+about.
+
+@item /events
+The stream: a response is queued at once, and the content reader
+suspends between steps. The body is a sequence of server-sent events,
+which needs no library on the browser side.
+
+@item /slow
+The other pattern: the connection is suspended in the access handler
+and a complete answer is queued once the worker is done.
+
+@item /fast
+Answered on the spot. The page has a button that requests it and
+reports the round trip time; it stays in the low milliseconds while
+jobs are running, which is the point of suspending rather than
+blocking.
+@end table
+
+@noindent
+The program takes the port, the duration of a step in milliseconds and
+the number of steps on the command line. Passing @code{0} as the port
+makes the operating system choose a free one, which the program then
+reports via @code{MHD_DAEMON_INFO_BIND_PORT}:
+
+@verbatim
+$ ./asyncresponse 0 250 20
+Listening on port 44321
+@end verbatim
+@noindent
+
+The accompanying @code{test_asyncresponse.c} starts exactly that
+binary and checks over HTTP that @code{/slow} waits for its worker,
+that the body of @code{/events} really arrives in pieces spread over
+the lifetime of the request rather than in one burst at the end, that
+@code{/fast} is still served promptly while jobs are parked, that a
+client which hangs up in the middle takes neither the server nor the
+worker with it, and that the process exits cleanly when it is signalled
+while a connection is suspended. It also compares the server's CPU
+time against the wall clock time of a transfer, which is the only one
+of those checks that notices the difference between a suspended
+connection and a busy-waiting one.
+
+
+@heading Remarks
+
+@emph{MHD} does not detect that a client has disconnected while a
+connection is suspended, and the connection timeout does not run
+either. A suspended connection stays suspended until somebody resumes
+it, without exception. Any timeout on the work itself is therefore the
+application's responsibility.
+
+Suspended connections still count against the connection limits, both
+the global one and the per-IP one. A server that parks connections for
+minutes needs its limits sized for the number of jobs it expects to
+have in flight, not for the number it expects to be transmitting.
+
+If the client vanishes in the middle of a chunked response, the next
+write fails and @emph{MHD} logs it---with
+@code{MHD_USE_ERROR_LOG} enabled the example prints @code{Failed to
+send the chunked response body ... The socket is no longer available
+for sending}. That is the normal course of events for an aborted
+download and not a sign of a problem.
+
+Finally, none of this is specific to @code{select()}. The same
+argument holds for @code{poll()} and for @code{epoll}, where the
+daemon's descriptor can be obtained with
+@code{MHD_DAEMON_INFO_EPOLL_FD} and added to an epoll set of the
+application's own; @code{suspend_resume_epoll.c} among the
+distribution's examples shows that variant.
+
+
+@heading Exercises
+
+@itemize @bullet
+
+@item
+Give the jobs a deadline. Since a suspended connection never times
+out on its own, add a check to the main loop that resumes---and
+answers with @code{503}---any job that has been running for too long,
+and confirm with a step duration long enough to trigger it.
+
+@item
+Remove the @code{MHD_suspend_connection()} call from the content
+reader, leaving only the @code{return 0}, and watch the server process
+with @code{top} while a single client downloads the stream. Then put
+it back and watch again. Both servers send the same bytes.
+
+@item
+Take the resume out of the worker's final block, so that a finished job
+leaves its connection parked, and then stop the server while a job is
+in flight. Watch what the shutdown does---and what @code{valgrind}
+has to say about it---and work out which of the four steps above was
+violated.
+
+@item
+Replace the per-request thread with a fixed pool of worker threads and
+a queue of pending jobs. The reference counting and the
+@code{abandoned} flag should not need to change at all; if they do, the
+ownership rules were not as clean as they looked.
+
+@item
+Serve the stream to two browser windows at once and confirm from the
+timestamps that the two jobs run concurrently and are interleaved by a
+single-threaded event loop.
+
+@end itemize
diff --git a/doc/chapters/callbackresponse.inc b/doc/chapters/callbackresponse.inc
@@ -367,6 +367,12 @@ million times and burns a full core; the same callback suspending the
connection is entered exactly once more and the daemon uses no
measurable CPU time at all.
+Suspending brings requirements of its own---on who may call what from
+which thread, on how a worker thread wakes up a blocked event loop, and
+on who releases the shared state when a client disappears halfway
+through. @ref{Answering from another thread} works all of that out on
+a complete program.
+
@heading Trailers
@@ -456,7 +462,10 @@ handle in the context struct, and suspend the connection whenever the
buffer is empty. Do not forget @code{MHD_ALLOW_SUSPEND_RESUME} when
starting the daemon. Then remove the call to
@code{MHD_suspend_connection()}, leaving only the @code{return 0}, and
-compare the CPU usage of the server process.
+compare the CPU usage of the server process. A complete solution,
+together with the ownership rules that a shared buffer between a
+worker and the event loop needs, is the subject of
+@ref{Answering from another thread}.
@item
Compute a checksum over the generated document while it is being sent
diff --git a/doc/examples/.gitignore b/doc/examples/.gitignore
@@ -1,3 +1,5 @@
+/asyncresponse
+/test_asyncresponse
/upgrade
/tlsauthentication
/simplepost
@@ -12,3 +14,6 @@
*.o
*.lo
*.la
+# Produced by the automake test harness
+*.log
+*.trs
diff --git a/doc/examples/Makefile.am b/doc/examples/Makefile.am
@@ -3,7 +3,7 @@ SUBDIRS = .
AM_CPPFLAGS = \
-I$(top_srcdir)/src/include \
- $(CPPFLAGS_ac)
+ $(CPPFLAGS_ac) $(LIBCURL_CPPFLAGS)
AM_CFLAGS = $(CFLAGS_ac) @LIBGCRYPT_CFLAGS@
@@ -41,17 +41,44 @@ endif
endif
if HAVE_POSTPROCESSOR
-noinst_PROGRAMS += simplepost largepost sessions
+noinst_PROGRAMS += simplepost largepost sessions
+endif
+
+if HAVE_POSIX_THREADS
+noinst_PROGRAMS += asyncresponse
+if HAVE_FORK_WAITPID
+if RUN_LIBCURL_TESTS
+check_PROGRAMS = test_asyncresponse
+TESTS = $(check_PROGRAMS)
+endif
+endif
endif
if HAVE_W32
AM_CPPFLAGS += -DWINDOWS
endif
+asyncresponse_SOURCES = \
+ asyncresponse.c
+asyncresponse_CFLAGS = \
+ $(AM_CFLAGS) $(PTHREAD_CFLAGS)
+asyncresponse_LDADD = \
+ $(PTHREAD_LIBS) \
+ $(top_builddir)/src/microhttpd/libmicrohttpd.la
+
+test_asyncresponse_SOURCES = \
+ test_asyncresponse.c
+test_asyncresponse_CFLAGS = \
+ $(AM_CFLAGS) $(PTHREAD_CFLAGS)
+test_asyncresponse_LDADD = \
+ $(PTHREAD_LIBS) \
+ $(top_builddir)/src/microhttpd/libmicrohttpd.la \
+ @LIBCURL@
+
basicauthentication_SOURCES = \
- basicauthentication.c
+ basicauthentication.c
basicauthentication_LDADD = \
- $(top_builddir)/src/microhttpd/libmicrohttpd.la
+ $(top_builddir)/src/microhttpd/libmicrohttpd.la
callbackresponse_SOURCES = \
callbackresponse.c
diff --git a/doc/examples/asyncresponse.c b/doc/examples/asyncresponse.c
@@ -0,0 +1,1193 @@
+/* Feel free to use this example code in any way
+ you see fit (Public Domain) */
+
+/**
+ * @file asyncresponse.c
+ * @brief Example for answering a request from another thread while
+ * the daemon is driven by an external select() loop. The
+ * main thread owns libmicrohttpd and does nothing but poll;
+ * every slow request gets a worker thread of its own and the
+ * connection is suspended for as long as that worker has
+ * nothing to say.
+ *
+ * "GET /" serves a small page whose JavaScript renders the
+ * data as it trickles in, "GET /events" is the stream behind
+ * it (a response is queued immediately and the content reader
+ * suspends between chunks), "GET /slow" suspends in the access
+ * handler until a complete answer is ready, and "GET /fast" is
+ * answered on the spot so that it can be used to show that the
+ * event loop never blocks.
+ *
+ * This example needs POSIX threads.
+ * @author Christian Grothoff
+ */
+
+#include <sys/types.h>
+#include <sys/select.h>
+#include <sys/socket.h>
+#include <microhttpd.h>
+#include <errno.h>
+#include <pthread.h>
+#include <signal.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+
+/**
+ * Port to listen on if none was given on the command line. Pass 0 to
+ * let the operating system pick a free port; the port that was
+ * actually bound is then printed on standard output.
+ */
+#define DEFAULT_PORT 8888
+
+/**
+ * Time the worker thread pretends to need for one step, in
+ * milliseconds.
+ */
+#define DEFAULT_STEP_MS 250
+
+/**
+ * Number of steps that make up one job.
+ */
+#define DEFAULT_STEPS 20
+
+/**
+ * Size of the buffer in which a job collects the bytes that the
+ * content reader has not picked up yet.
+ */
+#define MAX_PAYLOAD 4096
+
+/**
+ * Block size we ask MHD to use when it queries our content reader.
+ */
+#define IO_BLOCK_SIZE 1024
+
+/**
+ * How long we are willing to wait for the last connections to go away
+ * when the server is shutting down, in milliseconds.
+ */
+#define DRAIN_TIMEOUT_MS 2000
+
+
+/**
+ * The front page. Its JavaScript opens the "/events" stream and adds
+ * a row for every server-sent event as it arrives, so that the delay
+ * between the steps is plainly visible. The "ping" button fires a
+ * request against "/fast" and reports the round trip time; it stays in
+ * the low milliseconds even while several jobs are running, which is
+ * the whole point of suspending instead of blocking.
+ */
+static const char *PAGE =
+ "<!DOCTYPE html>\n"
+ "<html lang='en'>\n"
+ "<head>\n"
+ "<meta charset='utf-8'>\n"
+ "<title>libmicrohttpd: answering from another thread</title>\n"
+ "<style>\n"
+ "body { font-family: sans-serif; max-width: 40em; margin: 2em auto; }\n"
+ "#bar { height: 1em; border: 1px solid #888; margin: 1em 0; }\n"
+ "#fill { height: 100%; width: 0; background: #4a90d9; transition: width .2s; }\n"
+ "#log { font-family: monospace; font-size: 90%; list-style: none; padding: 0; }\n"
+ "#log li { border-bottom: 1px solid #eee; padding: 2px 0; }\n"
+ "button { margin-right: .5em; }\n"
+ "</style>\n"
+ "</head>\n"
+ "<body>\n"
+ "<h1>Answering from another thread</h1>\n"
+ "<p>The server runs a single external <code>select()</code> loop. The job\n"
+ "below is computed by a thread of its own, and the connection carrying it is\n"
+ "suspended whenever that thread has nothing to say. Nothing is buffered: each\n"
+ "row appears the moment the worker produced it.</p>\n"
+ "<button id='go'>Start a job</button>\n"
+ "<button id='ping'>Ping /fast</button>\n"
+ "<div id='bar'><div id='fill'></div></div>\n"
+ "<p id='status'>idle</p>\n"
+ "<ul id='log'></ul>\n"
+ "<script>\n"
+ "const $ = (id) => document.getElementById(id);\n"
+ "let es = null;\n"
+ "let t0 = 0;\n"
+ "const stop = () => { if (es) { es.close(); es = null; } };\n"
+ "$('go').onclick = () => {\n"
+ " stop ();\n"
+ " $('log').innerHTML = '';\n"
+ " $('fill').style.width = '0';\n"
+ " $('status').textContent = 'running';\n"
+ " t0 = performance.now ();\n"
+ " es = new EventSource ('/events');\n"
+ " es.addEventListener ('step', (e) => {\n"
+ " const d = JSON.parse (e.data);\n"
+ " const li = document.createElement ('li');\n"
+ " li.textContent = '+' + Math.round (performance.now () - t0) +\n"
+ " ' ms ' + d.label;\n"
+ " $('log').appendChild (li);\n"
+ " $('fill').style.width = (100 * d.n / d.total) + '%';\n"
+ " });\n"
+ " es.addEventListener ('done', () => {\n"
+ " stop ();\n"
+ " $('status').textContent = 'done after ' +\n"
+ " Math.round (performance.now () - t0) + ' ms';\n"
+ " });\n"
+ " es.onerror = () => { stop (); $('status').textContent = 'connection lost'; };\n"
+ "};\n"
+ "$('ping').onclick = async () => {\n"
+ " const t = performance.now ();\n"
+ " await fetch ('/fast', { cache: 'no-store' });\n"
+ " $('status').textContent = '/fast answered in ' +\n"
+ " Math.round (performance.now () - t) + ' ms';\n"
+ "};\n"
+ "</script>\n"
+ "</body>\n"
+ "</html>\n";
+
+
+/**
+ * State shared between the thread that runs libmicrohttpd and the
+ * worker thread that produces the answer. Everything below @e lock is
+ * protected by it.
+ *
+ * The structure is reference counted, with one reference held by MHD
+ * (dropped in #stream_done() or #request_completed()) and one held by
+ * the worker (dropped when the worker function returns). Reaching zero
+ * does not free the job: only the main thread does that, in
+ * #reap_jobs(), because it is also the thread that has to join the
+ * worker.
+ */
+struct Job
+{
+ /**
+ * Kept in a doubly linked list of all jobs, so that the shutdown
+ * code can reach every worker. The list is protected by
+ * #jobs_lock, never by @e lock.
+ */
+ struct Job *next;
+
+ /**
+ * See @e next.
+ */
+ struct Job *prev;
+
+ /**
+ * Protects every field below, and---just as importantly---keeps the
+ * connection alive across #MHD_resume_connection().
+ */
+ pthread_mutex_t lock;
+
+ /**
+ * Used to wake the worker out of its sleep when the client goes away
+ * or the server is shutting down.
+ */
+ pthread_cond_t cond;
+
+ /**
+ * The worker thread. Only touched by the main thread, under
+ * #jobs_lock.
+ */
+ pthread_t tid;
+
+ /**
+ * The connection this job answers. Set to NULL by the MHD side as
+ * soon as MHD is done with the connection; the worker must not touch
+ * it after that, which is why @e abandoned is checked under @e lock
+ * before every #MHD_resume_connection().
+ */
+ struct MHD_Connection *connection;
+
+ /**
+ * Bytes produced by the worker that the content reader has not
+ * picked up yet.
+ */
+ char payload[MAX_PAYLOAD];
+
+ /**
+ * Number of valid bytes in @e payload.
+ */
+ size_t fill;
+
+ /**
+ * Number of bytes of @e payload already handed to MHD.
+ */
+ size_t off;
+
+ /**
+ * Number of references, see the comment on the structure.
+ */
+ unsigned int rc;
+
+ /**
+ * Non-zero if the answer is streamed as it is produced ("/events"),
+ * zero if the client only ever sees the finished result ("/slow").
+ * A streaming job resumes the connection after every step, the other
+ * kind only once, at the very end.
+ */
+ int stream;
+
+ /**
+ * Non-zero once the worker has produced everything it is going to
+ * produce.
+ */
+ int finished;
+
+ /**
+ * Non-zero while the connection is suspended (or is just about to
+ * be, which under @e lock is the same thing).
+ */
+ int suspended;
+
+ /**
+ * Non-zero once MHD is done with the connection. The worker must
+ * not call any MHD function on @e connection any more.
+ */
+ int abandoned;
+
+ /**
+ * Non-zero if the worker should give up early.
+ */
+ int stop;
+
+ /**
+ * Non-zero once the worker thread was started. Main thread only.
+ */
+ int started;
+
+ /**
+ * Non-zero once the worker thread was joined. Main thread only.
+ */
+ int joined;
+};
+
+
+/**
+ * Head of the list of all jobs.
+ */
+static struct Job *jobs_head;
+
+/**
+ * Protects #jobs_head and the list links of every job. Lock order is
+ * #jobs_lock before any `struct Job`'s @e lock, never the other way
+ * round.
+ */
+static pthread_mutex_t jobs_lock = PTHREAD_MUTEX_INITIALIZER;
+
+/**
+ * Set by the signal handler, read by the main loop.
+ */
+static volatile sig_atomic_t shutdown_requested;
+
+/**
+ * Written to by the signal handler so that a blocking select() returns
+ * at once. MHD's own wakeup goes through its inter-thread
+ * communication channel, but our signal is our own business.
+ */
+static int wake_pipe[2] = { -1, -1 };
+
+/**
+ * Milliseconds the worker sleeps per step.
+ */
+static unsigned int step_ms = DEFAULT_STEP_MS;
+
+/**
+ * Number of steps that make up one job.
+ */
+static unsigned int total_steps = DEFAULT_STEPS;
+
+
+/**
+ * Signal handler for SIGINT and SIGTERM. Does the two things that are
+ * safe to do here: set a flag and poke a pipe.
+ *
+ * @param sig the signal that was received
+ */
+static void
+signal_handler (int sig)
+{
+ static const char c = 'x';
+
+ (void) sig; /* Unused. Silent compiler warning. */
+ shutdown_requested = 1;
+ if (0 > write (wake_pipe[1],
+ &c,
+ 1))
+ {
+ /* Nothing useful can be done about this here. */
+ }
+}
+
+
+/**
+ * Allocate a job and put it on the global list.
+ *
+ * @param connection the connection the job answers
+ * @param stream non-zero to deliver the steps as they are produced
+ * @return the new job with a reference count of two (one for MHD, one
+ * for the worker that the caller is about to start), NULL on
+ * error
+ */
+static struct Job *
+job_create (struct MHD_Connection *connection,
+ int stream)
+{
+ struct Job *job;
+
+ job = malloc (sizeof (struct Job));
+ if (NULL == job)
+ return NULL;
+ memset (job,
+ 0,
+ sizeof (struct Job));
+ if (0 != pthread_mutex_init (&job->lock,
+ NULL))
+ {
+ free (job);
+ return NULL;
+ }
+ if (0 != pthread_cond_init (&job->cond,
+ NULL))
+ {
+ pthread_mutex_destroy (&job->lock);
+ free (job);
+ return NULL;
+ }
+ job->connection = connection;
+ job->stream = stream;
+ job->rc = 2;
+ pthread_mutex_lock (&jobs_lock);
+ job->next = jobs_head;
+ if (NULL != jobs_head)
+ jobs_head->prev = job;
+ jobs_head = job;
+ pthread_mutex_unlock (&jobs_lock);
+ return job;
+}
+
+
+/**
+ * Drop a reference. Note that this never frees the job: the main
+ * thread has to join the worker first, and only it may do that. See
+ * #reap_jobs().
+ *
+ * @param job the job to release
+ */
+static void
+job_unref (struct Job *job)
+{
+ pthread_mutex_lock (&job->lock);
+ job->rc--;
+ pthread_mutex_unlock (&job->lock);
+}
+
+
+/**
+ * Called by the MHD side once MHD is done with the connection. After
+ * this returns, the worker will not touch the connection again.
+ *
+ * @param job the job to detach from its connection
+ */
+static void
+job_detach (struct Job *job)
+{
+ pthread_mutex_lock (&job->lock);
+ job->abandoned = 1;
+ job->connection = NULL;
+ /* Wake the worker out of its sleep: if the client hung up in the
+ middle of the transfer there is no point in producing the rest. */
+ pthread_cond_signal (&job->cond);
+ pthread_mutex_unlock (&job->lock);
+ job_unref (job);
+}
+
+
+/**
+ * Resume the connection of @a job if it is suspended.
+ *
+ * Must be called with @a job's lock held, and it deliberately calls
+ * into MHD while holding it: the MHD thread has to take the very same
+ * lock in #job_detach() before it can finish the connection, so as
+ * long as we hold the lock the connection cannot go away underneath
+ * us. The reverse order never occurs---MHD invokes our callbacks
+ * without holding any of its own locks that #MHD_resume_connection()
+ * would need---so this cannot deadlock.
+ *
+ * @param job the job whose connection to resume
+ */
+static void
+job_resume_locked (struct Job *job)
+{
+ if ( (0 == job->suspended) ||
+ (0 != job->abandoned) )
+ return;
+ job->suspended = 0;
+ MHD_resume_connection (job->connection);
+}
+
+
+/**
+ * Sleep for one step, or until the job is woken up.
+ *
+ * @param job the job to sleep on
+ * @return non-zero if the worker should stop early
+ */
+static int
+job_sleep (struct Job *job)
+{
+ struct timespec ts;
+ int stop;
+
+ if (0 != clock_gettime (CLOCK_REALTIME,
+ &ts))
+ return 1;
+ ts.tv_sec += (time_t) (step_ms / 1000);
+ ts.tv_nsec += (long) (step_ms % 1000) * 1000000L;
+ if (1000000000L <= ts.tv_nsec)
+ {
+ ts.tv_nsec -= 1000000000L;
+ ts.tv_sec++;
+ }
+ pthread_mutex_lock (&job->lock);
+ while ( (0 == job->stop) &&
+ (0 == job->abandoned) )
+ {
+ if (ETIMEDOUT == pthread_cond_timedwait (&job->cond,
+ &job->lock,
+ &ts))
+ break;
+ }
+ stop = (0 != job->stop) || (0 != job->abandoned);
+ pthread_mutex_unlock (&job->lock);
+ return stop;
+}
+
+
+/**
+ * Append @a len bytes from @a data to the job's payload buffer,
+ * compacting the buffer first. Must be called with @a job's lock
+ * held. A real application would need a policy for the case that the
+ * client cannot keep up; here we simply drop the step, which cannot
+ * happen with the sizes used in this example.
+ *
+ * @param job the job to append to
+ * @param data the bytes to append
+ * @param len the number of bytes to append
+ */
+static void
+job_append_locked (struct Job *job,
+ const char *data,
+ size_t len)
+{
+ if (0 != job->off)
+ {
+ memmove (job->payload,
+ &job->payload[job->off],
+ job->fill - job->off);
+ job->fill -= job->off;
+ job->off = 0;
+ }
+ if (len > MAX_PAYLOAD - job->fill)
+ return;
+ memcpy (&job->payload[job->fill],
+ data,
+ len);
+ job->fill += len;
+}
+
+
+/**
+ * The worker. This stands in for whatever expensive computation,
+ * database cursor or remote query a real application would run: it
+ * produces one server-sent event per step and resumes the connection
+ * whenever it has something new to offer.
+ *
+ * Note that the only MHD function this thread ever calls is
+ * #MHD_resume_connection(). Building and queueing the response is the
+ * business of the thread that runs the daemon.
+ *
+ * @param cls the `struct Job` to work on
+ * @return always NULL
+ */
+static void *
+worker (void *cls)
+{
+ struct Job *job = cls;
+ char event[256];
+ unsigned int i;
+ int len;
+
+ for (i = 1; i <= total_steps; i++)
+ {
+ if (0 != job_sleep (job))
+ break;
+ if (0 != job->stream)
+ len = snprintf (event,
+ sizeof (event),
+ "event: step\ndata: {\"n\":%u,\"total\":%u,"
+ "\"label\":\"step %u of %u done\"}\n\n",
+ i,
+ total_steps,
+ i,
+ total_steps);
+ else
+ len = snprintf (event,
+ sizeof (event),
+ "step %u of %u done\n",
+ i,
+ total_steps);
+ if ( (0 >= len) ||
+ (sizeof (event) <= (size_t) len) )
+ break;
+ pthread_mutex_lock (&job->lock);
+ job_append_locked (job,
+ event,
+ (size_t) len);
+ /* Only a streaming job has anything to show yet. The other kind
+ stays suspended until the very last step. */
+ if (0 != job->stream)
+ job_resume_locked (job);
+ pthread_mutex_unlock (&job->lock);
+ }
+ if (0 != job->stream)
+ len = snprintf (event,
+ sizeof (event),
+ "event: done\ndata: {\"n\":%u,\"total\":%u}\n\n",
+ total_steps,
+ total_steps);
+ else
+ len = snprintf (event,
+ sizeof (event),
+ "all %u steps done\n",
+ total_steps);
+ pthread_mutex_lock (&job->lock);
+ if ( (0 < len) &&
+ (sizeof (event) > (size_t) len) )
+ job_append_locked (job,
+ event,
+ (size_t) len);
+ job->finished = 1;
+ /* The last resume is not optional: if the connection were left
+ suspended, nothing would ever wake it up again, and the daemon
+ could not be stopped. */
+ job_resume_locked (job);
+ pthread_mutex_unlock (&job->lock);
+ job_unref (job);
+ return NULL;
+}
+
+
+/**
+ * Start the worker thread of @a job.
+ *
+ * @param job the job whose worker to start
+ * @return #MHD_YES on success
+ */
+static enum MHD_Result
+job_start (struct Job *job)
+{
+ pthread_mutex_lock (&jobs_lock);
+ if (0 != pthread_create (&job->tid,
+ NULL,
+ &worker,
+ job))
+ {
+ pthread_mutex_unlock (&jobs_lock);
+ return MHD_NO;
+ }
+ job->started = 1;
+ pthread_mutex_unlock (&jobs_lock);
+ return MHD_YES;
+}
+
+
+/**
+ * Content reader for the "/events" stream. This is the interesting
+ * one: when the worker has produced nothing since the last call we
+ * suspend the connection and return zero, instead of returning zero on
+ * its own---which would make MHD ask again immediately and burn a CPU
+ * core for the whole duration of the transfer.
+ *
+ * @param cls our `struct Job`
+ * @param pos number of bytes already returned for this response
+ * @param buf where to copy the data
+ * @param max maximum number of bytes to copy to @a buf
+ * @return number of bytes written to @a buf, 0 if the connection was
+ * suspended, MHD_CONTENT_READER_END_OF_STREAM at the end
+ */
+static ssize_t
+stream_reader (void *cls,
+ uint64_t pos,
+ char *buf,
+ size_t max)
+{
+ struct Job *job = cls;
+ size_t ready;
+
+ (void) pos; /* Unused. Silent compiler warning. */
+ pthread_mutex_lock (&job->lock);
+ if (job->off == job->fill)
+ {
+ job->off = 0;
+ job->fill = 0;
+ if (0 != job->finished)
+ {
+ pthread_mutex_unlock (&job->lock);
+ return MHD_CONTENT_READER_END_OF_STREAM;
+ }
+ /* Nothing to send yet. Take the connection out of the event loop;
+ the worker will put it back in. */
+ job->suspended = 1;
+ MHD_suspend_connection (job->connection);
+ pthread_mutex_unlock (&job->lock);
+ return 0;
+ }
+ ready = job->fill - job->off;
+ if (ready > max)
+ ready = max;
+ memcpy (buf,
+ &job->payload[job->off],
+ ready);
+ job->off += ready;
+ pthread_mutex_unlock (&job->lock);
+ return (ssize_t) ready;
+}
+
+
+/**
+ * Called by MHD when the response object of an "/events" request is
+ * destroyed, which happens for a completed transfer just as well as
+ * for a client that went away in the middle of one. This is the MHD
+ * side of the job's reference count.
+ *
+ * @param cls our `struct Job`
+ */
+static void
+stream_done (void *cls)
+{
+ job_detach (cls);
+}
+
+
+/**
+ * Handle "GET /events": queue a streaming response right away and let
+ * the content reader deal with the fact that the data does not exist
+ * yet.
+ *
+ * @param connection the connection to answer
+ * @return #MHD_YES on success
+ */
+static enum MHD_Result
+handle_events (struct MHD_Connection *connection)
+{
+ struct MHD_Response *response;
+ struct Job *job;
+ enum MHD_Result ret;
+
+ job = job_create (connection,
+ 1);
+ if (NULL == job)
+ return MHD_NO;
+ response = MHD_create_response_from_callback (MHD_SIZE_UNKNOWN,
+ IO_BLOCK_SIZE,
+ &stream_reader,
+ job,
+ &stream_done);
+ if (NULL == response)
+ {
+ /* No response object exists, so nobody will call stream_done() for
+ us; drop both references by hand. The worker was never
+ started. */
+ job_unref (job);
+ job_unref (job);
+ return MHD_NO;
+ }
+ if (MHD_NO == job_start (job))
+ {
+ MHD_destroy_response (response);
+ job_unref (job);
+ return MHD_NO;
+ }
+ (void) MHD_add_response_header (response,
+ MHD_HTTP_HEADER_CONTENT_TYPE,
+ "text/event-stream");
+ (void) MHD_add_response_header (response,
+ MHD_HTTP_HEADER_CACHE_CONTROL,
+ "no-cache");
+ ret = MHD_queue_response (connection,
+ MHD_HTTP_OK,
+ response);
+ MHD_destroy_response (response);
+ return ret;
+}
+
+
+/**
+ * Handle "GET /slow": the answer is only useful as a whole, so instead
+ * of streaming we suspend the connection until the worker is done.
+ * MHD then calls this function again---on its own thread---and that is
+ * where the response is queued. #MHD_queue_response() is never called
+ * from the worker.
+ *
+ * @param connection the connection to answer
+ * @param req_cls the per-request pointer, holding our `struct Job`
+ * @return #MHD_YES on success
+ */
+static enum MHD_Result
+handle_slow (struct MHD_Connection *connection,
+ void **req_cls)
+{
+ struct MHD_Response *response;
+ struct Job *job = *req_cls;
+ char answer[MAX_PAYLOAD];
+ size_t len;
+ enum MHD_Result ret;
+
+ if (NULL == job)
+ {
+ /* First call for this request: start the work and ask MHD to come
+ back to us. */
+ job = job_create (connection,
+ 0);
+ if (NULL == job)
+ return MHD_NO;
+ if (MHD_NO == job_start (job))
+ {
+ job_unref (job);
+ job_unref (job);
+ return MHD_NO;
+ }
+ *req_cls = job;
+ return MHD_YES;
+ }
+ pthread_mutex_lock (&job->lock);
+ if (0 == job->finished)
+ {
+ /* Still working. Park the connection; the worker resumes it when
+ it is done, and MHD will then enter this function once more. */
+ job->suspended = 1;
+ MHD_suspend_connection (connection);
+ pthread_mutex_unlock (&job->lock);
+ return MHD_YES;
+ }
+ len = job->fill - job->off;
+ if (len > sizeof (answer))
+ len = sizeof (answer);
+ memcpy (answer,
+ &job->payload[job->off],
+ len);
+ pthread_mutex_unlock (&job->lock);
+
+ response = MHD_create_response_from_buffer_copy (len,
+ answer);
+ if (NULL == response)
+ return MHD_NO;
+ (void) MHD_add_response_header (response,
+ MHD_HTTP_HEADER_CONTENT_TYPE,
+ "text/plain");
+ ret = MHD_queue_response (connection,
+ MHD_HTTP_OK,
+ response);
+ MHD_destroy_response (response);
+ return ret;
+}
+
+
+/**
+ * Queue a complete response that is already sitting in memory.
+ *
+ * @param connection the connection to answer
+ * @param status the HTTP status code to use
+ * @param mime the value for the "Content-Type" header
+ * @param body the body to send
+ * @return #MHD_YES on success
+ */
+static enum MHD_Result
+queue_static (struct MHD_Connection *connection,
+ unsigned int status,
+ const char *mime,
+ const char *body)
+{
+ struct MHD_Response *response;
+ enum MHD_Result ret;
+
+ response = MHD_create_response_from_buffer_copy (strlen (body),
+ body);
+ if (NULL == response)
+ return MHD_NO;
+ (void) MHD_add_response_header (response,
+ MHD_HTTP_HEADER_CONTENT_TYPE,
+ mime);
+ (void) MHD_add_response_header (response,
+ MHD_HTTP_HEADER_CACHE_CONTROL,
+ "no-store");
+ ret = MHD_queue_response (connection,
+ status,
+ response);
+ MHD_destroy_response (response);
+ return ret;
+}
+
+
+static enum MHD_Result
+answer_to_connection (void *cls,
+ struct MHD_Connection *connection,
+ const char *url,
+ const char *method,
+ const char *version,
+ const char *upload_data,
+ size_t *upload_data_size,
+ void **req_cls)
+{
+ (void) cls; /* Unused. Silent compiler warning. */
+ (void) version; /* Unused. Silent compiler warning. */
+ (void) upload_data; /* Unused. Silent compiler warning. */
+ (void) upload_data_size; /* Unused. Silent compiler warning. */
+
+ if (0 != strcmp (method,
+ MHD_HTTP_METHOD_GET))
+ return MHD_NO;
+ if (0 == strcmp (url,
+ "/"))
+ return queue_static (connection,
+ MHD_HTTP_OK,
+ "text/html",
+ PAGE);
+ if (0 == strcmp (url,
+ "/fast"))
+ return queue_static (connection,
+ MHD_HTTP_OK,
+ "text/plain",
+ "pong\n");
+ if ( (0 == strcmp (url,
+ "/events")) ||
+ (0 == strcmp (url,
+ "/slow")) )
+ {
+ if (0 != shutdown_requested)
+ return queue_static (connection,
+ MHD_HTTP_SERVICE_UNAVAILABLE,
+ "text/plain",
+ "shutting down\n");
+ if (0 == strcmp (url,
+ "/events"))
+ return handle_events (connection);
+ return handle_slow (connection,
+ req_cls);
+ }
+ return queue_static (connection,
+ MHD_HTTP_NOT_FOUND,
+ "text/plain",
+ "not found\n");
+}
+
+
+/**
+ * Called by MHD when a request is done. For "/slow" this is where the
+ * MHD side of the reference count is dropped; the other handlers leave
+ * @a req_cls alone, so there is nothing to do for them.
+ *
+ * @param cls closure, unused
+ * @param connection the connection that finished
+ * @param req_cls the per-request pointer
+ * @param toe why the request ended
+ */
+static void
+request_completed (void *cls,
+ struct MHD_Connection *connection,
+ void **req_cls,
+ enum MHD_RequestTerminationCode toe)
+{
+ struct Job *job = *req_cls;
+
+ (void) cls; /* Unused. Silent compiler warning. */
+ (void) connection; /* Unused. Silent compiler warning. */
+ (void) toe; /* Unused. Silent compiler warning. */
+ if (NULL == job)
+ return;
+ *req_cls = NULL;
+ job_detach (job);
+}
+
+
+/**
+ * Free every job that nobody references any more, joining its worker
+ * on the way. Called from the main loop, and thus always from the
+ * thread that also runs the daemon.
+ */
+static void
+reap_jobs (void)
+{
+ struct Job *job;
+ struct Job *next;
+ unsigned int rc;
+
+ pthread_mutex_lock (&jobs_lock);
+ for (job = jobs_head; NULL != job; job = next)
+ {
+ next = job->next;
+ pthread_mutex_lock (&job->lock);
+ rc = job->rc;
+ pthread_mutex_unlock (&job->lock);
+ if (0 != rc)
+ continue;
+ if ( (0 != job->started) &&
+ (0 == job->joined) )
+ {
+ pthread_join (job->tid,
+ NULL);
+ job->joined = 1;
+ }
+ if (NULL != job->prev)
+ job->prev->next = job->next;
+ else
+ jobs_head = job->next;
+ if (NULL != job->next)
+ job->next->prev = job->prev;
+ pthread_cond_destroy (&job->cond);
+ pthread_mutex_destroy (&job->lock);
+ free (job);
+ }
+ pthread_mutex_unlock (&jobs_lock);
+}
+
+
+/**
+ * Ask every worker to stop and wait until they all did.
+ *
+ * This has to happen before the daemon is stopped: a worker that is
+ * still asleep may be holding the only promise to resume a suspended
+ * connection, and stopping a daemon that has suspended connections is
+ * an API violation. Once every worker has returned, no connection can
+ * be suspended any more, because the workers resume before they exit
+ * and the content reader only ever suspends while a worker is running.
+ */
+static void
+stop_all_jobs (void)
+{
+ struct Job *job;
+
+ pthread_mutex_lock (&jobs_lock);
+ for (job = jobs_head; NULL != job; job = job->next)
+ {
+ pthread_mutex_lock (&job->lock);
+ job->stop = 1;
+ pthread_cond_signal (&job->cond);
+ pthread_mutex_unlock (&job->lock);
+ }
+ for (job = jobs_head; NULL != job; job = job->next)
+ {
+ if ( (0 != job->started) &&
+ (0 == job->joined) )
+ {
+ pthread_join (job->tid,
+ NULL);
+ job->joined = 1;
+ }
+ }
+ pthread_mutex_unlock (&jobs_lock);
+}
+
+
+/**
+ * Number of connections the daemon is currently handling.
+ *
+ * @param daemon the daemon to query
+ * @return the number of connections, 0 if it cannot be determined
+ */
+static unsigned int
+connection_count (struct MHD_Daemon *daemon)
+{
+ const union MHD_DaemonInfo *info;
+
+ info = MHD_get_daemon_info (daemon,
+ MHD_DAEMON_INFO_CURRENT_CONNECTIONS);
+ if (NULL == info)
+ return 0;
+ return info->num_connections;
+}
+
+
+int
+main (int argc,
+ char *const *argv)
+{
+ struct MHD_Daemon *daemon;
+ const union MHD_DaemonInfo *info;
+ struct sigaction sa;
+ fd_set rs;
+ fd_set ws;
+ fd_set es;
+ struct timeval tv;
+ struct timeval *tvp;
+ MHD_socket max;
+ uint64_t mhd_timeout;
+ unsigned long port = DEFAULT_PORT;
+ unsigned int drained_ms = 0;
+ char drain_buf[64];
+ int draining = 0;
+ int ret = 0;
+
+ if (1 < argc)
+ port = strtoul (argv[1],
+ NULL,
+ 10);
+ if (2 < argc)
+ step_ms = (unsigned int) strtoul (argv[2],
+ NULL,
+ 10);
+ if (3 < argc)
+ total_steps = (unsigned int) strtoul (argv[3],
+ NULL,
+ 10);
+ if ( (65535 < port) ||
+ (0 == total_steps) )
+ {
+ fprintf (stderr,
+ "Usage: %s [PORT [STEP_MS [STEPS]]]\n"
+ "Pass 0 as PORT to let the system pick a free one.\n",
+ argv[0]);
+ return 1;
+ }
+
+ if (0 > pipe (wake_pipe))
+ {
+ fprintf (stderr,
+ "Failed to create wakeup pipe: %s\n",
+ strerror (errno));
+ return 1;
+ }
+ memset (&sa,
+ 0,
+ sizeof (sa));
+ sa.sa_handler = &signal_handler;
+ sigaction (SIGINT,
+ &sa,
+ NULL);
+ sigaction (SIGTERM,
+ &sa,
+ NULL);
+ sa.sa_handler = SIG_IGN;
+ sigaction (SIGPIPE,
+ &sa,
+ NULL);
+
+ /* No MHD_USE_INTERNAL_POLLING_THREAD: the loop below is ours.
+ MHD_ALLOW_SUSPEND_RESUME implies MHD_USE_ITC, and that channel is
+ what lets a worker thread break us out of the select() call. */
+ daemon = MHD_start_daemon (MHD_ALLOW_SUSPEND_RESUME | MHD_USE_ERROR_LOG,
+ (uint16_t) port,
+ NULL, NULL,
+ &answer_to_connection, NULL,
+ MHD_OPTION_NOTIFY_COMPLETED,
+ &request_completed, NULL,
+ MHD_OPTION_END);
+ if (NULL == daemon)
+ {
+ fprintf (stderr,
+ "Failed to start daemon.\n");
+ return 1;
+ }
+ info = MHD_get_daemon_info (daemon,
+ MHD_DAEMON_INFO_BIND_PORT);
+ if ( (NULL == info) ||
+ (0 == info->port) )
+ {
+ fprintf (stderr,
+ "Failed to determine bound port.\n");
+ MHD_stop_daemon (daemon);
+ return 1;
+ }
+ printf ("Listening on port %u\n",
+ (unsigned int) info->port);
+ fflush (stdout);
+
+ while (1)
+ {
+ if ( (0 != shutdown_requested) &&
+ (0 == draining) )
+ {
+ /* Every worker has to be gone before the daemon may be stopped. */
+ stop_all_jobs ();
+ draining = 1;
+ }
+ if ( (0 != draining) &&
+ ( (0 == connection_count (daemon)) ||
+ (DRAIN_TIMEOUT_MS <= drained_ms) ) )
+ break;
+
+ FD_ZERO (&rs);
+ FD_ZERO (&ws);
+ FD_ZERO (&es);
+ max = 0;
+ if (MHD_YES != MHD_get_fdset (daemon,
+ &rs,
+ &ws,
+ &es,
+ &max))
+ {
+ fprintf (stderr,
+ "Failed to obtain the file descriptor set.\n");
+ ret = 1;
+ break;
+ }
+ /* Our own descriptors go into the very same sets. */
+ FD_SET (wake_pipe[0],
+ &rs);
+ if (max < wake_pipe[0])
+ max = wake_pipe[0];
+
+ if (0 != draining)
+ {
+ /* Look at the drain deadline regularly. */
+ tv.tv_sec = 0;
+ tv.tv_usec = 50 * 1000;
+ tvp = &tv;
+ drained_ms += 50;
+ }
+ else if (MHD_YES == MHD_get_timeout64 (daemon,
+ &mhd_timeout))
+ {
+ tv.tv_sec = (time_t) (mhd_timeout / 1000);
+ tv.tv_usec = ((long) (mhd_timeout % 1000)) * 1000;
+ tvp = &tv;
+ }
+ else
+ {
+ /* MHD has nothing to wait for. With every connection suspended
+ this is the normal case, and the loop simply sleeps until a
+ worker resumes one---which reaches us through MHD's ITC, as it
+ is part of the read set above. */
+ tvp = NULL;
+ }
+
+ if (-1 == select ((int) max + 1,
+ &rs,
+ &ws,
+ &es,
+ tvp))
+ {
+ if (EINTR == errno)
+ continue;
+ fprintf (stderr,
+ "Aborting due to error during select: %s\n",
+ strerror (errno));
+ ret = 1;
+ break;
+ }
+ if (FD_ISSET (wake_pipe[0],
+ &rs))
+ (void) read (wake_pipe[0],
+ drain_buf,
+ sizeof (drain_buf));
+ MHD_run_from_select (daemon,
+ &rs,
+ &ws,
+ &es);
+ reap_jobs ();
+ }
+
+ stop_all_jobs ();
+ reap_jobs ();
+ MHD_stop_daemon (daemon);
+ close (wake_pipe[0]);
+ close (wake_pipe[1]);
+ return ret;
+}
diff --git a/doc/examples/test_asyncresponse.c b/doc/examples/test_asyncresponse.c
@@ -0,0 +1,671 @@
+/* Feel free to use this example code in any way
+ you see fit (Public Domain) */
+
+/**
+ * @file test_asyncresponse.c
+ * @brief Test for the "asyncresponse" tutorial example. The example
+ * is started as a child process on an ephemeral port and then
+ * exercised over HTTP, so that what is tested is exactly the
+ * code the tutorial prints.
+ *
+ * Beyond checking that the right bytes come back, the test
+ * verifies the two properties that the chapter is actually
+ * about: that the body of "/events" really trickles in rather
+ * than arriving in one piece at the end, and that the server
+ * burns no CPU while it waits. The latter is what tells a
+ * properly suspended connection apart from a content reader
+ * that returns zero in a loop---both deliver the same bytes.
+ * @author Christian Grothoff
+ */
+
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <curl/curl.h>
+#include <errno.h>
+#include <pthread.h>
+#include <signal.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+
+/**
+ * Milliseconds the example should spend per step.
+ */
+#define STEP_MS 50
+
+/**
+ * Number of steps per job, and thus (STEPS * STEP_MS) milliseconds of
+ * work for one request.
+ */
+#define STEPS 20
+
+/**
+ * Nominal duration of one job in milliseconds.
+ */
+#define JOB_MS (STEPS * STEP_MS)
+
+/**
+ * Upper bound for any single request, in seconds.
+ */
+#define REQUEST_TIMEOUT 30
+
+/**
+ * Largest response body we are prepared to keep.
+ */
+#define MAX_BODY 65536
+
+
+/**
+ * What one HTTP request collected.
+ */
+struct Fetch
+{
+ /**
+ * The body, as far as it was received.
+ */
+ char body[MAX_BODY];
+
+ /**
+ * Number of valid bytes in @e body.
+ */
+ size_t len;
+
+ /**
+ * Number of times the write callback was invoked.
+ */
+ unsigned int chunks;
+
+ /**
+ * Time of the first invocation of the write callback, in
+ * milliseconds since the start of the request.
+ */
+ long first_ms;
+
+ /**
+ * Time of the last invocation of the write callback, in
+ * milliseconds since the start of the request.
+ */
+ long last_ms;
+
+ /**
+ * Start of the request.
+ */
+ struct timespec start;
+
+ /**
+ * Abort the transfer after this many callbacks; zero to never abort.
+ */
+ unsigned int abort_after;
+
+ /**
+ * Result of curl_easy_perform().
+ */
+ CURLcode res;
+};
+
+
+/**
+ * The example's process ID.
+ */
+static pid_t server_pid;
+
+/**
+ * The port the example bound.
+ */
+static unsigned int server_port;
+
+/**
+ * Number of checks that failed.
+ */
+static unsigned int failures;
+
+
+/**
+ * Report the outcome of one check.
+ *
+ * @param ok non-zero if the check passed
+ * @param what what was being checked
+ */
+static void
+check (int ok,
+ const char *what)
+{
+ if (! ok)
+ failures++;
+ fprintf (stderr,
+ "%s: %s\n",
+ ok ? "PASS" : "FAIL",
+ what);
+}
+
+
+/**
+ * @return the current value of the monotonic clock, in milliseconds
+ */
+static long
+now_ms (void)
+{
+ struct timespec ts;
+
+ if (0 != clock_gettime (CLOCK_MONOTONIC,
+ &ts))
+ return 0;
+ return (long) ts.tv_sec * 1000 + ts.tv_nsec / 1000000;
+}
+
+
+/**
+ * @param since the earlier point in time
+ * @return milliseconds elapsed since @a since
+ */
+static long
+elapsed_ms (const struct timespec *since)
+{
+ struct timespec ts;
+
+ if (0 != clock_gettime (CLOCK_MONOTONIC,
+ &ts))
+ return 0;
+ return (long) (ts.tv_sec - since->tv_sec) * 1000
+ + (ts.tv_nsec - since->tv_nsec) / 1000000;
+}
+
+
+/**
+ * Read the CPU time consumed by process @a pid so far. Only
+ * implemented for Linux; everywhere else the test simply skips the
+ * check that uses it.
+ *
+ * @param pid the process to look at
+ * @param[out] ms where to store the sum of user and system time in
+ * milliseconds
+ * @return non-zero on success
+ */
+static int
+cpu_time_ms (pid_t pid,
+ long *ms)
+{
+#ifdef __linux__
+ char path[64];
+ char buf[1024];
+ char *p;
+ FILE *f;
+ size_t got;
+ unsigned long utime;
+ unsigned long stime;
+ long ticks;
+
+ snprintf (path,
+ sizeof (path),
+ "/proc/%ld/stat",
+ (long) pid);
+ f = fopen (path,
+ "r");
+ if (NULL == f)
+ return 0;
+ got = fread (buf,
+ 1,
+ sizeof (buf) - 1,
+ f);
+ fclose (f);
+ if (0 == got)
+ return 0;
+ buf[got] = '\0';
+ /* The second field is the executable name and may contain spaces and
+ parentheses, so start parsing behind its closing one. */
+ p = strrchr (buf,
+ ')');
+ if (NULL == p)
+ return 0;
+ if (2 != sscanf (p + 1,
+ " %*c %*d %*d %*d %*d %*d %*u %*u %*u %*u %*u %lu %lu",
+ &utime,
+ &stime))
+ return 0;
+ ticks = sysconf (_SC_CLK_TCK);
+ if (0 >= ticks)
+ return 0;
+ *ms = (long) ((utime + stime) * 1000 / (unsigned long) ticks);
+ return 1;
+#else
+ (void) pid;
+ (void) ms;
+ return 0;
+#endif
+}
+
+
+static size_t
+write_cb (void *ptr,
+ size_t size,
+ size_t nmemb,
+ void *cls)
+{
+ struct Fetch *f = cls;
+ size_t total = size * nmemb;
+
+ f->chunks++;
+ if (1 == f->chunks)
+ f->first_ms = elapsed_ms (&f->start);
+ f->last_ms = elapsed_ms (&f->start);
+ if ( (0 != f->abort_after) &&
+ (f->chunks >= f->abort_after) )
+ return 0; /* makes curl abort the transfer */
+ if (total > sizeof (f->body) - f->len - 1)
+ total = sizeof (f->body) - f->len - 1;
+ memcpy (&f->body[f->len],
+ ptr,
+ total);
+ f->len += total;
+ f->body[f->len] = '\0';
+ return size * nmemb;
+}
+
+
+/**
+ * Perform one GET against the example.
+ *
+ * @param path the path to request
+ * @param[out] f where to store what was received; @e abort_after has to
+ * be set by the caller, everything else is initialised here
+ * @return milliseconds the request took
+ */
+static long
+fetch (const char *path,
+ struct Fetch *f)
+{
+ CURL *curl;
+ char url[128];
+ unsigned int abort_after = f->abort_after;
+
+ memset (f,
+ 0,
+ sizeof (struct Fetch));
+ f->abort_after = abort_after;
+ f->res = CURLE_FAILED_INIT;
+ snprintf (url,
+ sizeof (url),
+ "http://127.0.0.1:%u%s",
+ server_port,
+ path);
+ curl = curl_easy_init ();
+ if (NULL == curl)
+ return 0;
+ curl_easy_setopt (curl, CURLOPT_URL, url);
+ curl_easy_setopt (curl, CURLOPT_WRITEFUNCTION, &write_cb);
+ curl_easy_setopt (curl, CURLOPT_WRITEDATA, f);
+ curl_easy_setopt (curl, CURLOPT_TIMEOUT, (long) REQUEST_TIMEOUT);
+ curl_easy_setopt (curl, CURLOPT_NOSIGNAL, 1L);
+ clock_gettime (CLOCK_MONOTONIC,
+ &f->start);
+ f->res = curl_easy_perform (curl);
+ curl_easy_cleanup (curl);
+ return elapsed_ms (&f->start);
+}
+
+
+/**
+ * Thread body that streams "/events" in the background while the main
+ * thread does something else.
+ *
+ * @param cls a `struct Fetch` whose @e abort_after is already set
+ * @return always NULL
+ */
+static void *
+background_stream (void *cls)
+{
+ (void) fetch ("/events",
+ cls);
+ return NULL;
+}
+
+
+/**
+ * Start the example on an ephemeral port and learn which one it got.
+ *
+ * @return non-zero on success
+ */
+static int
+start_server (void)
+{
+ char steps[16];
+ char step_ms[16];
+ int fds[2];
+ FILE *out;
+ char line[128];
+
+ if (0 != pipe (fds))
+ return 0;
+ snprintf (step_ms,
+ sizeof (step_ms),
+ "%u",
+ (unsigned int) STEP_MS);
+ snprintf (steps,
+ sizeof (steps),
+ "%u",
+ (unsigned int) STEPS);
+ server_pid = fork ();
+ if (-1 == server_pid)
+ {
+ close (fds[0]);
+ close (fds[1]);
+ return 0;
+ }
+ if (0 == server_pid)
+ {
+ close (fds[0]);
+ if (STDOUT_FILENO != fds[1])
+ {
+ dup2 (fds[1],
+ STDOUT_FILENO);
+ close (fds[1]);
+ }
+ execl ("./asyncresponse",
+ "asyncresponse",
+ "0",
+ step_ms,
+ steps,
+ (char *) NULL);
+ fprintf (stderr,
+ "Failed to exec ./asyncresponse: %s\n",
+ strerror (errno));
+ _exit (77);
+ }
+ close (fds[1]);
+ out = fdopen (fds[0],
+ "r");
+ if (NULL == out)
+ {
+ close (fds[0]);
+ return 0;
+ }
+ if (NULL == fgets (line,
+ sizeof (line),
+ out))
+ {
+ fclose (out);
+ return 0;
+ }
+ fclose (out);
+ if (1 != sscanf (line,
+ "Listening on port %u",
+ &server_port))
+ {
+ fprintf (stderr,
+ "Unexpected greeting from the example: %s",
+ line);
+ return 0;
+ }
+ return 1;
+}
+
+
+/**
+ * "GET /slow" suspends in the access handler and is answered as a
+ * whole once the worker is done.
+ */
+static void
+test_slow (void)
+{
+ struct Fetch f;
+ long ms;
+
+ memset (&f,
+ 0,
+ sizeof (f));
+ ms = fetch ("/slow",
+ &f);
+ check (CURLE_OK == f.res,
+ "/slow completed");
+ check (NULL != strstr (f.body,
+ "all 20 steps done"),
+ "/slow delivered the complete answer");
+ /* The worker sleeps STEP_MS per step, so the answer cannot possibly
+ be ready earlier. Only a lower bound is checked: a loaded machine
+ may take arbitrarily longer. */
+ check (ms >= JOB_MS / 2,
+ "/slow waited for the worker");
+}
+
+
+/**
+ * "GET /events" streams as the worker produces, and the server stays
+ * idle in between. This is the check that a content reader returning
+ * zero in a loop would fail.
+ */
+static void
+test_events_are_incremental (void)
+{
+ struct Fetch f;
+ long cpu_before;
+ long cpu_after;
+ long ms;
+ int have_cpu;
+
+ memset (&f,
+ 0,
+ sizeof (f));
+ have_cpu = cpu_time_ms (server_pid,
+ &cpu_before);
+ ms = fetch ("/events",
+ &f);
+ check (CURLE_OK == f.res,
+ "/events completed");
+ check (NULL != strstr (f.body,
+ "event: done"),
+ "/events delivered the final event");
+ check (5 <= f.chunks,
+ "/events arrived in several pieces");
+ /* The decisive one: the first piece has to show up long before the
+ last. A response that is assembled first and sent afterwards would
+ have first_ms very close to last_ms. */
+ check (f.last_ms - f.first_ms >= JOB_MS / 2,
+ "/events trickled in rather than arriving at once");
+
+ if (! have_cpu)
+ {
+ fprintf (stderr,
+ "SKIP: no way to read the server's CPU time on this "
+ "platform\n");
+ return;
+ }
+ if (! cpu_time_ms (server_pid,
+ &cpu_after))
+ {
+ fprintf (stderr,
+ "SKIP: could not read the server's CPU time\n");
+ return;
+ }
+ /* The server spent almost all of that second waiting. If the
+ connection were not suspended, MHD would poll the content reader
+ as fast as it can and this would be close to 100% of one core. */
+ fprintf (stderr,
+ "server used %ld ms of CPU during %ld ms of streaming\n",
+ cpu_after - cpu_before,
+ ms);
+ check ((cpu_after - cpu_before) * 4 < ms,
+ "server stayed idle while the connection was suspended");
+}
+
+
+/**
+ * The event loop keeps serving other requests while jobs are parked.
+ */
+static void
+test_loop_stays_responsive (void)
+{
+ struct Fetch stream;
+ struct Fetch ping;
+ pthread_t tid;
+ long worst = 0;
+ long ms;
+ unsigned int i;
+
+ memset (&stream,
+ 0,
+ sizeof (stream));
+ if (0 != pthread_create (&tid,
+ NULL,
+ &background_stream,
+ &stream))
+ {
+ check (0,
+ "could not start the background stream");
+ return;
+ }
+ for (i = 0; i < 5; i++)
+ {
+ memset (&ping,
+ 0,
+ sizeof (ping));
+ ms = fetch ("/fast",
+ &ping);
+ if ( (CURLE_OK != ping.res) ||
+ (NULL == strstr (ping.body,
+ "pong")) )
+ worst = REQUEST_TIMEOUT * 1000;
+ else if (ms > worst)
+ worst = ms;
+ usleep (50 * 1000);
+ }
+ pthread_join (tid,
+ NULL);
+ fprintf (stderr,
+ "slowest /fast while a job was running: %ld ms\n",
+ worst);
+ check (worst < 500,
+ "/fast was served promptly while a job was suspended");
+}
+
+
+/**
+ * A client that goes away in the middle must not take the server with
+ * it, and the worker behind it has to be cleaned up.
+ */
+static void
+test_client_abort (void)
+{
+ struct Fetch f;
+ struct Fetch ping;
+
+ memset (&f,
+ 0,
+ sizeof (f));
+ f.abort_after = 3;
+ (void) fetch ("/events",
+ &f);
+ check (CURLE_OK != f.res,
+ "aborted transfer was reported as such");
+ memset (&ping,
+ 0,
+ sizeof (ping));
+ (void) fetch ("/fast",
+ &ping);
+ check ( (CURLE_OK == ping.res) &&
+ (NULL != strstr (ping.body,
+ "pong")),
+ "server survived a client that hung up mid-stream");
+}
+
+
+/**
+ * Stopping the daemon while a connection is suspended is an API
+ * violation, so the example has to resume everything first. If it got
+ * that wrong we would see a panic or a hang here instead of a clean
+ * exit.
+ */
+static void
+test_clean_shutdown (void)
+{
+ struct Fetch stream;
+ pthread_t tid;
+ int status;
+ pid_t got;
+
+ memset (&stream,
+ 0,
+ sizeof (stream));
+ if (0 != pthread_create (&tid,
+ NULL,
+ &background_stream,
+ &stream))
+ {
+ check (0,
+ "could not start the background stream");
+ return;
+ }
+ /* Give the job time to get going, so that its connection really is
+ suspended when the signal arrives. */
+ usleep ((useconds_t) (STEP_MS * 3) * 1000);
+ kill (server_pid,
+ SIGTERM);
+ pthread_join (tid,
+ NULL);
+ got = waitpid (server_pid,
+ &status,
+ 0);
+ server_pid = -1;
+ check (0 < got,
+ "reaped the server");
+ check (WIFEXITED (status) && (0 == WEXITSTATUS (status)),
+ "server shut down cleanly with a suspended connection");
+}
+
+
+int
+main (void)
+{
+ long deadline;
+
+ if (0 != curl_global_init (CURL_GLOBAL_ALL))
+ return 77;
+ if (! start_server ())
+ {
+ fprintf (stderr,
+ "Could not start ./asyncresponse\n");
+ curl_global_cleanup ();
+ return 77;
+ }
+ fprintf (stderr,
+ "example listening on port %u\n",
+ server_port);
+
+ /* Wait for the listen socket to be usable. */
+ deadline = now_ms () + 5000;
+ while (now_ms () < deadline)
+ {
+ struct Fetch f;
+
+ memset (&f,
+ 0,
+ sizeof (f));
+ (void) fetch ("/fast",
+ &f);
+ if (CURLE_OK == f.res)
+ break;
+ usleep (50 * 1000);
+ }
+
+ test_slow ();
+ test_events_are_incremental ();
+ test_loop_stays_responsive ();
+ test_client_abort ();
+ test_clean_shutdown ();
+
+ if (-1 != server_pid)
+ {
+ kill (server_pid,
+ SIGKILL);
+ waitpid (server_pid,
+ NULL,
+ 0);
+ }
+ curl_global_cleanup ();
+ if (0 != failures)
+ fprintf (stderr,
+ "%u check(s) failed\n",
+ failures);
+ return (0 == failures) ? 0 : 1;
+}
diff --git a/doc/libmicrohttpd-tutorial.texi b/doc/libmicrohttpd-tutorial.texi
@@ -68,6 +68,7 @@ Free Documentation License".
* Improved processing of POST data::
* Session management::
* Generating responses on the fly::
+* Answering from another thread::
* Adding a layer of security::
* Upgrading connections::
* Bibliography::
@@ -111,6 +112,10 @@ Free Documentation License".
@chapter Generating responses on the fly
@include chapters/callbackresponse.inc
+@node Answering from another thread
+@chapter Answering from another thread
+@include chapters/asyncresponses.inc
+
@node Adding a layer of security
@chapter Adding a layer of security
@include chapters/tlsauthentication.inc
@@ -138,6 +143,7 @@ Free Documentation License".
* largepost.c::
* sessions.c::
* callbackresponse.c::
+* asyncresponse.c::
* tlsauthentication.c::
* upgrade.c::
@end menu
@@ -190,6 +196,12 @@ Free Documentation License".
@verbatiminclude examples/callbackresponse.c
@end smalldisplay
+@node asyncresponse.c
+@section asyncresponse.c
+@smalldisplay
+@verbatiminclude examples/asyncresponse.c
+@end smalldisplay
+
@node tlsauthentication.c
@section tlsauthentication.c
@smalldisplay
diff --git a/doc/libmicrohttpd.texi b/doc/libmicrohttpd.texi
@@ -2482,10 +2482,27 @@ make sure to run @code{MHD_run} afterwards (before again calling
the set returned by @code{MHD_get_fdset} and you may end up with a
connection that is stuck until the next network activity.
+This function may be called from any thread, which is what makes it
+usable to hand work to threads of your own. In ``external'' select
+mode it writes to the inter-thread communication channel that
+@code{MHD_ALLOW_SUSPEND_RESUME} sets up, and since
+@code{MHD_get_fdset} adds the reading end of that channel to the read
+set, a call from another thread interrupts a @code{select} that your
+application is blocked in. That matters because
+@code{MHD_get_timeout} reports no timeout at all while every
+connection is suspended, so your application would otherwise block
+indefinitely. The channel is level triggered, so a resume issued
+between @code{MHD_get_fdset} and @code{select} cannot be lost.
+
You can check whether a connection is currently suspended using
@code{MHD_get_connection_info} by querying for
@code{MHD_CONNECTION_INFO_CONNECTION_SUSPENDED}.
+A worked example of suspending connections while worker threads
+produce the answers, driven by an external @code{select} loop, is
+given in the chapter ``Answering from another thread'' of the MHD
+tutorial.
+
@table @var
@item connection
the connection to resume
