From:

Will Bryant <will.bryant@gmail.com>
  To: 
libmicrohttpd development and user mailinglist <libmicrohttpd@gnu.org>
  Date: 
Yesterday 03:59:23 PM
  Attachments: 
 0001-use-a-pipe-to-signal-shutdown-to-select-and-poll.patch
   
I attach my current work.  I have attempted to preserve and restore as much of the 'old' pipe shutdown code that was present in earlier versions of the library, but have removed the sections that aren't relevant now that there is a socket-by-socket client shutdown procedure.

On OS X, all the 'regular' tests pass.  The two perf test programs both fail on a random subset of the tests with "Failed to bind to port 1081: Address already in use" (or the other ports that the perf tests use).

I initially thought that this was a problem with this patch, but after I couldn't find any problem with it I tried backporting the perf test programs to the older 0.9.7 version of libmicrohttpd, which is before the listening socket shutdown code went in, and it turned out that the older versions failed in the same way.

On OpenIndiana (oi_151a - but should behave the same as other SunOS/Solaris/OpenSolaris/Illumos kernels), the same problem occurs.  (I haven't tried the backport there too.)

Linux is known to have slightly different behavior for the SOL_REUSEADDR option which might explain this, but it's quite possible that this is just a timing issue with the cleanup of the client threads in these perf test programs (if you wait for 1s between the tests then the problem goes away).  

On OS X, using both debugging printfs and shelling out to lsof -itcp, I have confirmed that the listening socket has been closed successfully before the error occurs.   So I am starting to suspect that it is not possible to bind to an address that was used as a server socket until all the client sockets have also been cleaned up - but that's just a theory at this point.

Anyway, as far as this patch goes, OS X looks a lot healthier - as healthy as it was in 0.9.7 (0.9.8 and 0.9.9 both failed to build entirely).

There is one further test failure on openindiana, which I believe is not related to this patch - all the same 'regular' tests pass except daemontest_get_response_cleanup, which dies due to an unhandled SIGPIPE when trying to write to the socket after killing the curl process:

Program received signal SIGPIPE, Broken pipe.
[Switching to Thread 2 (LWP 2)]
0xfed634f7 in __so_send () from /lib/libc.so.1
(gdb) bt
#0  0xfed634f7 in __so_send () from /lib/libc.so.1
#1  0xfed511d7 in _so_send () from /lib/libc.so.1
#2  0xfe7dcb26 in send () from /lib/libsocket.so.1
#3  0xfef56d29 in send_param_adapter (connection=0x20, other=0xfe99080f, i=0) at daemon.c:764
#4  0xfef54a0b in do_write (connection=0x8073ad0) at connection.c:1495
#5  0xfef54f6f in MHD_connection_handle_write (connection=0x8073ad0) at connection.c:1909
#6  0xfef57b3c in MHD_select (daemon=0x80737a8, may_block=0) at daemon.c:1367
#7  0xfef57ff4 in MHD_select_thread (cls=0x80737a8) at daemon.c:1615
#8  0xfed5f0f3 in _thrp_setup () from /lib/libc.so.1
#9  0xfed5f3a0 in ?? () from /lib/libc.so.1
#10 0xfe9a0240 in ?? ()
#11 0x00000000 in ?? ()
(gdb) info threads
* 4 Thread 2 (LWP 2)  0xfed634f7 in __so_send () from /lib/libc.so.1
  3 LWP    2          0xfed634f7 in __so_send () from /lib/libc.so.1
  2 Thread 1 (LWP 1)  0xfed63e87 in __waitid () from /lib/libc.so.1
  1 LWP    1          0xfed63e87 in __waitid () from /lib/libc.so.1
(gdb) thread 2
[Switching to thread 2 (Thread 1 (LWP 1))]#0  0xfed63e87 in __waitid () from /lib/libc.so.1
(gdb) bt
#0  0xfed63e87 in __waitid () from /lib/libc.so.1
#1  0xfed515bd in waitid () from /lib/libc.so.1
#2  0xfed00035 in waitpid () from /lib/libc.so.1
#3  0x08051423 in kill_curl (pid=4479) at daemontest_get_response_cleanup.c:70
#4  0x0805157d in main (argc=1, argv=0x6b) at daemontest_get_response_cleanup.c:141

0001-use-a-pipe-to-signal-shutdown-to-select-and-poll.patch

1 files changed, 1 insertions, 0 deletions

diff --git a/AUTHORS b/AUTHORS
index 59dac571..6a59fd52 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -33,6 +33,7 @@ Eivind Sarto <ivan@espial.com>
 Thomas Stalder <gnunet@netsolux.ch>
 Zhimin Huang <zhimin.huang@bqvision.com>
 Jan Seeger <jan.seeger@thenybble.de>
+Will Bryant <will.bryant@gmail.com>
 
 Documentation contributions also came from:
 Marco Maggi <marco.maggi-ipsu@poste.it>