summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--README56
-rw-r--r--TODO150
-rw-r--r--contrib/defaults.conf8
-rw-r--r--src/datacache/Makefile.am70
-rw-r--r--src/datacache/perf_datacache.c52
-rw-r--r--src/datacache/perf_datacache_data_mysql.conf12
-rw-r--r--src/datacache/perf_datacache_data_sqlite.conf (renamed from src/datacache/perf_datacache_data.conf)0
-rw-r--r--src/datacache/plugin_datacache_mysql.c1092
-rw-r--r--src/datacache/test_datacache.c50
-rw-r--r--src/datacache/test_datacache_data_mysql.conf12
-rw-r--r--src/datacache/test_datacache_data_sqlite.conf (renamed from src/datacache/test_datacache_data.conf)0
-rw-r--r--src/datacache/test_datacache_quota.c50
-rw-r--r--src/datastore/Makefile.am4
-rw-r--r--src/datastore/plugin_datastore_mysql.c40
14 files changed, 1385 insertions, 211 deletions
diff --git a/README b/README
index 905d97d3b..2bf9f6e27 100644
--- a/README
+++ b/README
@@ -19,17 +19,23 @@ https://gnunet.org/.
Dependencies:
=============
-For the impatient, here is the list of immediate dependencies for
-running GNUnet:
+Please note that for many of its dependencies GNUnet requires very
+recent versions of the libraries which are often NOT to be found in
+stable distributions in 2010. While using older packages may in some
+cases on some operating systems may seem to work in some limited
+fashion, we are in many cases aware of serious problems with older
+packages. Hence please make sure to use the versions listed below.
+
+These are the direct dependencies for running GNUnet:
- libextractor >= 0.6.1
-- libmicrohttpd >= 0.4.6
+- libmicrohttpd >= 0.9.0
- libgcrypt >= 1.2
- libgmp >= 4.0
-- libcurl >= 7.15.4
+- libcurl >= 7.21.0
- libltdl >= 2.2 (part of GNU libtool)
- sqlite >= 3.0 (alternative to MySQL)
-- mysql >= ??? (not yet supported)
+- mysql >= 5.1 (alternative to sqLite)
- postgres >= ??? (not yet supported)
Recommended autotools for compiling the SVN version are:
@@ -37,8 +43,6 @@ Recommended autotools for compiling the SVN version are:
- automake >= 1.11.1
- libtool >= 2.2
-See also README.debian for a list of Debian packages.
-
How to install?
===============
@@ -47,6 +51,12 @@ The fastest way is to use a binary package if it is available for your
system. For a more detailed description, read the installation
instructions on the webpage at https://gnunet.org/installation.
+Note that some functions of GNUnet require "root" access. GNUnet will
+install (tiny) SUID binaries for those functions is you run "make
+install" as root. If you do not, GNUnet will still work, but some
+functionality will not be available (including certain forms of NAT
+traversal).
+
GNUnet requires the GNU MP library (http://www.gnu.org/software/gmp/)
and libgcrypt (http://www.gnupg.org/). You can specify the path to
libgcrypt by passing "--with-gcrypt=PATH" to configure. You will also
@@ -195,20 +205,30 @@ testcase to the Mantis bugtracking system at
https://gnunet.org/bugs/.
-Running http on port 80
-=======================
+Running http on port 80 and https on port 443
+=============================================
-In order to hide GNUnet's HTTP traffic perfectly, you might consider
-running GNUnet's HTTP transport on port 80. However, we do not
-recommend running GNUnet as root. Instead, forward port 80 to say
-8080 with this command (as root, in your startup scripts):
+In order to hide GNUnet's HTTP/HTTPS traffic perfectly, you might
+consider running GNUnet's HTTP/HTTPS transport on port 80/443.
+However, we do not recommend running GNUnet as root. Instead, forward
+port 80 to say 8080 with this command (as root, in your startup
+scripts):
# iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080
-Then set in the HTTP section of gnunet.conf the "ADVERTISED-PORT"
-to "80" and "PORT" to 8080. You can do the same trick for the
-TCP and UDP transports if you want to map them to a priviledged
-port (from the point of view of the network).
+or for HTTPS
+
+# iptables -t nat -A PREROUTING -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 4433
+
+Then set in the HTTP section of gnunet.conf the "ADVERTISED-PORT" to
+"80" and "PORT" to 8080 and similarly in the HTTPS section the
+"ADVERTISED-PORT" to "443" and "PORT" to 4433.
+
+You can do the same trick for the TCP and UDP transports if you want
+to map them to a priviledged port (from the point of view of the
+network). However, we are not aware of this providing any advantages
+at this point.
+
Running the SMTP transport
@@ -218,7 +238,7 @@ Running the SMTP transport
Running GNUnet over SMTP (e-mail) is a bit more involved. Note that
you do not need to run all transports (only running the NAT transport
is the only thing that will not work). If you really want to do
-P2P over SMTP, read the instructions at http://gnunet.org/smtp.php3
+P2P over SMTP, read the instructions at http://gnunet.org/smtp
Stay tuned
diff --git a/TODO b/TODO
index 369528688..c6906eef6 100644
--- a/TODO
+++ b/TODO
@@ -3,6 +3,7 @@
- only connect() sockets that are ready (select()) [Nils]
[On W32, we need to select after calling socket before doing connect etc.]
* CORE:
+ - derived key generation [Nils]
- Jun 27 11:51:54 core-7670 ERROR Assertion failed at gnunet-service-core.c:3616.
(transport notified us that we connected to ourselves!!!)
- transport-level disconnect (i.e. TCP) does not cause core-level
@@ -12,36 +13,34 @@
=> may have been fixed with instant-notification of disconnect
to core on session-oriented connection hick-up; not perfect but
likely good enough until we get ATS going; still should be tested...
- => "peers connected (transport)" now instantly goes to ZERO (core statistic),
+` => "peers connected (transport)" now instantly goes to ZERO (core statistic),
but "established sessions" stays up...
- - derived key generation [Nils]
-* PWNAT: [Nate/MW/Nils]
+ * PWNAT: [Nate/MW/Nils]
- W32 port
* GNUNET-GTK: [CG]
- - bugs:
- + handle 'lost parent' case for recursive downloads (need to move children!)
+ - handle 'lost parent' case for recursive downloads (need to move children!)
0.9.0pre3:
-* Determine RC bugs and fix those!
+* Determine RC bugs and fix those (release should have no known real bugs)
* DATASTORE: [LT]
- GNUNET_DATASTORE_cancel method not tested [LT]
* TESTING: [Nate]
- test basic peer re-configure
- - consider changing API for peer-group termination to
- call continuation when done
-* TOPOLOGY:
+* TOPOLOGY: [Nate]
- needs more testing (especially F2F topology) & transport blacklisting
+* TRANSPORT-TCP [MW]:
+ - should use hash map to look up sessions
* NAT/UPNP: [MW]
- finalize API design
- code clean up
- testing
- integration with transport service
-* MYSQL database backends: [CG]
- - datacache
+ - also do UPnP-based (external) IP detection
+ (Note: build library always, build UPnP service when dependencies like libxml2 are available)
* FS: [CG]
- library:
+ reconstruct IBLOCKS from DBLOCKS if possible (during download; see FIXME in fs_download)
- + add support for pushing "already seen" search results to FS service for bloomfilter (can wait)
+ + add support for pushing "already seen" search results to FS service for bloomfilter
+ use different 'priority' for probe downloads vs. normal downloads
- service:
+ trust: do not charge when "idle" / load considerations (migration, routing)
@@ -56,25 +55,26 @@
+ download
+ search
+ unindex
-* ARM: [CG/Safey]
- - better tracking of which config changes actually need to cause process restarts by ARM.
- - handle gnunet-arm -k in combination with auto-start magic (what is the right thing here?)
- - discover dependencies between services
+ - re-implement gnunet-auto-share
* GNUNET-GTK:
- optimize meta-data for directories in 'add_dir_at_iter'
- - add progress dialog for 'add_dir_at_iter' scan (can take a while...)
- finish publish dialog details:
+ normalize keywords (edit subdialog)
+ set/view previews (edit subdialog)
- - add tool bar
- implement download by URI dialog; figure out where to display those downloads!
- figure out where in the GUI we should show active uploads/unindex operations and allow aborts
- implement unindex operation (use dialog with all indexed files for selection)
- - do meaningful update to status line (starting up, peer running, #connections, shutdown, ...)
- events:
+ search error
+ publish error
+ unindex error
+* POSTGRES database backends: [CG]
+ - datacache
+ - datastore
+* ARM: [CG/Safey]
+ - better tracking of which config changes actually need to cause process restarts by ARM.
+ - handle gnunet-arm -k in combination with auto-start magic (what is the right thing here?)
+ - discover dependencies between services
* MONKEY: [Safey]
- better crash management (attach debugging support, capture and analyze
debug output, detect random vs. deterministic crashes)
@@ -82,38 +82,26 @@
- '-f FILENAME' option to write report to file instead of e-mail (for testing!)
0.9.0:
-* new webpage:
+* Determine RC bugs and fix those (release should have no known real bugs)
+* new webpage: [BL]
- convert documentation pages to books
- update books (especially for developers)
- make a NICE download page and figure out how to enable developers to publish TGZs nicely
- port "contact" page
- - add content type for "todo" items?
-* POSTGRES database backends: [CG]
- - datacache
- - datastore
-* Determine RC bugs and fix those!
-* SETUP:
+ - add content type for "todo" items
+* SETUP: [CG]
- design & implement new setup tool
* TBENCH: [MW]
- good to have for transport/DV evaluation!
* TRACEKIT: [MW]
- - good to have for DV/DHT evaluation!
-* DV:
- - performance tests
+ - good to have for DHT evaluation!
* DHT: [Nate]
- performance tests
-* STATISTICS:
- - test notification-based statistics API [LT]
- - implement statistics GUI
-* PEERINFO: [NN]
- - move peerinfo to new GUI?
- - extend peer dialog with green-yellow-red connectivity status lights
- - extend peer dialog with country flags and names
-0.9.x:
+0.9.1:
* TRANSPORT: [MW]
- WiFi transport backend [DB]
- - SMTP transport backend
+ - implement gnunet-transport (transport configurator / tester)
- Implement method of learning our external addresses from
other peers; need some kind of threshold-based
scheme, limiting both the total number of addresses that we accept
@@ -129,50 +117,70 @@
a way to easily "veto" addresses off the list!
=> If MiM attacker uses vetoed address, blacklist the specific IP for
the presumed neighbour!
- - implement gnunet-transport (transport configurator / tester)
- - UPnP-based IP detection
- (Note: build library always, build service when libxml2/etc. are available)
-* DV:
- - proper bandwidth allocation
-* FS: [CG]
- - Remove KBlocks in gnunet-unindex (see discussion with Kenneth Almquist on gnunet-devs in 9/2009)
-* PEERINFO: [NN]
- - expire 'ancient' HELLOs (those without valid addresses AND that
- we have not 'used' (for their public keys) in a while; need a way
- to track actual 'use')
- - make sue we also trigger notifications whenever HELLOs expire
-* VPN [PT]
-* UTIL: [CG]
- - allow limiting UNIX socket access by UID/GID
-
-
-
-Optimizations:
-* TCP:
- - should use hash map to look up sessions
-* STATISTICS:
- - should use BIO instead of mmap
-* TRANSPORT:
- need to periodically probe latency/transport cost changes & possibly switch transport
- should use hash map to look up Neighbours (service AND plugins!)
+* DV: [Nate]
+ - proper bandwidth allocation
+ - performance tests
* PEERINFO:
- merge multiple HELLOs of the same peer in the transmission queue
(theoretically reduces overhead; bounds message queue size)
- merge multiple iteration requests over "all" peers in the queue
(theoretically reduces overhead; bounds messgae queue size)
-* FS:
- - use different queue prioritization for probe-downloads vs. normal downloads (!?)
+* STATISTICS: [CG]
+ - should use BIO instead of mmap
+* FS: [CG]
+ - Remove KBlocks in gnunet-unindex (see discussion with Kenneth Almquist on gnunet-devs in 9/2009)
+ - use different queue prioritization for probe-downloads vs. normal downloads
+* UTIL: [CG]
+ - allow limiting UNIX socket access by UID/GID
+* GNUNET-GTK: [CG]
+ - add tool bar
+ - do meaningful update to status line (starting up, peer running, #connections, shutdown, ...)
+ - add progress dialog for 'add_dir_at_iter' scan (can take a while...)
+ - NS list in search dialog should use colors to offset our own namespaces from the others
+ - right-clicking on NS list in search dialog should open menu that allows
+ * viewing full meta data
+ * deletion of namespace info
-Minor features:
+0.9.2:
+* PEERINFO: [NN]
+ - expire 'ancient' HELLOs (those without valid addresses AND that
+ we have not 'used' (for their public keys) in a while; need a way
+ to track actual 'use')
+ - make sue we also trigger notifications whenever HELLOs expire
* TCP:
- repeatedly resolve hostname and look up interfaces to determine our own IP
- [./transport/plugin_transport_tcp.c:391]: (style) struct or union member 'Plugin::address_update_task' is never used (related to issue above)
* TRANSPORT:
- [./transport/gnunet-service-transport.c:173]: (style) struct or union member 'TransportPlugin::rebuild' is never used (related to TCP not refreshing external addresses?)
+ - WiFi transport backend
+ * nice signal strength adjustment [MW]
+ * energy cost in ATS [MW]
* BLOCKS:
- - testcase would be nice...
-* GNUNET-GTK:
- - NS list in search dialog should use colors to offset our own namespaces from the others
- - right-clicking on NS list in search dialog should open menu that allows
- * viewing full meta data
- * deletion of namespace info
+ - testcase would be nice
+ - generic block support for DHT
+* STATISTICS:
+ - test notification-based statistics API [LT]
+ - implement statistics GUI (=> start from gnunet-gtk by button!)
+* PEERINFO: [NN]
+ - move peerinfo to new GUI (=> start from gnunet-gtk by button!)
+ - extend peer dialog with green-yellow-red connectivity status lights
+ - extend peer dialog with country flags and names
+
+0.9.3:
+* SMTP transport backend:
+ - sending (SMTP/SMTPS)
+ - receiving (IMAP/IMAPS/POP?)
+ - rate limiting
+ - improved batching
+ - resource limit integration with ATS
+* VPN [PT]
+ - DNS hijacking
+ - DNS exit
+ - TCP entry/exit
+ - UDP entry/exit
+ - internal services
+ - integration with DHT routing
+ - optimized routes (beyond DHT/DV)
+ - "DNS" .gnunet
diff --git a/contrib/defaults.conf b/contrib/defaults.conf
index 74efa2260..94d64d932 100644
--- a/contrib/defaults.conf
+++ b/contrib/defaults.conf
@@ -205,6 +205,14 @@ USER = gnunet
# HOST =
# PORT =
+[datacache-mysql]
+DATABASE = gnunetcheck
+# CONFIG = ~/.my.cnf
+USER = gnunet
+# PASSWORD =
+# HOST =
+# PORT =
+
[fs]
AUTOSTART = YES
INDEXDB = $SERVICEHOME/idxinfo.lst
diff --git a/src/datacache/Makefile.am b/src/datacache/Makefile.am
index a10343046..0598c2673 100644
--- a/src/datacache/Makefile.am
+++ b/src/datacache/Makefile.am
@@ -14,6 +14,9 @@ endif
if HAVE_SQLITE
SQLITE_PLUGIN = libgnunet_plugin_datacache_sqlite.la
endif
+if HAVE_MYSQL
+ MYSQL_PLUGIN = libgnunet_plugin_datacache_mysql.la
+endif
lib_LTLIBRARIES = \
libgnunetdatacache.la
@@ -31,6 +34,7 @@ libgnunetdatacache_la_LDFLAGS = \
plugin_LTLIBRARIES = \
$(SQLITE_PLUGIN) \
+ $(MYSQL_PLUGIN) \
libgnunet_plugin_datacache_template.la
@@ -42,6 +46,17 @@ libgnunet_plugin_datacache_sqlite_la_LIBADD = \
libgnunet_plugin_datacache_sqlite_la_LDFLAGS = \
$(GN_PLUGIN_LDFLAGS)
+libgnunet_plugin_datacache_mysql_la_SOURCES = \
+ plugin_datacache_mysql.c
+libgnunet_plugin_datacache_mysql_la_LIBADD = \
+ $(top_builddir)/src/statistics/libgnunetstatistics.la \
+ $(top_builddir)/src/util/libgnunetutil.la \
+ $(GN_PLUGIN_LDFLAGS) $(MYSQL_LDFLAGS) -lmysqlclient
+libgnunet_plugin_datacache_mysql_la_CPPFLAGS = \
+ $(MYSQL_CPPFLAGS)
+libgnunet_plugin_datacache_mysql_la_LDFLAGS = \
+ $(GN_PLUGIN_LDFLAGS) $(MYSQL_LDFLAGS) -lmysqlclient
+
libgnunet_plugin_datacache_template_la_SOURCES = \
plugin_datacache_template.c
libgnunet_plugin_datacache_template_la_LIBADD = \
@@ -51,36 +66,65 @@ libgnunet_plugin_datacache_template_la_LDFLAGS = \
if HAVE_SQLITE
- SQLITE_TESTS = \
- test_datacache \
- test_datacache_quota \
- perf_datacache
+SQLITE_TESTS = \
+ test_datacache_sqlite \
+ test_datacache_quota_sqlite \
+ perf_datacache_sqlite
endif
-check_PROGRAMS = $(SQLITE_TESTS)
+if HAVE_MYSQL
+MYSQL_TESTS = \
+ test_datacache_mysql \
+ test_datacache_quota_mysql \
+ perf_datacache_mysql
+endif
+
+check_PROGRAMS = \
+ $(SQLITE_TESTS) \
+ $(MYSQL_TESTS)
if !DISABLE_TEST_RUN
TESTS = $(check_PROGRAMS)
endif
-test_datacache_SOURCES = \
+test_datacache_sqlite_SOURCES = \
+ test_datacache.c
+test_datacache_sqlite_LDADD = \
+ $(top_builddir)/src/datacache/libgnunetdatacache.la \
+ $(top_builddir)/src/util/libgnunetutil.la
+
+test_datacache_quota_sqlite_SOURCES = \
+ test_datacache_quota.c
+test_datacache_quota_sqlite_LDADD = \
+ $(top_builddir)/src/datacache/libgnunetdatacache.la \
+ $(top_builddir)/src/util/libgnunetutil.la
+
+perf_datacache_sqlite_SOURCES = \
+ perf_datacache.c
+perf_datacache_sqlite_LDADD = \
+ $(top_builddir)/src/datacache/libgnunetdatacache.la \
+ $(top_builddir)/src/util/libgnunetutil.la
+
+test_datacache_mysql_SOURCES = \
test_datacache.c
-test_datacache_LDADD = \
+test_datacache_mysql_LDADD = \
$(top_builddir)/src/datacache/libgnunetdatacache.la \
$(top_builddir)/src/util/libgnunetutil.la
-test_datacache_quota_SOURCES = \
+test_datacache_quota_mysql_SOURCES = \
test_datacache_quota.c
-test_datacache_quota_LDADD = \
+test_datacache_quota_mysql_LDADD = \
$(top_builddir)/src/datacache/libgnunetdatacache.la \
$(top_builddir)/src/util/libgnunetutil.la
-perf_datacache_SOURCES = \
+perf_datacache_mysql_SOURCES = \
perf_datacache.c
-perf_datacache_LDADD = \
+perf_datacache_mysql_LDADD = \
$(top_builddir)/src/datacache/libgnunetdatacache.la \
$(top_builddir)/src/util/libgnunetutil.la
EXTRA_DIST = \
- test_datacache_data.conf \
- perf_datacache_data.conf
+ test_datacache_data_sqlite.conf \
+ perf_datacache_data_sqlite.conf \
+ test_datacache_data_mysql.conf \
+ perf_datacache_data_mysql.conf
diff --git a/src/datacache/perf_datacache.c b/src/datacache/perf_datacache.c
index a577927d0..e2eb474d4 100644
--- a/src/datacache/perf_datacache.c
+++ b/src/datacache/perf_datacache.c
@@ -1,6 +1,6 @@
/*
This file is part of GNUnet.
- (C) 2006, 2009 Christian Grothoff (and other contributing authors)
+ (C) 2006, 2009, 2010 Christian Grothoff (and other contributing authors)
GNUnet is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published
@@ -36,6 +36,12 @@ static int ok;
static unsigned int found;
+/**
+ * Name of plugin under test.
+ */
+static const char *plugin_name;
+
+
static int
checkIt (void *cls,
struct GNUNET_TIME_Absolute exp,
@@ -119,12 +125,15 @@ FAILURE:
}
-static int
-check ()
+int
+main (int argc, char *argv[])
{
- char *const argv[] = { "perf-datacache-api",
+ const char *pos;
+ char cfg_name[128];
+ char *const xargv[] = {
+ "perf-datacache",
"-c",
- "perf_datacache_data.conf",
+ cfg_name,
#if VERBOSE
"-L", "DEBUG",
#endif
@@ -133,31 +142,28 @@ check ()
struct GNUNET_GETOPT_CommandLineOption options[] = {
GNUNET_GETOPT_OPTION_END
};
- GNUNET_PROGRAM_run ((sizeof (argv) / sizeof (char *)) - 1,
- argv, "perf-datacache-api", "nohelp",
- options, &run, NULL);
- if (ok != 0)
- fprintf (stderr, "Missed some perfcases: %d\n", ok);
- return ok;
-}
-
-
-int
-main (int argc, char *argv[])
-{
- int ret;
- GNUNET_DISK_directory_remove ("/tmp/perf-gnunetd-datacache");
- GNUNET_log_setup ("perf-datacache-api",
+ GNUNET_log_setup ("perf-datacache",
#if VERBOSE
"DEBUG",
#else
"WARNING",
#endif
NULL);
- ret = check ();
-
- return ret;
+ /* determine name of plugin to use */
+ plugin_name = argv[0];
+ while (NULL != (pos = strstr(plugin_name, "_")))
+ plugin_name = pos+1;
+ GNUNET_snprintf (cfg_name,
+ sizeof (cfg_name),
+ "perf_datacache_data_%s.conf",
+ plugin_name);
+ GNUNET_PROGRAM_run ((sizeof (xargv) / sizeof (char *)) - 1,
+ xargv, "perf-datacache", "nohelp",
+ options, &run, NULL);
+ if (ok != 0)
+ fprintf (stderr, "Missed some perfcases: %d\n", ok);
+ return ok;
}
/* end of perf_datacache.c */
diff --git a/src/datacache/perf_datacache_data_mysql.conf b/src/datacache/perf_datacache_data_mysql.conf
new file mode 100644
index 000000000..5a5f97c63
--- /dev/null
+++ b/src/datacache/perf_datacache_data_mysql.conf
@@ -0,0 +1,12 @@
+
+[perfcache]
+QUOTA = 500000
+DATABASE = mysql
+
+[datacache-mysql]
+DATABASE = gnunetcheck
+# CONFIG = ~/.my.cnf
+# USER =
+# PASSWORD =
+# HOST =
+# PORT = \ No newline at end of file
diff --git a/src/datacache/perf_datacache_data.conf b/src/datacache/perf_datacache_data_sqlite.conf
index 55e178d99..55e178d99 100644
--- a/src/datacache/perf_datacache_data.conf
+++ b/src/datacache/perf_datacache_data_sqlite.conf
diff --git a/src/datacache/plugin_datacache_mysql.c b/src/datacache/plugin_datacache_mysql.c
new file mode 100644
index 000000000..4559a9d39
--- /dev/null
+++ b/src/datacache/plugin_datacache_mysql.c
@@ -0,0 +1,1092 @@
+/*
+ This file is part of GNUnet
+ (C) 2006, 2009, 2010 Christian Grothoff (and other contributing authors)
+
+ GNUnet is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published
+ by the Free Software Foundation; either version 3, or (at your
+ option) any later version.
+
+ GNUnet is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with GNUnet; see the file COPYING. If not, write to the
+ Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ Boston, MA 02111-1307, USA.
+*/
+
+/**
+ * @file datacache/plugin_datacache_mysql.c
+ * @brief mysql for an implementation of a database backend for the datacache
+ * @author Christian Grothoff
+ *
+ * SETUP INSTRUCTIONS:
+ *
+ * 1) Access mysql as root,
+ * <pre>
+ *
+ * $ mysql -u root -p
+ *
+ * </pre>
+ * and do the following. [You should replace $USER with the username
+ * that will be running the gnunetd process].
+ * <pre>
+ *
+ CREATE DATABASE gnunet;
+ GRANT select,insert,update,delete,create,alter,drop,create temporary tables
+ ON gnunet.* TO $USER@localhost;
+ SET PASSWORD FOR $USER@localhost=PASSWORD('$the_password_you_like');
+ FLUSH PRIVILEGES;
+ *
+ * </pre>
+ * 2) In the $HOME directory of $USER, create a ".my.cnf" file
+ * with the following lines
+ * <pre>
+
+ [client]
+ user=$USER
+ password=$the_password_you_like
+
+ * </pre>
+ *
+ * Thats it -- now you can configure your datastores in GNUnet to
+ * use MySQL. Note that .my.cnf file is a security risk unless its on
+ * a safe partition etc. The $HOME/.my.cnf can of course be a symbolic
+ * link. Even greater security risk can be achieved by setting no
+ * password for $USER. Luckily $USER has only priviledges to mess
+ * up GNUnet's tables, nothing else (unless you give him more,
+ * of course).<p>
+ *
+ * 3) Still, perhaps you should briefly try if the DB connection
+ * works. First, login as $USER. Then use,
+ *
+ * <pre>
+ * $ mysql -u $USER -p $the_password_you_like
+ * mysql> use gnunet;
+ * </pre>
+ *
+ * If you get the message &quot;Database changed&quot; it probably works.
+ *
+ * [If you get &quot;ERROR 2002: Can't connect to local MySQL server
+ * through socket '/tmp/mysql.sock' (2)&quot; it may be resolvable by
+ * &quot;ln -s /var/run/mysqld/mysqld.sock /tmp/mysql.sock&quot;
+ * so there may be some additional trouble depending on your mysql setup.]
+ *
+ * REPAIRING TABLES:
+ * - Its probably healthy to check your tables for inconsistencies
+ * every now and then.
+ * - If you get odd SEGVs on gnunetd startup, it might be that the mysql
+ * databases have been corrupted.
+ * - The tables can be verified/fixed in two ways;
+ * 1) by running mysqlcheck -A, or
+ * 2) by executing (inside of mysql using the GNUnet database):
+ * mysql> SHOW TABLES;
+ * mysql> REPAIR TABLE gnXXX;
+ *
+ * Make sure to replace XXX with the actual names of all tables.
+ *
+ * PROBLEMS?
+ *
+ * If you have problems related to the mysql module, your best
+ * friend is probably the mysql manual. The first thing to check
+ * is that mysql is basically operational, that you can connect
+ * to it, create tables, issue queries etc.
+ */
+#include "platform.h"
+#include "gnunet_util_lib.h"
+#include "plugin_datacache.h"
+#include <mysql/mysql.h>
+
+#define DEBUG_DATACACHE_MYSQL GNUNET_NO
+
+/**
+ * Estimate of the per-entry overhead (including indices).
+ */
+#define OVERHEAD ((4*2+4*2+8*2+8*2+sizeof(GNUNET_HashCode)*5+8))
+
+/**
+ * Maximum number of supported parameters for a prepared
+ * statement. Increase if needed.
+ */
+#define MAX_PARAM 16
+
+/**
+ * Die with an error message that indicates
+ * a failure of the command 'cmd' with the message given
+ * by strerror(errno).
+ */
+#define DIE_MYSQL(cmd, dbh) do { GNUNET_log(GNUNET_ERROR_TYPE__ERROR, _("`%s' failed at %s:%d with error: %s\n"), cmd, __FILE__, __LINE__, mysql_error((dbh)->dbf)); abort(); } while(0);
+
+/**
+ * Log an error message at log-level 'level' that indicates
+ * a failure of the command 'cmd' on file 'filename'
+ * with the message given by strerror(errno).
+ */
+#define LOG_MYSQL(level, cmd, dbh) do { GNUNET_log(level, _("`%s' failed at %s:%d with error: %s\n"), cmd, __FILE__, __LINE__, mysql_error((dbh)->dbf)); } while(0);
+
+struct GNUNET_MysqlStatementHandle
+{
+ struct GNUNET_MysqlStatementHandle *next;
+
+ struct GNUNET_MysqlStatementHandle *prev;
+
+ char *query;
+
+ MYSQL_STMT *statement;
+
+ int valid;
+
+};
+
+
+/**
+ * Context for all functions in this plugin.
+ */
+struct Plugin
+{
+ /**
+ * Our execution environment.
+ */
+ struct GNUNET_DATACACHE_PluginEnvironment *env;
+
+ /**
+ * Handle to the mysql database.
+ */
+ MYSQL *dbf;
+
+ struct GNUNET_MysqlStatementHandle *shead;
+
+ struct GNUNET_MysqlStatementHandle *stail;
+
+ /**
+ * Filename of "my.cnf" (msyql configuration).
+ */
+ char *cnffile;
+
+#define SELECT_VALUE_STMT "SELECT value,expire FROM gn080dstore FORCE INDEX (hashidx) WHERE hash=? AND type=? AND expire >= ? LIMIT 1 OFFSET ?"
+ struct GNUNET_MysqlStatementHandle *select_value;
+
+#define COUNT_VALUE_STMT "SELECT count(*) FROM gn080dstore FORCE INDEX (hashidx) WHERE hash=? AND type=? AND expire >= ?"
+ struct GNUNET_MysqlStatementHandle *count_value;
+
+#define SELECT_OLD_VALUE_STMT "SELECT hash, vhash, type, value FROM gn080dstore FORCE INDEX (expireidx) ORDER BY puttime ASC LIMIT 1"
+ struct GNUNET_MysqlStatementHandle *select_old_value;
+
+#define DELETE_VALUE_STMT "DELETE FROM gn080dstore WHERE hash = ? AND vhash = ? AND type = ? AND value = ?"
+ struct GNUNET_MysqlStatementHandle *delete_value;
+
+#define INSERT_VALUE_STMT "INSERT INTO gn080dstore (type, puttime, expire, hash, vhash, value) "\
+ "VALUES (?, ?, ?, ?, ?, ?)"
+ struct GNUNET_MysqlStatementHandle *insert_value;
+
+#define UPDATE_VALUE_STMT "UPDATE gn080dstore FORCE INDEX (allidx) SET puttime=?, expire=? "\
+ "WHERE hash=? AND vhash=? AND type=?"
+ struct GNUNET_MysqlStatementHandle *update_value;
+
+};
+
+
+/**
+ * Obtain the location of ".my.cnf".
+ * @return NULL on error
+ */
+static char *
+get_my_cnf_path (const struct GNUNET_CONFIGURATION_Handle *cfg)
+{
+ char *cnffile;
+ char *home_dir;
+ struct stat st;
+#ifndef WINDOWS
+ struct passwd *pw;
+#endif
+ int configured;
+
+#ifndef WINDOWS
+ pw = getpwuid (getuid ());
+ if (!pw)
+ {
+ GNUNET_log_strerror (GNUNET_ERROR_TYPE_ERROR,
+ "getpwuid");
+ return NULL;
+ }
+ if (GNUNET_YES ==
+ GNUNET_CONFIGURATION_have_value (cfg,
+ "datacache-mysql", "CONFIG"))
+ {
+ GNUNET_assert (GNUNET_OK ==
+ GNUNET_CONFIGURATION_get_value_filename (cfg,
+ "datacache-mysql", "CONFIG", &cnffile));
+ configured = GNUNET_YES;
+ }
+ else
+ {
+ home_dir = GNUNET_strdup (pw->pw_dir);
+#else
+ home_dir = (char *) GNUNET_malloc (_MAX_PATH + 1);
+ plibc_conv_to_win_path ("~/", home_dir);
+#endif
+ GNUNET_asprintf (&cnffile, "%s/.my.cnf", home_dir);
+ GNUNET_free (home_dir);
+ configured = GNUNET_NO;
+ }
+ GNUNET_log (GNUNET_ERROR_TYPE_INFO,
+ _("Trying to use file `%s' for MySQL configuration.\n"),
+ cnffile);
+ if ((0 != STAT (cnffile, &st)) ||
+ (0 != ACCESS (cnffile, R_OK)) || (!S_ISREG (st.st_mode)))
+ {
+ if (configured == GNUNET_YES)
+ GNUNET_log (GNUNET_ERROR_TYPE_ERROR,
+ _("Could not access file `%s': %s\n"), cnffile,
+ STRERROR (errno));
+ GNUNET_free (cnffile);
+ return NULL;
+ }
+ return cnffile;
+}
+
+
+/**
+ * Free a prepared statement.
+ */
+static void
+prepared_statement_destroy (struct Plugin *plugin,
+ struct GNUNET_MysqlStatementHandle
+ *s)
+{
+ GNUNET_CONTAINER_DLL_remove (plugin->shead,
+ plugin->stail,
+ s);
+ if (s->valid)
+ mysql_stmt_close (s->statement);
+ GNUNET_free (s->query);
+ GNUNET_free (s);
+}
+
+
+/**
+ * Close database connection and all prepared statements (we got a DB
+ * disconnect error).
+ */
+static int
+iclose (struct Plugin *plugin)
+{
+ struct GNUNET_MysqlStatementHandle *spos;
+
+ spos = plugin->shead;
+ while (NULL != plugin->shead)
+ prepared_statement_destroy (plugin,
+ plugin->shead);
+ if (plugin->dbf != NULL)
+ {
+ mysql_close (plugin->dbf);
+ plugin->dbf = NULL;
+ }
+ return GNUNET_OK;
+}
+
+
+/**
+ * Open the connection with the database (and initialize
+ * our default options).
+ *
+ * @return GNUNET_OK on success
+ */
+static int
+iopen (struct Plugin *ret)
+{
+ char *mysql_dbname;
+ char *mysql_server;
+ char *mysql_user;
+ char *mysql_password;
+ unsigned long long mysql_port;
+ my_bool reconnect;
+ unsigned int timeout;
+
+ ret->dbf = mysql_init (NULL);
+ if (ret->dbf == NULL)
+ return GNUNET_SYSERR;
+ if (ret->cnffile != NULL)
+ mysql_options (ret->dbf, MYSQL_READ_DEFAULT_FILE, ret->cnffile);
+ mysql_options (ret->dbf, MYSQL_READ_DEFAULT_GROUP, "client");
+ reconnect = 0;
+ mysql_options (ret->dbf, MYSQL_OPT_RECONNECT, &reconnect);
+ mysql_options (ret->dbf,
+ MYSQL_OPT_CONNECT_TIMEOUT, (const void *) &timeout);
+ mysql_options(ret->dbf, MYSQL_SET_CHARSET_NAME, "UTF8");
+ timeout = 60; /* in seconds */
+ mysql_options (ret->dbf, MYSQL_OPT_READ_TIMEOUT, (const void *) &timeout);
+ mysql_options (ret->dbf, MYSQL_OPT_WRITE_TIMEOUT, (const void *) &timeout);
+ mysql_dbname = NULL;
+ if (GNUNET_YES == GNUNET_CONFIGURATION_have_value (ret->env->cfg,
+ "datacache-mysql", "DATABASE"))
+ GNUNET_assert (GNUNET_OK ==
+ GNUNET_CONFIGURATION_get_value_string (ret->env->cfg,
+ "datacache-mysql", "DATABASE",
+ &mysql_dbname));
+ else
+ mysql_dbname = GNUNET_strdup ("gnunet");
+ mysql_user = NULL;
+ if (GNUNET_YES == GNUNET_CONFIGURATION_have_value (ret->env->cfg,
+ "datacache-mysql", "USER"))
+ {
+ GNUNET_assert (GNUNET_OK ==
+ GNUNET_CONFIGURATION_get_value_string (ret->env->cfg,
+ "datacache-mysql", "USER",
+ &mysql_user));
+ }
+ mysql_password = NULL;
+ if (GNUNET_YES == GNUNET_CONFIGURATION_have_value (ret->env->cfg,
+ "datacache-mysql", "PASSWORD"))
+ {
+ GNUNET_assert (GNUNET_OK ==
+ GNUNET_CONFIGURATION_get_value_string (ret->env->cfg,
+ "datacache-mysql", "PASSWORD",
+ &mysql_password));
+ }
+ mysql_server = NULL;
+ if (GNUNET_YES == GNUNET_CONFIGURATION_have_value (ret->env->cfg,
+ "datacache-mysql", "HOST"))
+ {
+ GNUNET_assert (GNUNET_OK ==
+ GNUNET_CONFIGURATION_get_value_string (ret->env->cfg,
+ "datacache-mysql", "HOST",
+ &mysql_server));
+ }
+ mysql_port = 0;
+ if (GNUNET_YES == GNUNET_CONFIGURATION_have_value (ret->env->cfg,
+ "datacache-mysql", "PORT"))
+ {
+ GNUNET_assert (GNUNET_OK ==
+ GNUNET_CONFIGURATION_get_value_number (ret->env->cfg, "datacache-mysql",
+ "PORT", &mysql_port));
+ }
+
+ GNUNET_assert (mysql_dbname != NULL);
+ mysql_real_connect (ret->dbf, mysql_server, mysql_user, mysql_password,
+ mysql_dbname, (unsigned int) mysql_port, NULL,
+ CLIENT_IGNORE_SIGPIPE);
+ GNUNET_free_non_null (mysql_server);
+ GNUNET_free_non_null (mysql_user);
+ GNUNET_free_non_null (mysql_password);
+ GNUNET_free (mysql_dbname);
+ if (mysql_error (ret->dbf)[0])
+ {
+ LOG_MYSQL (GNUNET_ERROR_TYPE_ERROR,
+ "mysql_real_connect", ret);
+ return GNUNET_SYSERR;
+ }
+ return GNUNET_OK;
+}
+
+
+/**
+ * Run the given MySQL statement.
+ *
+ * @return GNUNET_OK on success, GNUNET_SYSERR on error
+ */
+static int
+run_statement (struct Plugin *plugin,
+ const char *statement)
+{
+ if ((NULL == plugin->dbf) && (GNUNET_OK != iopen (plugin)))
+ return GNUNET_SYSERR;
+ mysql_query (plugin->dbf, statement);
+ if (mysql_error (plugin->dbf)[0])
+ {
+ LOG_MYSQL (GNUNET_ERROR_TYPE_ERROR,
+ "mysql_query", plugin);
+ iclose (plugin);
+ return GNUNET_SYSERR;
+ }
+ return GNUNET_OK;
+}
+
+/**
+ * Create a prepared statement.
+ *
+ * @return NULL on error
+ */
+static struct GNUNET_MysqlStatementHandle *
+prepared_statement_create (struct Plugin *plugin,
+ const char *statement)
+{
+ struct GNUNET_MysqlStatementHandle *ret;
+
+ ret = GNUNET_malloc (sizeof (struct GNUNET_MysqlStatementHandle));
+ ret->query = GNUNET_strdup (statement);
+ GNUNET_CONTAINER_DLL_insert (plugin->shead,
+ plugin->stail,
+ ret);
+ return ret;
+}
+
+
+/**
+ * Prepare a statement for running.
+ *
+ * @return GNUNET_OK on success
+ */
+static int
+prepare_statement (struct Plugin *plugin,
+ struct GNUNET_MysqlStatementHandle *ret)
+{
+ if (GNUNET_YES == ret->valid)
+ return GNUNET_OK;
+ if ((NULL == plugin->dbf) &&
+ (GNUNET_OK != iopen (plugin)))
+ return GNUNET_SYSERR;
+ ret->statement = mysql_stmt_init (plugin->dbf);
+ if (ret->statement == NULL)
+ {
+ iclose (plugin);
+ return GNUNET_SYSERR;
+ }
+ if (mysql_stmt_prepare (ret->statement,
+ ret->query,
+ strlen (ret->query)))
+ {
+ LOG_MYSQL (GNUNET_ERROR_TYPE_ERROR,
+ "mysql_stmt_prepare",
+ plugin);
+ mysql_stmt_close (ret->statement);
+ ret->statement = NULL;
+ iclose (plugin);
+ return GNUNET_SYSERR;
+ }
+ ret->valid = GNUNET_YES;
+ return GNUNET_OK;
+
+}
+
+
+/**
+ * Bind the parameters for the given MySQL statement
+ * and run it.
+ *
+ * @param s statement to bind and run
+ * @param ap arguments for the binding
+ * @return GNUNET_SYSERR on error, GNUNET_OK on success
+ */
+static int
+init_params (struct Plugin *plugin,
+ struct GNUNET_MysqlStatementHandle *s,
+ va_list ap)
+{
+ MYSQL_BIND qbind[MAX_PARAM];
+ unsigned int pc;
+ unsigned int off;
+ enum enum_field_types ft;
+
+ pc = mysql_stmt_param_count (s->statement);
+ if (pc > MAX_PARAM)
+ {
+ /* increase internal constant! */
+ GNUNET_break (0);
+ return GNUNET_SYSERR;
+ }
+ memset (qbind, 0, sizeof (qbind));
+ off = 0;
+ ft = 0;
+ while ((pc > 0) && (-1 != (ft = va_arg (ap, enum enum_field_types))))
+ {
+ qbind[off].buffer_type = ft;
+ switch (ft)
+ {
+ case MYSQL_TYPE_FLOAT:
+ qbind[off].buffer = va_arg (ap, float *);
+ break;
+ case MYSQL_TYPE_LONGLONG:
+ qbind[off].buffer = va_arg (ap, unsigned long long *);
+ qbind[off].is_unsigned = va_arg (ap, int);
+ break;
+ case MYSQL_TYPE_LONG:
+ qbind[off].buffer = va_arg (ap, unsigned int *);
+ qbind[off].is_unsigned = va_arg (ap, int);
+ break;
+ case MYSQL_TYPE_VAR_STRING:
+ case MYSQL_TYPE_STRING:
+ case MYSQL_TYPE_BLOB:
+ qbind[off].buffer = va_arg (ap, void *);
+ qbind[off].buffer_length = va_arg (ap, unsigned long);
+ qbind[off].length = va_arg (ap, unsigned long *);
+ break;
+ default:
+ /* unsupported type */
+ GNUNET_break (0);
+ return GNUNET_SYSERR;
+ }
+ pc--;
+ off++;
+ }
+ if (!((pc == 0) && (ft != -1) && (va_arg (ap, int) == -1)))
+ {
+ GNUNET_break (0);
+ return GNUNET_SYSERR;
+ }
+ if (mysql_stmt_bind_param (s->statement, qbind))
+ {
+ GNUNET_log (GNUNET_ERROR_TYPE_ERROR,
+ _("`%s' failed at %s:%d with error: %s\n"),
+ "mysql_stmt_bind_param",
+ __FILE__, __LINE__, mysql_stmt_error (s->statement));
+ iclose (plugin);
+ return GNUNET_SYSERR;
+ }
+ if (mysql_stmt_execute (s->statement))
+ {
+ GNUNET_log (GNUNET_ERROR_TYPE_ERROR,
+ _("`%s' failed at %s:%d with error: %s\n"),
+ "mysql_stmt_execute",
+ __FILE__, __LINE__, mysql_stmt_error (s->statement));
+ iclose (plugin);
+ return GNUNET_SYSERR;
+ }
+ return GNUNET_OK;
+}
+
+/**
+ * Type of a callback that will be called for each
+ * data set returned from MySQL.
+ *
+ * @param cls user-defined argument
+ * @param num_values number of elements in values
+ * @param values values returned by MySQL
+ * @return GNUNET_OK to continue iterating, GNUNET_SYSERR to abort
+ */
+typedef int (*GNUNET_MysqlDataProcessor) (void *cls,
+ unsigned int num_values,
+ MYSQL_BIND * values);
+
+
+/**
+ * Run a prepared SELECT statement.
+ *
+ * @param result_size number of elements in results array
+ * @param results pointer to already initialized MYSQL_BIND
+ * array (of sufficient size) for passing results
+ * @param processor function to call on each result
+ * @param processor_cls extra argument to processor
+ * @param ... pairs and triplets of "MYSQL_TYPE_XXX" keys and their respective
+ * values (size + buffer-reference for pointers); terminated
+ * with "-1"
+ * @return GNUNET_SYSERR on error, otherwise
+ * the number of successfully affected (or queried) rows
+ */
+static int
+prepared_statement_run_select (struct Plugin *plugin,
+ struct GNUNET_MysqlStatementHandle
+ *s,
+ unsigned int result_size,
+ MYSQL_BIND * results,
+ GNUNET_MysqlDataProcessor
+ processor, void *processor_cls,
+ ...)
+{
+ va_list ap;
+ int ret;
+ unsigned int rsize;
+ int total;
+
+ if (GNUNET_OK != prepare_statement (plugin, s))
+ {
+ GNUNET_break (0);
+ return GNUNET_SYSERR;
+ }
+ va_start (ap, processor_cls);
+ if (GNUNET_OK != init_params (plugin, s, ap))
+ {
+ GNUNET_break (0);
+ va_end (ap);
+ return GNUNET_SYSERR;
+ }
+ va_end (ap);
+ rsize = mysql_stmt_field_count (s->statement);
+ if (rsize > result_size)
+ {
+ GNUNET_break (0);
+ return GNUNET_SYSERR;
+ }
+ if (mysql_stmt_bind_result (s->statement, results))
+ {
+ GNUNET_log (GNUNET_ERROR_TYPE_ERROR,
+ _("`%s' failed at %s:%d with error: %s\n"),
+ "mysql_stmt_bind_result",
+ __FILE__, __LINE__, mysql_stmt_error (s->statement));
+ iclose (plugin);
+ return GNUNET_SYSERR;
+ }
+
+ total = 0;
+ while (1)
+ {
+ ret = mysql_stmt_fetch (s->statement);
+ if (ret == MYSQL_NO_DATA)
+ break;
+ if (ret != 0)
+ {
+ GNUNET_log (GNUNET_ERROR_TYPE_ERROR,
+ _("`%s' failed at %s:%d with error: %s\n"),
+ "mysql_stmt_fetch",
+ __FILE__, __LINE__, mysql_stmt_error (s->statement));
+ iclose (plugin);
+ return GNUNET_SYSERR;
+ }
+ if (processor != NULL)
+ if (GNUNET_OK != processor (processor_cls, rsize, results))
+ break;
+ total++;
+ }
+ mysql_stmt_reset (s->statement);
+ return total;
+}
+
+
+
+/**
+ * Run a prepared statement that does NOT produce results.
+ *
+ * @param ... pairs and triplets of "MYSQL_TYPE_XXX" keys and their respective
+ * values (size + buffer-reference for pointers); terminated
+ * with "-1"
+ * @param insert_id NULL or address where to store the row ID of whatever
+ * was inserted (only for INSERT statements!)
+ * @return GNUNET_SYSERR on error, otherwise
+ * the number of successfully affected rows
+ */
+static int
+prepared_statement_run (struct Plugin *plugin,
+ struct GNUNET_MysqlStatementHandle *s,
+ unsigned long long *insert_id, ...)
+{
+ va_list ap;
+ int affected;
+
+ if (GNUNET_OK != prepare_statement (plugin, s))
+ return GNUNET_SYSERR;
+ va_start (ap, insert_id);
+ if (GNUNET_OK != init_params (plugin, s, ap))
+ {
+ va_end (ap);
+ return GNUNET_SYSERR;
+ }
+ va_end (ap);
+ affected = mysql_stmt_affected_rows (s->statement);
+ if (NULL != insert_id)
+ *insert_id = (unsigned long long) mysql_stmt_insert_id (s->statement);
+ mysql_stmt_reset (s->statement);
+ return affected;
+}
+
+
+static int
+itable (struct Plugin *plugin)
+{
+#define MRUNS(a) (GNUNET_OK != run_statement (plugin, a) )
+ if (MRUNS ("CREATE TEMPORARY TABLE gn080dstore ("
+ " type INT(11) UNSIGNED NOT NULL DEFAULT 0,"
+ " puttime BIGINT UNSIGNED NOT NULL DEFAULT 0,"
+ " expire BIGINT UNSIGNED NOT NULL DEFAULT 0,"
+ " hash BINARY(64) NOT NULL DEFAULT '',"
+ " vhash BINARY(64) NOT NULL DEFAULT '',"
+ " value BLOB NOT NULL DEFAULT '',"
+ " INDEX hashidx (hash(64),type,expire),"
+ " INDEX allidx (hash(64),vhash(64),type),"
+ " INDEX expireidx (puttime)" ") ENGINE=InnoDB") ||
+ MRUNS ("SET AUTOCOMMIT = 1"))
+ return GNUNET_SYSERR;
+#undef MRUNS
+#define PINIT(a,b) (NULL == (a = prepared_statement_create(plugin, b)))
+ if (PINIT (plugin->select_value, SELECT_VALUE_STMT) ||
+ PINIT (plugin->count_value, COUNT_VALUE_STMT) ||
+ PINIT (plugin->select_old_value, SELECT_OLD_VALUE_STMT) ||
+ PINIT (plugin->delete_value, DELETE_VALUE_STMT) ||
+ PINIT (plugin->insert_value, INSERT_VALUE_STMT) ||
+ PINIT (plugin->update_value, UPDATE_VALUE_STMT))
+ return GNUNET_SYSERR;
+#undef PINIT
+ return GNUNET_OK;
+}
+
+
+/**
+ * Store an item in the datastore.
+ *
+ * @param cls closure (our "struct Plugin")
+ * @param key key to store data under
+ * @param size number of bytes in data
+ * @param data data to store
+ * @param type type of the value
+ * @param discard_time when to discard the value in any case
+ * @return 0 on error, number of bytes used otherwise
+ */
+static uint32_t
+mysql_plugin_put (void *cls,
+ const GNUNET_HashCode * key,
+ uint32_t size,
+ const char *data,
+ enum GNUNET_BLOCK_Type type,
+ struct GNUNET_TIME_Absolute discard_time)
+{
+ struct Plugin *plugin = cls;
+ struct GNUNET_TIME_Absolute now;
+ unsigned long k_length;
+ unsigned long h_length;
+ unsigned long v_length;
+ unsigned long long v_now;
+ unsigned long long v_discard_time;
+ unsigned int v_type;
+ GNUNET_HashCode vhash;
+ int ret;
+
+ if (size > GNUNET_SERVER_MAX_MESSAGE_SIZE)
+ return GNUNET_SYSERR;
+ GNUNET_CRYPTO_hash (data, size, &vhash);
+ now = GNUNET_TIME_absolute_get ();
+
+ /* first try UPDATE */
+ h_length = sizeof (GNUNET_HashCode);
+ k_length = sizeof (GNUNET_HashCode);
+ v_length = size;
+ v_type = type;
+ v_now = (unsigned long long) now.value;
+ v_discard_time = (unsigned long long) discard_time.value;
+ if (GNUNET_OK ==
+ prepared_statement_run (plugin,
+ plugin->update_value,
+ NULL,
+ MYSQL_TYPE_LONGLONG,
+ &v_now,
+ GNUNET_YES,
+ MYSQL_TYPE_LONGLONG,
+ &v_discard_time,
+ GNUNET_YES,
+ MYSQL_TYPE_BLOB,
+ key,
+ sizeof (GNUNET_HashCode),
+ &k_length,
+ MYSQL_TYPE_BLOB,
+ &vhash,
+ sizeof (GNUNET_HashCode),
+ &h_length,
+ MYSQL_TYPE_LONG,
+ &v_type,
+ GNUNET_YES, -1))
+ return GNUNET_OK;
+
+ /* now try INSERT */
+ h_length = sizeof (GNUNET_HashCode);
+ k_length = sizeof (GNUNET_HashCode);
+ v_length = size;
+ if (GNUNET_OK !=
+ (ret = prepared_statement_run (plugin,
+ plugin->insert_value,
+ NULL,
+ MYSQL_TYPE_LONG,
+ &type,
+ GNUNET_YES,
+ MYSQL_TYPE_LONGLONG,
+ &v_now,
+ GNUNET_YES,
+ MYSQL_TYPE_LONGLONG,
+ &v_discard_time,
+ GNUNET_YES,
+ MYSQL_TYPE_BLOB,
+ key,
+ sizeof (GNUNET_HashCode),
+ &k_length,
+ MYSQL_TYPE_BLOB,
+ &vhash,
+ sizeof (GNUNET_HashCode),
+ &h_length,
+ MYSQL_TYPE_BLOB,
+ data,
+ (unsigned long) size,
+ &v_length, -1)))
+ {
+ if (ret == GNUNET_SYSERR)
+ itable (plugin);
+ return GNUNET_SYSERR;
+ }
+ return size + OVERHEAD;
+}
+
+
+static int
+return_ok (void *cls, unsigned int num_values, MYSQL_BIND * values)
+{
+ return GNUNET_OK;
+}
+
+
+/**
+ * Iterate over the results for a particular key
+ * in the datastore.
+ *
+ * @param cls closure (our "struct Plugin")
+ * @param key
+ * @param type entries of which type are relevant?
+ * @param iter maybe NULL (to just count)
+ * @param iter_cls closure for iter
+ * @return the number of results found
+ */
+static unsigned int
+mysql_plugin_get (void *cls,
+ const GNUNET_HashCode * key,
+ enum GNUNET_BLOCK_Type type,
+ GNUNET_DATACACHE_Iterator iter,
+ void *iter_cls)
+{
+ struct Plugin *plugin = cls;
+ MYSQL_BIND rbind[3];
+ unsigned long h_length;
+ unsigned long v_length;
+ unsigned long long v_expire;
+ struct GNUNET_TIME_Absolute now;
+ struct GNUNET_TIME_Absolute expire;
+ unsigned int cnt;
+ unsigned long long total;
+ unsigned long long v_now;
+ unsigned int off;
+ unsigned int v_type;
+ int ret;
+ char buffer[GNUNET_SERVER_MAX_MESSAGE_SIZE];
+
+ now = GNUNET_TIME_absolute_get ();
+ h_length = sizeof (GNUNET_HashCode);
+ v_length = sizeof (buffer);
+ total = -1;
+ memset (rbind, 0, sizeof (rbind));
+ rbind[0].buffer_type = MYSQL_TYPE_LONGLONG;
+ rbind[0].buffer = &total;
+ rbind[0].is_unsigned = GNUNET_YES;
+ v_type = type;
+ v_now = (unsigned long long) now.value;
+ if ((GNUNET_OK !=
+ (ret = prepared_statement_run_select (plugin,
+ plugin->count_value,
+ 1,
+ rbind,
+ return_ok,
+ NULL,
+ MYSQL_TYPE_BLOB,
+ key,
+ sizeof
+ (GNUNET_HashCode),
+ &h_length,
+ MYSQL_TYPE_LONG,
+ &v_type, GNUNET_YES,
+ MYSQL_TYPE_LONGLONG,
+ &v_now, GNUNET_YES,
+ -1)))
+ || (-1 == total))
+ {
+ if (ret == GNUNET_SYSERR)
+ itable (plugin);
+ return GNUNET_SYSERR;
+ }
+ if ((iter == NULL) || (total == 0))
+ return (int) total;
+
+ off = GNUNET_CRYPTO_random_u32 (GNUNET_CRYPTO_QUALITY_WEAK, total);
+ cnt = 0;
+ while (cnt < total)
+ {
+ memset (rbind, 0, sizeof (rbind));
+ rbind[0].buffer_type = MYSQL_TYPE_BLOB;
+ rbind[0].buffer_length = sizeof (buffer);
+ rbind[0].length = &v_length;
+ rbind[0].buffer = buffer;
+ rbind[1].buffer_type = MYSQL_TYPE_LONGLONG;
+ rbind[1].is_unsigned = 1;
+ rbind[1].buffer = &v_expire;
+ off = (off + 1) % total;
+ if (GNUNET_OK !=
+ (ret = prepared_statement_run_select (plugin,
+ plugin->select_value,
+ 2,
+ rbind,
+ return_ok,
+ NULL,
+ MYSQL_TYPE_BLOB,
+ key,
+ sizeof
+ (GNUNET_HashCode),
+ &h_length,
+ MYSQL_TYPE_LONG,
+ &v_type,
+ GNUNET_YES,
+ MYSQL_TYPE_LONGLONG,
+ &v_now,
+ GNUNET_YES,
+ MYSQL_TYPE_LONG,
+ &off,
+ GNUNET_YES,
+ -1)))
+ {
+ if (ret == GNUNET_SYSERR)
+ itable (plugin);
+ return GNUNET_SYSERR;
+ }
+ cnt++;
+ expire.value = v_expire;
+ if (GNUNET_OK != iter (iter_cls,
+ expire,
+ key,
+ v_length, buffer,
+ type))
+ break;
+ }
+ return cnt;
+}
+
+
+/**
+ * Delete the entry with the lowest expiration value
+ * from the datacache right now.
+ *
+ * @param cls closure (our "struct Plugin")
+ * @return GNUNET_OK on success, GNUNET_SYSERR on error
+ */
+static int
+mysql_plugin_del (void *cls)
+{
+ struct Plugin *plugin = cls;
+
+ MYSQL_BIND rbind[5];
+ unsigned int v_type;
+ GNUNET_HashCode v_key;
+ GNUNET_HashCode vhash;
+ unsigned long k_length;
+ unsigned long h_length;
+ unsigned long v_length;
+ int ret;
+ char buffer[GNUNET_SERVER_MAX_MESSAGE_SIZE];
+
+ k_length = sizeof (GNUNET_HashCode);
+ h_length = sizeof (GNUNET_HashCode);
+ v_length = sizeof (buffer);
+ memset (rbind, 0, sizeof (rbind));
+ rbind[0].buffer_type = MYSQL_TYPE_BLOB;
+ rbind[0].buffer_length = sizeof (GNUNET_HashCode);
+ rbind[0].length = &k_length;
+ rbind[0].buffer = &v_key;
+ rbind[1].buffer_type = MYSQL_TYPE_BLOB;
+ rbind[1].buffer_length = sizeof (GNUNET_HashCode);
+ rbind[1].length = &h_length;
+ rbind[1].buffer = &vhash;
+ rbind[2].buffer_type = MYSQL_TYPE_LONG;
+ rbind[2].is_unsigned = 1;
+ rbind[2].buffer = &v_type;
+ rbind[3].buffer_type = MYSQL_TYPE_BLOB;
+ rbind[3].buffer_length = sizeof (buffer);
+ rbind[3].length = &v_length;
+ rbind[3].buffer = buffer;
+ if ((GNUNET_OK !=
+ (ret = prepared_statement_run_select (plugin,
+ plugin->select_old_value,
+ 4,
+ rbind,
+ return_ok,
+ NULL,
+ -1))) ||
+ (GNUNET_OK !=
+ (ret = prepared_statement_run (plugin,
+ plugin->delete_value,
+ NULL,
+ MYSQL_TYPE_BLOB,
+ &v_key,
+ sizeof (GNUNET_HashCode),
+ &k_length,
+ MYSQL_TYPE_BLOB,
+ &vhash,
+ sizeof (GNUNET_HashCode),
+ &h_length,
+ MYSQL_TYPE_LONG,
+ &v_type,
+ GNUNET_YES,
+ MYSQL_TYPE_BLOB,
+ buffer,
+ (unsigned long)
+ sizeof (buffer),
+ &v_length, -1))))
+ {
+ if (ret == GNUNET_SYSERR)
+ itable (plugin);
+ return GNUNET_SYSERR;
+ }
+ plugin->env->delete_notify (plugin->env->cls,
+ &v_key,
+ v_length + OVERHEAD);
+
+ return GNUNET_OK;
+}
+
+
+/**
+ * Entry point for the plugin.
+ *
+ * @param cls closure (the "struct GNUNET_DATACACHE_PluginEnvironmnet")
+ * @return the plugin's closure (our "struct Plugin")
+ */
+void *
+libgnunet_plugin_datacache_mysql_init (void *cls)
+{
+ struct GNUNET_DATACACHE_PluginEnvironment *env = cls;
+ struct GNUNET_DATACACHE_PluginFunctions *api;
+ struct Plugin *plugin;
+
+ plugin = GNUNET_malloc (sizeof (struct Plugin));
+ plugin->env = env;
+ plugin->cnffile = get_my_cnf_path (env->cfg);
+ if (GNUNET_OK !=
+ iopen (plugin))
+ {
+ GNUNET_free_non_null (plugin->cnffile);
+ GNUNET_free (plugin);
+ return NULL;
+ }
+ if (GNUNET_OK !=
+ itable (plugin))
+ {
+ iclose (plugin);
+ GNUNET_free_non_null (plugin->cnffile);
+ GNUNET_free (plugin);
+ return NULL;
+ }
+ api = GNUNET_malloc (sizeof (struct GNUNET_DATACACHE_PluginFunctions));
+ api->cls = plugin;
+ api->get = &mysql_plugin_get;
+ api->put = &mysql_plugin_put;
+ api->del = &mysql_plugin_del;
+ GNUNET_log_from (GNUNET_ERROR_TYPE_INFO,
+ "mysql", _("MySQL datacache running\n"));
+ return api;
+}
+
+
+/**
+ * Exit point from the plugin.
+ *
+ * @param cls closure (our "struct Plugin")
+ * @return NULL
+ */
+void *
+libgnunet_plugin_datacache_mysql_done (void *cls)
+{
+ struct GNUNET_DATACACHE_PluginFunctions *api = cls;
+ struct Plugin *plugin = api->cls;
+
+ iclose (plugin);
+ GNUNET_free_non_null (plugin->cnffile);
+ GNUNET_free (plugin);
+ GNUNET_free (api);
+ mysql_library_end ();
+ return NULL;
+}
+
+
+/* end of plugin_datacache_mysql.c */
diff --git a/src/datacache/test_datacache.c b/src/datacache/test_datacache.c
index 8e09a7692..c5acf365a 100644
--- a/src/datacache/test_datacache.c
+++ b/src/datacache/test_datacache.c
@@ -1,6 +1,6 @@
/*
This file is part of GNUnet.
- (C) 2006, 2009 Christian Grothoff (and other contributing authors)
+ (C) 2006, 2009, 2010 Christian Grothoff (and other contributing authors)
GNUnet is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published
@@ -32,6 +32,11 @@
static int ok;
+/**
+ * Name of plugin under test.
+ */
+static const char *plugin_name;
+
static int
checkIt (void *cls,
@@ -120,12 +125,15 @@ FAILURE:
}
-static int
-check ()
+int
+main (int argc, char *argv[])
{
- char *const argv[] = { "test-datacache-api",
+ const char *pos;
+ char cfg_name[128];
+ char *const xargv[] = {
+ "test-datacache",
"-c",
- "test_datacache_data.conf",
+ cfg_name,
#if VERBOSE
"-L", "DEBUG",
#endif
@@ -134,30 +142,28 @@ check ()
struct GNUNET_GETOPT_CommandLineOption options[] = {
GNUNET_GETOPT_OPTION_END
};
- GNUNET_PROGRAM_run ((sizeof (argv) / sizeof (char *)) - 1,
- argv, "test-datacache-api", "nohelp",
- options, &run, NULL);
- if (ok != 0)
- fprintf (stderr, "Missed some testcases: %d\n", ok);
- return ok;
-}
-
-int
-main (int argc, char *argv[])
-{
- int ret;
-
- GNUNET_log_setup ("test-datacache-api",
+ GNUNET_log_setup ("test-datacache",
#if VERBOSE
"DEBUG",
#else
"WARNING",
#endif
NULL);
- ret = check ();
-
- return ret;
+ /* determine name of plugin to use */
+ plugin_name = argv[0];
+ while (NULL != (pos = strstr(plugin_name, "_")))
+ plugin_name = pos+1;
+ GNUNET_snprintf (cfg_name,
+ sizeof (cfg_name),
+ "test_datacache_data_%s.conf",
+ plugin_name);
+ GNUNET_PROGRAM_run ((sizeof (xargv) / sizeof (char *)) - 1,
+ xargv, "test-datacache", "nohelp",
+ options, &run, NULL);
+ if (ok != 0)
+ fprintf (stderr, "Missed some testcases: %d\n", ok);
+ return ok;
}
/* end of test_datacache.c */
diff --git a/src/datacache/test_datacache_data_mysql.conf b/src/datacache/test_datacache_data_mysql.conf
new file mode 100644
index 000000000..3086497df
--- /dev/null
+++ b/src/datacache/test_datacache_data_mysql.conf
@@ -0,0 +1,12 @@
+
+[testcache]
+QUOTA = 1000000
+DATABASE = mysql
+
+[datacache-mysql]
+DATABASE = gnunetcheck
+# CONFIG = ~/.my.cnf
+# USER =
+# PASSWORD =
+# HOST =
+# PORT = \ No newline at end of file
diff --git a/src/datacache/test_datacache_data.conf b/src/datacache/test_datacache_data_sqlite.conf
index 2e8606682..2e8606682 100644
--- a/src/datacache/test_datacache_data.conf
+++ b/src/datacache/test_datacache_data_sqlite.conf
diff --git a/src/datacache/test_datacache_quota.c b/src/datacache/test_datacache_quota.c
index f4686b6a3..9028196bd 100644
--- a/src/datacache/test_datacache_quota.c
+++ b/src/datacache/test_datacache_quota.c
@@ -1,6 +1,6 @@
/*
This file is part of GNUnet.
- (C) 2006, 2009 Christian Grothoff (and other contributing authors)
+ (C) 2006, 2009, 2010 Christian Grothoff (and other contributing authors)
GNUnet is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published
@@ -33,6 +33,11 @@
static int ok;
/**
+ * Name of plugin under test.
+ */
+static const char *plugin_name;
+
+/**
* Quota is 1 MB. Each iteration of the test puts in about 1 MB of
* data. We do 10 iterations. Afterwards we check that the data from
* the first 5 iterations has all been discarded and that at least
@@ -110,12 +115,15 @@ FAILURE:
}
-static int
-check ()
+int
+main (int argc, char *argv[])
{
- char *const argv[] = { "test-datacache-quota",
+ const char *pos;
+ char cfg_name[128];
+ char *const xargv[] = {
+ "test-datacache-quota",
"-c",
- "test_datacache_data.conf",
+ cfg_name,
#if VERBOSE
"-L", "DEBUG",
#endif
@@ -124,20 +132,7 @@ check ()
struct GNUNET_GETOPT_CommandLineOption options[] = {
GNUNET_GETOPT_OPTION_END
};
- GNUNET_PROGRAM_run ((sizeof (argv) / sizeof (char *)) - 1,
- argv, "test-datacache-quota", "nohelp",
- options, &run, NULL);
- if (ok != 0)
- fprintf (stderr, "Missed some testcases: %d\n", ok);
- return ok;
-}
-
-
-int
-main (int argc, char *argv[])
-{
- int ret;
-
+
GNUNET_log_setup ("test-datacache-quota",
#if VERBOSE
"DEBUG",
@@ -145,9 +140,20 @@ main (int argc, char *argv[])
"WARNING",
#endif
NULL);
- ret = check ();
-
- return ret;
+ /* determine name of plugin to use */
+ plugin_name = argv[0];
+ while (NULL != (pos = strstr(plugin_name, "_")))
+ plugin_name = pos+1;
+ GNUNET_snprintf (cfg_name,
+ sizeof (cfg_name),
+ "test_datacache_data_%s.conf",
+ plugin_name);
+ GNUNET_PROGRAM_run ((sizeof (xargv) / sizeof (char *)) - 1,
+ xargv, "test-datacache-quota", "nohelp",
+ options, &run, NULL);
+ if (ok != 0)
+ fprintf (stderr, "Missed some testcases: %d\n", ok);
+ return ok;
}
/* end of test_datacache_quota.c */
diff --git a/src/datastore/Makefile.am b/src/datastore/Makefile.am
index a96e22d4c..3c687e8ce 100644
--- a/src/datastore/Makefile.am
+++ b/src/datastore/Makefile.am
@@ -71,10 +71,10 @@ libgnunet_plugin_datastore_mysql_la_SOURCES = \
plugin_datastore_mysql.c
libgnunet_plugin_datastore_mysql_la_LIBADD = \
$(top_builddir)/src/statistics/libgnunetstatistics.la \
- $(top_builddir)/src/util/libgnunetutil.la $(XLIBS) -lz -lsqlite3
+ $(top_builddir)/src/util/libgnunetutil.la $(XLIBS) -lz -lmysqlclient
libgnunet_plugin_datastore_mysql_la_LDFLAGS = \
$(GN_PLUGIN_LDFLAGS) $(MYSQL_LDFLAGS) -lmysqlclient
-libgnunet_plugin_datastore_sqlite_la_CPFLAGS = \
+libgnunet_plugin_datastore_mysql_la_CPPFLAGS = \
$(MYSQL_CPPFLAGS)
diff --git a/src/datastore/plugin_datastore_mysql.c b/src/datastore/plugin_datastore_mysql.c
index 98c6c4ab3..c216e989c 100644
--- a/src/datastore/plugin_datastore_mysql.c
+++ b/src/datastore/plugin_datastore_mysql.c
@@ -572,46 +572,6 @@ run_statement (struct Plugin *plugin,
}
-#if 0
-/**
- * Run the given MySQL SELECT statement. The statement
- * must have only a single result (one column, one row).
- *
- * @return result on success, NULL on error
- */
-static char *
-run_statement_select (struct Plugin *plugin,
- const char *statement)
-{
- MYSQL_RES *sql_res;
- MYSQL_ROW sql_row;
- char *ret;
-
- if ((NULL == plugin->dbf) && (GNUNET_OK != iopen (plugin)))
- return NULL;
- mysql_query (plugin->dbf, statement);
- if ((mysql_error (plugin->dbf)[0]) ||
- (!(sql_res = mysql_use_result (plugin->dbf))) ||
- (!(sql_row = mysql_fetch_row (sql_res))))
- {
- LOG_MYSQL (GNUNET_ERROR_TYPE_ERROR,
- "mysql_query", plugin);
- return NULL;
- }
- if ((mysql_num_fields (sql_res) != 1) || (sql_row[0] == NULL))
- {
- GNUNET_break (mysql_num_fields (sql_res) == 1);
- if (sql_res != NULL)
- mysql_free_result (sql_res);
- return NULL;
- }
- ret = GNUNET_strdup (sql_row[0]);
- mysql_free_result (sql_res);
- return ret;
-}
-#endif
-
-
/**
* Create a prepared statement.
*