Hi,
My name is Luo Zengbo, a student from Zhejiang University, China. As
I mentioned in IRC before(my IRC nick name is poe_), I am interested in
GSoC project: Speed up the Test Suite. After discussed with Olly, I think
this project can be divided into three steps:
Step 1:
Rewrite every call of the function get_writable_database(). I will
pass writable database's name with test's name appending to avoid two
test
cases writing the same database at the same time.
Step 2:
Run test cases in concurrency with thread pool.
Step 3:
Adding some instrumentation code to record the run time of every
test.
As we know, multi-threads uses less resource and is easier to
communication than multi-processes, so I choose multi-threads. My questions
is that which multi-thread library is more suitable for Xapian? C++11's
<thread> or <pthread.h>? On the one hand, pthead is POSIX threads
library
which will work well in Unix-like system. But in Windows, we need to
install the third-party library such as pthread-w32. On the other hand,
C++11's thread is standard library so we need install a third-party
library, but maybe not supported by the earlier GCC compiler.
Any suggestion about the plan and the choice of library is appreciated.
Regards,
Luo Zengbo
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.xapian.org/pipermail/xapian-devel/attachments/20160312/c21f3a95/attachment.html>
On Sat, Mar 12, 2016 at 08:14:44PM +0800, ??? wrote:> My name is Luo Zengbo, a student from Zhejiang University, China. > As I mentioned in IRC before(my IRC nick name is poe_).Hi, Luo Zengbo! Welcome to Xapian.> As we know, multi-threads uses less resource and is easier to > communication than multi-processes, so I choose multi-threads.It is, however, harder to program for in some ways. (It's also not true in all operating systems that threads are significantly less resource hungry than processes.) Not saying that it's the wrong choice, just that when you come to justify that choice in your project proposal you shouldn't just assume that everyone agrees it's the right way to go.> My questions is that which multi-thread library is more suitable for > Xapian? C++11's <thread> or <pthread.h>? On the one hand, pthead is > POSIX threads library which will work well in Unix-like system. But > in Windows, we need to install the third-party library such as > pthread-w32. On the other hand, C++11's thread is standard library > so we need install a third-party library, but maybe not supported by > the earlier GCC compiler.If you rely on an external library, you could detect it in configure and then disable multi-threading for the test suite. Similarly, if you need <thread> you can detect whether that will work or not, and disable if necessary. Ideally we'd take a route which will work without too much pain on Windows. However right now I don't know anyone who's built Xapian successfully on Windows using the standard toolchain (autotools &c); and the old way of doing things is no longer maintained. So it's entirely possible that master doesn't even build on Windows at the moment anyway, in which case it's a little more moot. (But taking an approach which would make Windows support feasible in future would be preferable.) Apparently Visual Studio 2015 claims support for all the C++11 core language features in the concurrency group. In terms of GCC versions (for instance), we currently require GCC 4.7, under the general policy:> We aim to support compilation with any C++ compiler which conforms > to ISO C++11, or a reasonable approximation to it.Note that Debian stable ships with GCC 4.9; Ubuntu 14.04 LTS shipped with GCC 4.8. If you know what version of GCC is needed to provide suitable C++11 <thread> support, then you can figure out whether that's feasible for common platforms. (CentOS 7 and RHEL7 apparently ship with GCC 4.8.) I don't know what the clang equivalent version is (but a quick look at their site suggests C++11 should be entirely supported from 3.3 onward; 3.4 shipped in Ubuntu 14.04 LTS). It's not clear from the pthreads-win32 homepage, but I assume it implements pthreads in terms of Windows' own threading system? J -- James Aylett, occasional trouble-maker xapian.org
Thanks for your kind reply! Now I'm reading gtest source code for reference. Hope it helps. It will be appreciated if you can tell some other open source test frameworks which is useful for Xapian test suite. And I'll write the raw proposal in the next few days. 2016-03-12 23:50 GMT+08:00 James Aylett <james-xapian at tartarus.org>:> On Sat, Mar 12, 2016 at 08:14:44PM +0800, ??? wrote: > > > My name is Luo Zengbo, a student from Zhejiang University, China. > > As I mentioned in IRC before(my IRC nick name is poe_). > > Hi, Luo Zengbo! Welcome to Xapian. > > > As we know, multi-threads uses less resource and is easier to > > communication than multi-processes, so I choose multi-threads. > > It is, however, harder to program for in some ways. (It's also not > true in all operating systems that threads are significantly less > resource hungry than processes.) Not saying that it's the wrong > choice, just that when you come to justify that choice in your project > proposal you shouldn't just assume that everyone agrees it's the right > way to go. > > > My questions is that which multi-thread library is more suitable for > > Xapian? C++11's <thread> or <pthread.h>? On the one hand, pthead is > > POSIX threads library which will work well in Unix-like system. But > > in Windows, we need to install the third-party library such as > > pthread-w32. On the other hand, C++11's thread is standard library > > so we need install a third-party library, but maybe not supported by > > the earlier GCC compiler. > > If you rely on an external library, you could detect it in configure > and then disable multi-threading for the test suite. Similarly, if you > need <thread> you can detect whether that will work or not, and > disable if necessary. > > Ideally we'd take a route which will work without too much pain on > Windows. However right now I don't know anyone who's built Xapian > successfully on Windows using the standard toolchain (autotools &c); > and the old way of doing things is no longer maintained. So it's > entirely possible that master doesn't even build on Windows at the > moment anyway, in which case it's a little more moot. (But taking an > approach which would make Windows support feasible in future would be > preferable.) Apparently Visual Studio 2015 claims support for all the > C++11 core language features in the concurrency group. > > In terms of GCC versions (for instance), we currently require GCC 4.7, > under the general policy: > > > We aim to support compilation with any C++ compiler which conforms > > to ISO C++11, or a reasonable approximation to it. > > Note that Debian stable ships with GCC 4.9; Ubuntu 14.04 LTS shipped > with GCC 4.8. If you know what version of GCC is needed to provide > suitable C++11 <thread> support, then you can figure out whether > that's feasible for common platforms. (CentOS 7 and RHEL7 apparently > ship with GCC 4.8.) > > I don't know what the clang equivalent version is (but a quick look at > their site suggests C++11 should be entirely supported from 3.3 > onward; 3.4 shipped in Ubuntu 14.04 LTS). > > It's not clear from the pthreads-win32 homepage, but I assume it > implements pthreads in terms of Windows' own threading system? > > J > > -- > James Aylett, occasional trouble-maker > xapian.org >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20160313/b5f53bfe/attachment.html>