Xapian discuss - Mar 2010 - Debug build

Hello,

I have updated to Xapian 1.1.4 and maybe there is a
memory leak. I can run only dmalloc - valgrind would
be much too slow.
How can I build a debug-build of xapian?

Thanks a lot
Marcus

On Tue, Mar 16, 2010 at 06:26:08PM +0100, double wrote:
> I have updated to Xapian 1.1.4 and maybe there is a memory leak.
It's not impossible, but we do run the testsuite under valgrind, so the
memory leak would need to have a dangling reference, or not be exercised
by the testsuite.
> I can run only dmalloc - valgrind would be much too slow.
Personally I've found valgrind is fast enough - it only adds CPU overhead,
which is less of an issue when things tend to be I/O bound.
> How can I build a debug-build of xapian?
What do you mean by a "debug build"?

If you use GCC, configure compiles with "-g -O2" by default, so you
have
debug symbols by default.

INSTALL documents how to enable assertions, but that's not useful for leak
detection.

HACKING documents enabling call logging (but that's not useful for leak
checking either), and also has a section on "Using various debugging,
profiling, and leak-finding tools", but nobody's documented what is
required
for dmalloc yet.

I don't know anything about dmalloc, so you'll need to figure that part
out
yourself.  If you can produce a "recipe" for using dmalloc, let me
know and
I'll add it to the documentation.

Note that for leak checking C++ code, you probably also want to set
GLIBCXX_FORCE_NEW=1 in the environment and export it, otherwise STL memory
pooling will confuse the issue (tests/runtest does this for the testsuite
when running under valgrind).

Cheers,
    Olly

Hi Olly,

Thank you very much for your answer. In order to track down
the issue I forced a core dump on `atexit()`.

Xapian is 1.1.4, the database is flint. Here the dmalloc-report:

-->  13904  times:
0xb778cad4 <_ZNSs4_Rep9_S_createEjjRKSaIcE+100>:        0xc7047089
-->  9262  times:
0xb7cb8ad0 <FlintCursor+176>:   0x8308558b
-->  4631  times:
0xb7d552a2 <_ZNK6Xapian10BM25Weight5cloneEv+34>:        0x896446dd
-->  4631  times:
0xb7cd8560 <_ZNK10FlintTable10cursor_getEv+48>: 0x04247489
-->  4631  times:
0xb7cb8a70 <FlintCursor+80>:    0x2774ff85
-->  4631  times:
0xb7cbcfb9 <_ZNK13FlintDatabase14open_post_listERKSs+73>:       0x08247c89
-->  28  times:
0xb7d2b0fc 
<_ZN14QueryOptimiser10do_or_likeEPKN6Xapian5Query8InternalEd+252>:   
0x8908558b
-->  17  times:
0xb7cdd28d <_ZN10FlintTable15do_open_to_readEbj+205>:   0x4689c085
--> rest ignored - If needed I can provide a complete list.


The watched process only reads from the Xapian database. Updated are done
by a cron job every minute. The possible functions are (if it is helpful, I
can provide you the source of these wrapper functions):

List<Result> fts_search( const String &search, Integer skip, Integer 
count );
   -->   Xapian::Enquire() with Xapian::ValueWeightPostingSource()
List<Result> fts_similar( Integer id, Integer skip, Integer count );
   -->   Xapian::Query( Xapian::Query::OP_ELITE_SET );
StringList fts_suggestions( const String &search, Integer skip, Integer 
count );
   -->   Xapian::Query( Xapian::QueryParser::FLAG_PARTIAL );


This three functions have the same structure (on error the returned list 
is empty).
The exception: "The revision being read has been discarded - you should 
call
Xapian::Database::reopen() and retry the operation" is thrown quite often.

List<struct Result> fts_functionname( ... )
{
   List<struct Result> returnvalue;
   // "The revision being read has been discarded - you should call 
Xapian::Database::reopen() and retry the operation"
   for( Integer i=0; i < 3; ++i )
   {
       Xapian::Database &xapian_database = *::database;
       try
       {
           ...         //    <--- here the Xapian database is called
       }
       catch( const Xapian::DatabaseModifiedError &error )      //    
<---  this is thrown quite often
       {
           try {
               xapian_database.reopen();           // reopen the database
               continue;                           // retry...
           }
           catch( const Xapian::Error &error ) {
           }
       }
       catch( const Xapian::Error &error ) {
       }
       catch( ... ) {
       }
       break;
   }
   return returnvalue;
}

> If you use GCC, configure compiles with "-g -O2" by default, so
you have
> debug symbols by default.  
Maybe the debug symbols are cut off during linking? GDB is
not able to determine the code-lines.

Thanks a lot
Marcus