Xapian devel - Aug 2005 - uint4 in quartz/flint

We have:

----------------------------------------------------------------------
typedef unsigned long uint4;
----------------------------------------------------------------------

which on an LP64 system is a lie. If uint4 is used on disk (and it's
used in the btree implementation) then it would make the databases
unportable, which probably isn't a good idea (unless we already have
endianness issues, although even there, an ILP32 and an LP64 process
on the same machine should really be able to share databases, although
I haven't checked for alignment problems).

I'm also not sure why quartz_types.h / flint_types.h has a comment
about a double implementation in the following:

----------------------------------------------------------------------
/** An integer type which can store the sum of the lengths of the
documents
 *  in the database.
 *
 *  FIXME - change this to a double?
 */
typedef unsigned long long int flint_totlen_t;
----------------------------------------------------------------------

unsigned long long is dead right on both major data type sizes in use
at the moment. While double certainly takes up 64bits, I don't see why
an integer isn't the right choice. The comment comes from:

----------------------------------------------------------------------
r2514 | richard | 2000-10-20 18:23:11 +0100 (Fri, 20 Oct 2000) | 4
lines

In quartz: store document lengths in termlists, store total
length in a field in the record table, and add methods to access
all these.  QuartzDatabase::get_avlength() now implemented.
----------------------------------------------------------------------



I'm also not keen on the way that we use:

#define UINT64 unsigned long long
#define uint64_t unsigned long long

These should (a) be the same (:-), (b) be typedefs, and (c) not be
there anyway because #include <inttypes.h> will get them for us on
modern systems. (If they aren't there, we can add configure checks -
inttypes.h is in C99.)

(If we used inttypes.h in quartz/flint, the uint4 typedef would be
reliable, or actually just replaced by uint32_t.)

J

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  james at tartarus.org                               uncertaintydivision.org

On Fri, 2005-08-05 at 11:40 +0100, James Aylett wrote:
> I'm also not sure why quartz_types.h / flint_types.h has a comment
> about a double implementation in the following:
> 
> ----------------------------------------------------------------------
> /** An integer type which can store the sum of the lengths of the
> documents
>  *  in the database.
>  *
>  *  FIXME - change this to a double?
>  */
> typedef unsigned long long int flint_totlen_t;
For the record, since it looks like it was me who wrote that comment,
I'm not sure what the FIXME is going on about, and an integer type
sounds entirely the right thing here.  I'm afraid I can't remember what
I was thinking here...

-- 
Richard Boulton <richard at tartarus.org>

On Fri, Aug 05, 2005 at 11:40:29AM +0100, James Aylett
wrote:
> We have:
> 
> ----------------------------------------------------------------------
> typedef unsigned long uint4;
> ----------------------------------------------------------------------
> 
> which on an LP64 system is a lie. If uint4 is used on disk (and it's
> used in the btree implementation) then it would make the databases
> unportable, which probably isn't a good idea (unless we already have
> endianness issues, although even there, an ILP32 and an LP64 process
> on the same machine should really be able to share databases, although
> I haven't checked for alignment problems).
It's used to hold a quantity read (byte-by-byte) from disk.  So the
fact it may be MORE than 4 bytes isn't a portability problem, though it
is at best somewhat unaesthetic.

As far as I'm aware, databases are completely portable regardless of
endian or word size issues, or anything else.
> I'm also not sure why quartz_types.h / flint_types.h has a comment
> about a double implementation in the following:
> 
> ----------------------------------------------------------------------
> /** An integer type which can store the sum of the lengths of the
> documents
>  *  in the database.
>  *
>  *  FIXME - change this to a double?
>  */
> typedef unsigned long long int flint_totlen_t;
> ----------------------------------------------------------------------
> 
> unsigned long long is dead right on both major data type sizes in use
> at the moment. While double certainly takes up 64bits, I don't see why
> an integer isn't the right choice. The comment comes from:
Yeah, a double would be a poor choice, as we need to be able to hold the
total length losslessly.  I'll kill that FIXME.
> I'm also not keen on the way that we use:
> 
> #define UINT64 unsigned long long
> #define uint64_t unsigned long long
> 
> These should (a) be the same (:-), (b) be typedefs, and (c) not be
> there anyway because #include <inttypes.h> will get them for us on
> modern systems. (If they aren't there, we can add configure checks -
> inttypes.h is in C99.)
"UINT64" is only used in the legacy muscat36 backend.  Few people will
have any use for that code, so time which might be spent cleaning up
the code is much better spent elsewhere.

"uint64_t" is only used if TIMING_PATCH is defined (which it isn't
unless you force it to be).  That's a vestige of some profiling work
from someone at webtop (jj probably), and should probably just be
removed.
> (If we used inttypes.h in quartz/flint, the uint4 typedef would be
> reliable, or actually just replaced by uint32_t.)
We probably should, thought using inttypes.h from C++ puts you in the
slightly disturbing area between the C++ standard and the C99 revisions
to the C90 standard.  But hopefully configure tests can sort out any
problems there.

Cheers,
    Olly