Ext3 users - Sep 2002 - What happens with the indexed directory patch ?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,
I'm trying to install the htree indexed directory patch to my RedHat 
kernel-source-2.4.18-10, but it seems that doesn't go.

First I tried the ext3 patch (I have sent some emails to this list telling the 
problems I had with it).

Now I have just test the ext2 patch.
I get it from:
http://people.nl.linux.org/~phillips/htree/htree-2.4.18-2
and applied to my kernel source this way:
# cd /usr/src/linux
# cat htree-2.4.18-2 | patch -p1

All the compilation process goes well, I can reboot the computer with the new 
kernel, make a new ext2 file system over a free partition and mount it with 
the -o index option:
# mount /dev/hda7 /internet -o index
# mount
/dev/hda7 on /internet type ext2 (rw,index)

But when I make the test with the script I included at the end of this email, 
the results are not the expected from an indexed directory. The creation time 
of 2500 files for an (in theory) indexed directory, are nearly the same for 
the non indexed directory:

INDEXED DIRECTORY:
# mkdir /internet/test
# cd /internet/test
# ./mk_file_test.sh 2500
Thu Sep  5 17:40:19 CEST 2002
real    0m15.532s
user    0m5.970s
sys     0m9.090s
Thu Sep  5 17:40:34 CEST 2002

NON INDEXED DIRECTORY:
# mkdir /root/test
# cd /root/test
# ./mk_file_test.sh 2500
Thu Sep  5 17:22:40 CEST 2002
real    0m15.130s
user    0m5.740s
sys     0m9.270s
Thu Sep  5 17:22:55 CEST 2002


The size of both directories after run the test script are:
drwxr-xr-x    2 root     root        49152 Sep  5 17:40 /internet/test
drwxr-xr-x    2 root     root        32768 Sep  5 17:44 /root/test

The (in theory) indexed directory is greater than the non indexed one. This is 
normal, but why the nearly equal performance ? The creation time for the 
indexed directory must be lot of times more quick.

Has anyone tested this patch succesfully ?
Is the patch still incomplete ?
Am I doing something wrong ?
May someone help me ?

Please, help me. I believe that the htree indexed directory is a very very 
great improvement for the ext2/ext3 file system that must become a standard 
as soon  as possible.

Greetings.
- ---
Carles Xavier Munyoz Baldó
carles@descom.es
Descom Consulting
Telf: +34 965861024
Fax: +34 965861024
http://www.descom.es/
- ---

[...]
if [ -z "$1" ]; then
  echo "USAGE $0 last_directory_number"
  exit 0
fi

N=0
date
time while [ 1 ]; do
  > $N
  if [ "$N" = "$1" ]; then
    break
  fi
  N=`expr $N + 1`
done
echo
date
[...]
-----BEGIN PGP SIGNATURE-----
Version: PGP 6.5.8

iQA/AwUBPXd/UzvYAf7VZNaaEQJJXwCfY7RG80eFwuHX6LAygXnEQY4AOiEAn3qg
kXTwP7uwEAJG7buHHTXeSR+D
=ZqWs
-----END PGP SIGNATURE-----

On Thu, Sep 05, 2002 at 05:59:09PM +0200, Carles Xavier Munyoz Bald?
wrote:
> 
> Please, help me. I believe that the htree indexed directory is a very very 
> great improvement for the ext2/ext3 file system that must become a standard
> as soon  as possible.
I'm not sure why you're not seeing the improvement based on the Daniel
Phillip's ext2 patch (it worked for me) but let me give you an updated
set of patches and a test script for ext3.  

I'm still working on cleaning it up, and the version I'm sending you
still is missing some error checks and doesn't have the code cleanup
that is still on-going, but it's been slightly better tested than the
most bleeding edge code that I have.  (Translation: I've tested it for
two hours in my hotel room in Koeln, after I got back from the Linux
Kongress Social.  So **please** don't run this on a production system,
at least not until you've done your own testing and validation.)

(To Andreas: I've done the work to make the code support multiple hash
algorithms, so we can now smoothly transition between filesystems that
were running Daniel Phillip's original dx_hack_hash and the latest
format as supported by this patch and e2fsprogs 1.28).

Using this patch, it is very easy to demonstrate that if you don't
turn on the indexed directory filesystem feature, the test scripts
take a **lot** longer.  With the indexed directory feature turned on,
running the script only takes about 3 minutes of real time on my
laptop, and the disk drive light was on most of the time.  With the
indexed directory feature turned off, running the script took over 15
minutes and had only completed a tiny fraction of the directory
entries before I gave up and hit Control-C.  So you should an
orders-of-magnitude improvement with the indexed directory feature
turned on. 

Caveats: This patch and the test script requires the use of e2fsprogs
1.28.  Also, the mke2fs options in the test script are designed to
stress the htree code using a minimal amount of disk space on my
laptop.  They should not be used for production filesystems.  Finally,
the debugfs commands to initialize the filesystem will be
automatically done by mke2fs in the next release of e2fsprogs.

So rest assured, we *are* working on the htree indexed directory code.
I fully intend for this to get integrated into the 2.5 tree, even if
we don't have the NFS readdir fixes done at first.  

						- Ted