I still haven't been able to pinpoint exactly where bacula hangs up when LD_PRELOAD is set to use spd_readdir, but I have a suspect. bacula-fd gets directory entries with readdir_r, which is a function that is not reimplemented in spd_readdir. So when bacula calls opendir it gets the shadow version, which calls the original open, read, and closedir functions. It then returns its private dir_s structure. The (unshadowed) readdir_r then tries to work with dir_s. It looks as if I (or one of you gurus?) need to implement a wrapper for readdir_r. A quick looks suggests there may be a couple of subtleties (the spd_readdir struct dir_s is allocated, and so thread safe, but it's dir entry is not; and readdir_r is expecting some "real" system data structures back and users may have problems with fake ones). Ross
I've attached a modified version of Ted's spd_readdir.c that adds support for readdir_r and readdir64_r. It appears to be working (readdir64_r is the only new routine getting exercised), but should be taken as a rough cut. I also added a Makefile and a test program. It also looks as if this is giving me a huge speed improvement (at least x4) of my backups of my ext3 partitions. I'll try to report after a full and incremental backup complete, which will be a couple of days. Originally I tried taking the threading code from the system implementations of the original readdir_r. When that didn't work (since it was designed to be part of a libc build) I switched to pthreads. I don't know if recursive locking is essential; I activated it at one point while trying to get things to work. For big directories this code could use quite a lot of memory. It allows an optional max size, beyond which it reverts to the original system calls. I wonder if instead taking large directories in chunks would preserve much of the speedup while putting a bound on memory use. Ross Boylan -------------- next part -------------- A non-text attachment was scrubbed... Name: RBspd_dir.tgz Type: application/x-compressed-tar Size: 889 bytes Desc: not available URL: <http://listman.redhat.com/archives/ext3-users/attachments/20080608/edf71b04/attachment.bin>