search for: getdent

Hi, on one of my Systems I'm seeing a strange phenomenon. When I do a "ls /var/log/named/" that command just freezes. An strace shows that ls is endlessly doing this: ... getdents(3, /* 66 entries */, 32768) = 3888 getdents(3, /* 85 entries */, 32768) = 5000 getdents(3, /* 70 entries */, 32768) = 4128 getdents(3, /* 80 entries */, 32768) = 4696 mremap(0x2b8b2b181000, 311296, 618496, MREMAP_MAYMOVE) = 0x2b8b2b181000 getdents(3, /* 82 entries */, 32768) = 4736...

...3431 stat("/test1", {st_mode=S_IFDIR|0777, st_size=8, ...}) = 0 <0.000017> 14:03:10.123521 open("/test1", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3 <0.000018> 14:03:10.123578 fcntl(3, F_GETFD) = 0x1 (flags FD_CLOEXEC) <0.000013> 14:03:10.123637 getdents(3, /* 3 entries */, 32768) = 72 <0.000025> 14:03:10.123712 getdents(3, /* 1 entries */, 32768) = 24 <0.000016> 14:03:10.123768 getdents(3, /* 1 entries */, 32768) = 24 <0.000015> 14:03:10.123824 getdents(3, /* 1 entries */, 32768) = 24 <0.000015> 14:03:10.123880 get...

...stat("???w", 0xbfffe51c) = -1 ENOENT (No such file or directory) stat(".", {st_mode=S_IFDIR|0755, st_size=2048, ...}) = 0 open(".", O_RDONLY|O_NONBLOCK) = 9 fcntl(9, F_SETFD, FD_CLOEXEC) = 0 lseek(9, 0, SEEK_CUR) = 0 getdents(9, /* 55 entries */, 3933) = 1124 lseek(9, 0, SEEK_CUR) = 1024 getdents(9, /* 13 entries */, 3933) = 328 lseek(9, 0, SEEK_CUR) = 2048 getdents(9, /* 0 entries */, 3933) = 0 close(9) = 0 stat("./", {st_mode=S_...

...id(65534) = 0 chdir("/var/state/ups") = 0 write(2, " 0.008773\t", 12) = 12 write(2, "Searching for USB device...\n", 28) = 28 openat(AT_FDCWD, "/dev/bus/usb", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 4 getdents(4, /* 11 entries */, 32768) = 264 close(4) = 0 openat(AT_FDCWD, "/dev/bus/usb", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 4 getdents(4, /* 11 entries */, 32768) = 264 getdents(4, /* 0 entries */, 32768) = 0 close(4)...

...SA_RESTORER, 0x326ba30210}, NULL, 8) = 0 rt_sigaction(SIGHUP, {SIG_IGN}, NULL, 8) = 0 rt_sigaction(SIGPIPE, {SIG_IGN}, NULL, 8) = 0 open("/dev/bus/usb", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 4 fstat(4, {st_mode=S_IFDIR|0755, st_size=180, ...}) = 0 fcntl(4, F_SETFD, FD_CLOEXEC) = 0 getdents(4, /* 9 entries */, 4096) = 216 close(4) = 0 open("/dev/bus/usb", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 4 fstat(4, {st_mode=S_IFDIR|0755, st_size=180, ...}) = 0 fcntl(4, F_SETFD, FD_CLOEXEC) = 0 getdents(4, /* 9 entries */, 4096) = 216 getd...

...8 stat 0.01 170 fcntl 0.01 282 select 0.00 87 gettimeofday 0.00 105 sysconf 0.00 14 send 0.00 87 getdents 0.75 1727 fstat64 0.00 31 lseek64 0.00 8 lstat64 0.48 11809 stat64 0.09 170 93 statvfs64 0.00 7 acl...

On 17/09/17 18:03, Niklas Hamb?chen wrote: > So far the only difference between `ls` and `bup index` I could observe > is that `bup index` chdir()s into the directory to index, ls doesn't. > > But when I `cd` into the dir and run `ls` without directory argument, it > is still much faster than bup index for each stat(). Hmm, bup uses the fchdir() syscall to go into the target

I found the reason now, at least for this set of lstat()s I was looking at. bup first does all getdents(), obtaining all file names in the directory, and then stat()s them. Apparently this destroys some of gluster's caching, making stat()s ~100x slower. What caching could this be, and how could I convince gluster to serve these stat()s as fast as if a getdents() had been done just before them?

On Jan 12, 2014, at 11:07 AM, Ariel Wainer wrote: > On 11/01/14 15:27, Charles Lepple wrote: >> On Jan 10, 2014, at 4:46 PM, Ariel Wainer wrote: >> >>> On 10/01/14 01:53, Charles Lepple wrote: >>>> I am curious about why the Interrupt Out packet is sent by the Windows software if it isn't to turn off the UPS. Is it possible to do some more testing with

...g of f_pos to 2^31-1. So after enough creates (and deletes :)) in a directory we end up with an entry item whose key is past that value. f_pos gets rewound instead of being set to that magical EOF. readdir() gets stuck returning the entries after INT_MAX over and over (just one in this strace): getdents(3, {{d_ino=257, d_off=2147483647, d_reclen=32, d_name="file-54"}}, 32768) = 32 getdents(3, {{d_ino=257, d_off=2147483647, d_reclen=32, d_name="file-54"}}, 32768) = 32 It took around 10 hours on a workstationy box over here to reproduce this with createmany.c from the lustre te...

...seems to spend a lot of time rescanning the directory, especially when the server is loaded. I'm not sure whether this is triggered by Thunderbird or done regularly, but it takes longer when the server is loaded, so sometimes it seems that it is scanning continuously. Since it takes around 2000 getdents64() syscalls to scan my inbox, I'm wondering whether the buffer size is a bit small with 4KB and might benefit from being configurable: ... getdents64(8, /* 73 entries */, 4096) = 4088 getdents64(8, /* 73 entries */, 4096) = 4088 getdents64(8, /* 73 entries */, 4096) = 4088 getdents64(8...

...AP_ANONYMOUS, -1, 0) = 0x7f0be9228000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) ... open("/dev/bus/usb", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 4 fstat(4, {st_mode=S_IFDIR|0755, st_size=80, ...}) = 0 fcntl(4, F_SETFD, FD_CLOEXEC) = 0 getdents(4, /* 4 entries */, 4096) = 96 close(4) = 0 open("/dev/bus/usb", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 4 fstat(4, {st_mode=S_IFDIR|0755, st_size=80, ...}) = 0 fcntl(4, F_SETFD, FD_CLOEXEC) = 0 getdents(4, /* 4 entries */, 4096) = 96 getdent...

...2: On * servers: CentOS 6.3, 2.6.32-279.14.1.el6.centos.plus.x86_64 * cleints: CentOS 5.7, 2.6.18-274.12.1.el5 We have a directory containing 3,343 subdirectories. On some clients, ls lists only a subset of the directories (a different amount on different clients). On others, ls gets stuck in a getdents loop and consumes more and more memory until it hits ENOMEM. On yet others, it works fine. Having the bad clients remount or drop caches makes the problem temporarily go away, but eventually it comes back. The issue sounds a lot like bug #838784, but we are using xfs on the backend, and this se...

...n("/", O_RDONLY|0x200000, 0) ............................. = 6 [6951] fstat(6, 0x7fff8440) ........................................ = 0 [6951] fcntl(6, F_SETFD, 0) ........................................ = 0 [6951] sysfs(GETFSIND, 0xc015ac30, 0x777df220) ..................... = 10 [6951] getdents(6, 0x4003d820, 16384) .............................. = 152 [6951] stat64("/test", 0x7fff84e0) ................................. = 0 [6951] getdents(6, 0x4003d820, 16384) .............................. = 0 [6951] close(6) .................................................... = 0 [6951] R...

...-f -w -c rsync -a --dry-run /mnt/dir /tmp/nonexisting % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 52.35 69.682773 26617 2618 1 select 39.60 52.712292 1056 49907 getdents 8.00 10.655949 11 998102 lstat 0.02 0.022890 12 1900 read 0.01 0.017219 21 829 write 0.01 0.012093 14 868 munmap 0.01 0.006656 11 606 mmap...

...directory, and the directory *file* itself does get somewhat large. The directory name is MM-DD-YY so it changes every 24 hours. Here's the problem .... as the directory gets larger and at about 400K, the file creation slows down, and it slows down, it appears because samba is looping calling getdents() hundreds of times without doing anything else as if it needs to read every entry in the directory for every file 'open' requested. AND each PC seems to be bound to exactly one Samba server. The Samba 'listener' never forks more then one server per PC, and I guess that is expe...

...cntl(27, F_SETFD, FD_CLOEXEC) = 0 time([1100611783]) = 1100611783 rt_sigaction(SIGPIPE, {0x2a96c00940, [], 0x4000000}, {SIG_DFL}, 8) = 0 sendto(18, "<13>Nov 16 06:29:43 smbd_audit[2"..., 58, 0, NULL, 0) = 58 rt_sigaction(SIGPIPE, {SIG_DFL}, NULL, 8) = 0 getdents(27, /* 13 entries */, 4096) = 400 getdents(27, /* 0 entries */, 4096) = 0 close(27) = 0 stat("3277/TEST/DEFAULT.PLP", {st_mode=S_IFREG|0764, st_size=600, ...}) = 0 getxattr("3277/TEST/DEFAULT.PLP", "user.DOSATTRIB", 0x7fbfffd4f0,...

Hi Ben, do you know if the smallfile benchmark also does interleaved getdents() and lstat, which is what I found as being the key difference that creates the performance gap (further down this thread)? Also, wouldn't `--threads 8` change the performance numbers by factor 8 versus the plain `ls` and `rsync` that I did? Would you mind running those commands directly/pla...

...run /mnt/dir /tmp/nonexisting > > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 52.35 69.682773 26617 2618 1 select > 39.60 52.712292 1056 49907 getdents > 8.00 10.655949 11 998102 lstat > 0.02 0.022890 12 1900 read > 0.01 0.017219 21 829 write > 0.01 0.012093 14 868 munmap > 0.01 0.006656 11...

...mmap(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fef05033000 [pid 25549] mmap(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fef04f55000 [pid 25549] openat(AT_FDCWD, ".", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 5 [pid 25549] getdents(5, /* 2 entries */, 32768) = 48 [pid 25549] getdents(5, /* 0 entries */, 32768) = 0 [pid 25549] close(5) = 0 [pid 25549] select(4, [3], [3], [3], {60, 0}) = 1 (out [3], left {59, 999991}) [pid 25549] write(3, "\17\0\0\7\31\1.\0\0\20X\374Kx\355A\0\0\0", 19) = 19 [pid 25...