freebsd stable - Apr 2015 - FreeBSD/ZFS on 9.3-RELEASE chews up memory with "wide" directories when calling readdir, etc; causes trap 12 panics

Hi,
	Long story short, I had a lot of mail spooled up in /var/spool. When I did ls
/var/spool, ZFS chewed up almost all 12GB of my memory in <10 mins (because
there were enough files there) and the system eventually panicked because [I
assume that a memory allocation failed and] a trap 12 panic was caught. I don?t
have the exact details, but it should be relatively easy to repro (YMMV if you
have a boatload of RAM):

repro_end=10000000000
for i in $(seq 1 $repro_end); do mktemp tmp.XXXXXXXXXXXX; done
ls

	This might be ameliorated via r281026, but this change is only available in
CURRENT (so far), and I haven?t tested it.
	Are there any comments about this scalability issue with FreeBSD/ZFS?
Thanks,
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20150406/feb741ee/attachment.sig>

On Tue, Apr 7, 2015 at 12:37 AM, Garrett Cooper <yaneurabeya at gmail.com>
wrote:
> Hi,
>         Long story short, I had a lot of mail spooled up in /var/spool.
When I did ls /var/spool, ZFS chewed up almost all 12GB of my memory in <10
mins (because there were enough files there) and the system eventually panicked
because [I assume that a memory allocation failed and] a trap 12 panic was
caught. I don?t have the exact details, but it should be relatively easy to
repro (YMMV if you have a boatload of RAM):
>
> repro_end=10000000000
> for i in $(seq 1 $repro_end); do mktemp tmp.XXXXXXXXXXXX; done
> ls
>
>         This might be ameliorated via r281026, but this change is only
available in CURRENT (so far), and I haven?t tested it.
>         Are there any comments about this scalability issue with
FreeBSD/ZFS?
> Thanks,

I spent the last ~ 24 hours creating 58,567,635 empty files in one
directory.  I can ls it without crashing on a machine with 32 GB RAM.

# /usr/bin/time -l ls /tmp/tmp | wc
     1061.21 real       225.54 user        36.61 sys
   9720268  maximum resident set size
        28  average shared memory size
         8  average unshared data size
       128  average unshared stack size
   2425013  page reclaims
         0  page faults
         0  swaps
    108036  block input operations
         0  block output operations
         0  messages sent
         0  messages received
         0  signals received
    108004  voluntary context switches
      2428  involuntary context switches
 58567635 58567635 644243985

-Alan