Am 05.04.2016 um 13:22 schrieb Ralph Boehme:>> Am 05.04.2016 um 13:03 schrieb Ralph Boehme: >>> On Tue, Apr 05, 2016 at 12:43:37PM +0200, Reindl Harald wrote: >>>> what is wasting that much RAM if vfs_fruit is enabled? >>>> >>>> had 4 OOM-killers today after not face a single one in the last 7 years and >>>> on a different machine i swa a smbd process eating up 6500 MB on a HP >>>> Microserver with 8 GB RAM due copy a 300 GB folder with a Apple client and >>>> even not release the memory after the operation was finished >>>> >>>> root 24301 1.9 22.3 1323692 903600 ? SN 12:29 0:13 >>>> /usr/sbin/smbd -D >>>> >>>> samba-4.3.6-0.fc23.x86_64 >>> >>> just got a private mail from someone else who ran into the same issue. >>> >>> Can you run smbd under valgrind with memcheck? >> >> not that easy in production and it seems to be unpredictable to trigger > > it was mentioned that it is reproducible by deleting files and foldersfor sure it's enough copy a folder with many elements, the crash below was my boss trying to copy a folder with 880 files over a slow VPN tunnel which produced 3 crashes so far and given that i raised the RAM of the virtual machine yesterday from 4 GB to 5 GB there is no end in ressource usage until it crashs Apr 6 14:16:14 fileserver kernel: Out of memory: Kill process 9190 (smbd) score 552 or sacrifice child Apr 6 14:16:14 fileserver kernel: Killed process 9190 (smbd) total-vm:3370368kB, anon-rss:2730796kB, file-rss:1292kB Apr 6 14:43:41 fileserver kernel: afpd invoked oom-killer: gfp_mask=0x24201ca, order=0, oom_score_adj=0 Apr 6 14:16:14 fileserver kernel: Out of memory: Kill process 9190 (smbd) score 552 or sacrifice child Apr 6 14:43:42 fileserver kernel: Out of memory: Kill process 11611 (smbd) score 552 or sacrifice child Apr 6 15:34:28 fileserver kernel: Out of memory: Kill process 12022 (smbd) score 549 or sacrifice child -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/samba/attachments/20160406/20bc687f/signature.sig>
Am 06.04.2016 um 15:43 schrieb Reindl Harald:> Am 05.04.2016 um 13:22 schrieb Ralph Boehme: >>>> just got a private mail from someone else who ran into the same issue. >>>> >>>> Can you run smbd under valgrind with memcheck? >>> >>> not that easy in production and it seems to be unpredictable to trigger >> >> it was mentioned that it is reproducible by deleting files and folders > > for sure it's enough copy a folder with many elements, the crash below > was my boss trying to copy a folder with 880 files over a slow VPN > tunnel which produced 3 crashes so far and given that i raised the RAM > of the virtual machine yesterday from 4 GB to 5 GB there is no end in > ressource usage until it crashs > > Apr 6 14:16:14 fileserver kernel: Out of memory: Kill process 9190 > (smbd) score 552 or sacrifice child > Apr 6 14:16:14 fileserver kernel: Killed process 9190 (smbd) > total-vm:3370368kB, anon-rss:2730796kB, file-rss:1292kB > Apr 6 14:43:41 fileserver kernel: afpd invoked oom-killer: > gfp_mask=0x24201ca, order=0, oom_score_adj=0 > > Apr 6 14:16:14 fileserver kernel: Out of memory: Kill process 9190 > (smbd) score 552 or sacrifice child > Apr 6 14:43:42 fileserver kernel: Out of memory: Kill process 11611 > (smbd) score 552 or sacrifice child > Apr 6 15:34:28 fileserver kernel: Out of memory: Kill process 12022 > (smbd) score 549 or sacrifice childforgot to mention: don't matter if it's a windows or apple client, from the moment on you throw "vfs objects" in the mix the problem starts more or less randomly the crashes above was short after a hard restart becuase added "nt acl support = no" everywhere to work around braindead apple clients overriding "inherit permissions" (http://serverfault.com/questions/728667/file-permissions-issue-on-os-x-el-capitan-and-smb-share) hosts deny = all hide files = /.AppleDesktop/.AppleDouble/.Parent/desktop.ini/$RECYCLE.BIN/ veto files = /.AppleDesktop/.AppleDouble/.Parent/desktop.ini/$RECYCLE.BIN/ delete veto files = yes access based share enum = yes hide unreadable = yes inherit permissions = yes inherit acls = yes nt acl support = no browseable = yes writeable = yes guest ok = no wide links = no follow symlinks = no oplocks = no level2 oplocks = no vfs objects = catia fruit streams_xattr -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/samba/attachments/20160406/275989fc/signature.sig>
On Wed, Apr 06, 2016 at 03:43:12PM +0200, Reindl Harald wrote:> > > Am 05.04.2016 um 13:22 schrieb Ralph Boehme: > >>Am 05.04.2016 um 13:03 schrieb Ralph Boehme: > >>>On Tue, Apr 05, 2016 at 12:43:37PM +0200, Reindl Harald wrote: > >>>>what is wasting that much RAM if vfs_fruit is enabled? > >>>> > >>>>had 4 OOM-killers today after not face a single one in the last 7 years and > >>>>on a different machine i swa a smbd process eating up 6500 MB on a HP > >>>>Microserver with 8 GB RAM due copy a 300 GB folder with a Apple client and > >>>>even not release the memory after the operation was finished > >>>> > >>>>root 24301 1.9 22.3 1323692 903600 ? SN 12:29 0:13 > >>>>/usr/sbin/smbd -D > >>>> > >>>>samba-4.3.6-0.fc23.x86_64 > >>> > >>>just got a private mail from someone else who ran into the same issue. > >>> > >>>Can you run smbd under valgrind with memcheck? > >> > >>not that easy in production and it seems to be unpredictable to trigger > > > >it was mentioned that it is reproducible by deleting files and folders > > for sure it's enough copy a folder with many elements, the crash below was > my boss trying to copy a folder with 880 files over a slow VPN tunnel which > produced 3 crashes so far and given that i raised the RAM of the virtual > machine yesterday from 4 GB to 5 GB there is no end in ressource usage until > it crashsif you can reproduce it, great! valgrind memcheck please. I'm too busy atm to do it myself. Cheerio! -slow
Am 06.04.2016 um 15:58 schrieb Ralph Boehme:> On Wed, Apr 06, 2016 at 03:43:12PM +0200, Reindl Harald wrote: >> >> Am 05.04.2016 um 13:22 schrieb Ralph Boehme: >>>> Am 05.04.2016 um 13:03 schrieb Ralph Boehme: >>>>> On Tue, Apr 05, 2016 at 12:43:37PM +0200, Reindl Harald wrote: >>>>>> what is wasting that much RAM if vfs_fruit is enabled? >>>>>> >>>>>> had 4 OOM-killers today after not face a single one in the last 7 years and >>>>>> on a different machine i swa a smbd process eating up 6500 MB on a HP >>>>>> Microserver with 8 GB RAM due copy a 300 GB folder with a Apple client and >>>>>> even not release the memory after the operation was finished >>>>>> >>>>>> root 24301 1.9 22.3 1323692 903600 ? SN 12:29 0:13 >>>>>> /usr/sbin/smbd -D >>>>>> >>>>>> samba-4.3.6-0.fc23.x86_64 >>>>> >>>>> just got a private mail from someone else who ran into the same issue. >>>>> >>>>> Can you run smbd under valgrind with memcheck? >>>> >>>> not that easy in production and it seems to be unpredictable to trigger >>> >>> it was mentioned that it is reproducible by deleting files and folders >> >> for sure it's enough copy a folder with many elements, the crash below was >> my boss trying to copy a folder with 880 files over a slow VPN tunnel which >> produced 3 crashes so far and given that i raised the RAM of the virtual >> machine yesterday from 4 GB to 5 GB there is no end in ressource usage until >> it crashs > > if you can reproduce it, great! valgrind memcheck please. I'm too busy > atm to do it myselfask me while beeing sysadmin for 30 servers, db-admin, mail-admin, filserver-admin, dns-admin, software-developer and tech support for some hundret domains and responsible for any technical stuff down to storage-arrays and virtualization..... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/samba/attachments/20160406/32a8c521/signature.sig>