On Mon, 2020-11-09 at 08:55 -0800, Jeremy Allison via samba wrote:> On Mon, Nov 09, 2020 at 04:01:47PM +0100, Benedikt Kale? via samba > wrote: > > Dear List, > > > > I updated to samba 4.12.10-SerNet-Debian-9.buster from 4.12.9 > > and did a > > > > gluster volume set volume performance.write-behind off > > > > Now I get this in /var/log/syslog and randomly people don't get > > access > > to a shae. > > Can you load the debug symbols please ? That should give us > more data on where this is failing. > > Thanks ! > > Jeremy.Jeremy, To lock this down properly we might need to put (#include) the vfs module into a cmocka test, mocking the various gluster calls so we cover these codepaths. Andrew Bartlett -- Andrew Bartlett https://samba.org/~abartlet/ Authentication Developer, Samba Team https://samba.org Samba Developer, Catalyst IT https://catalyst.net.nz/services/samba
On Tue, Nov 10, 2020 at 07:41:56AM +1300, Andrew Bartlett wrote:> On Mon, 2020-11-09 at 08:55 -0800, Jeremy Allison via samba wrote: > > On Mon, Nov 09, 2020 at 04:01:47PM +0100, Benedikt Kale? via samba > > wrote: > > > Dear List, > > > > > > I updated to samba 4.12.10-SerNet-Debian-9.buster from 4.12.9 > > > and did a > > > > > > gluster volume set volume performance.write-behind off > > > > > > Now I get this in /var/log/syslog and randomly people don't get > > > access > > > to a shae. > > > > Can you load the debug symbols please ? That should give us > > more data on where this is failing. > > > > Thanks ! > > > > Jeremy. > > Jeremy, > > To lock this down properly we might need to put (#include) the vfs > module into a cmocka test, mocking the various gluster calls so we > cover these codepaths.I'm just worried my talloc tidyups might be at fault, but I honestly can't see where. Once we get a backtrace (and if it's not my fault :-) I'm going to be happy to throw this over to Guenther, Sachin and Anoop for test coverage :-).
On Tue, 2020-11-10 at 07:41 +1300, Andrew Bartlett via samba wrote:> > To lock this down properly we might need to put (#include) the vfs > module into a cmocka test, mocking the various gluster calls so we > cover these codepaths. >We do test this automatically using the samba-integration project at https://github.com/gluster/samba-integration This includes a Samba-CTDB setup using GlusterFS. This includes the latest version of GlusterFS with Samba-CTDB. This particular issue of the write-behind data corruption was caught through this testing infrastructure. However since the latest versions of Glusterfs we use does contain the passthrough option, we never call this code path for checking the volfile of the GlusterFS cluster in our test environment. This is a weekness we need to address. We are also working to increasing the number of tests we run in the environment and I am working through the test failures we see with other smb2 smbtorture tests. Sachin Prabhu
Hi Jeremy, I'm afraid this is indeed caused by the talloc tidyup as one can perfectly reproduce it wih the following patch for smbtorture (call bin/smbtorture ncacn_np:lo local.file.file_lines_parse). I guess we need to set up some warning signs around file_lines_parse() expressing that it takes talloced content and that one should not free the separated lines array separately. Cheers, Guenther On 09/11/2020 19:55, Jeremy Allison via samba wrote:> On Tue, Nov 10, 2020 at 07:41:56AM +1300, Andrew Bartlett wrote: >> On Mon, 2020-11-09 at 08:55 -0800, Jeremy Allison via samba wrote: >>> On Mon, Nov 09, 2020 at 04:01:47PM +0100, Benedikt Kale? via samba >>> wrote: >>>> Dear List, >>>> >>>> I updated to samba 4.12.10-SerNet-Debian-9.buster from 4.12.9 >>>> and did a >>>> >>>> gluster volume set volume performance.write-behind off >>>> >>>> Now I get this in /var/log/syslog and randomly people don't get >>>> access >>>> to a shae. >>> >>> Can you load the debug symbols please ? That should give us >>> more data on where this is failing. >>> >>> Thanks ! >>> >>> Jeremy. >> >> Jeremy, >> >> To lock this down properly we might need to put (#include) the vfs >> module into a cmocka test, mocking the various gluster calls so we >> cover these codepaths. > > I'm just worried my talloc tidyups might be at fault, > but I honestly can't see where. > > Once we get a backtrace (and if it's not my fault :-) > I'm going to be happy to throw this over to Guenther, > Sachin and Anoop for test coverage :-). >-- G?nther Deschner GPG-ID: 8EE11688 Red Hat gdeschner at redhat.com Samba Team gd at samba.org -------------- next part -------------- A non-text attachment was scrubbed... Name: test.patch Type: text/x-patch Size: 1628 bytes Desc: not available URL: <http://lists.samba.org/pipermail/samba/attachments/20201110/d055946c/test.bin>