samba - May 2020 - Win7 clients problem after upgrading samba file server to 4.12 on Arch

@Andrew
Unfortunately I didn't make notes on exact time of errors when doing my
tests...

But, I've spend last couple of hours on building samba from master:

   - Have made a clone of the virtual machine, with new name and network
   interface mac address, and a new dynamic IP assigned by my router.
   - Uninstalled Arch packages: samba, smbclient and cifs-tools (depending
   on smbclient)
   - As 'ala' downloaded
   https://download.samba.org/pub/samba/samba-latest.tar.gz and unpacked
   it, obtaining 'samba-4.12.2' directory
   - within this directory - run './configure' without any parameters
   - run 'make' without any parameters
   - in the meantime I had to install two small packages needed to complete
   the build
   - run 'sudo make install'
   - than had to figure out all differences in directories layouts, between
   master and arch layouts, correct PATH
   - configured smb and nmb services definitions for systemd (copy-pasted
   from Arch and amended according to new directories)
   - run 'sudo systemctl start nmb smb' - YES! successfully! (after a
tenth
   try.. also avahi was missing....)
   - connected to the new 'nas' from my Win7 clients, did some random
walks
   through mapped directories on both clients...

And what?
Sad news. The problem persists.

Well... Definitely need some sleep here.... Hope to hearing from you the
next day.

Thanks for support!
I have already learned a lot... Apparently will learn even more... Never
too old to learn... ;)

Regards, Pawel

wt., 5 maj 2020 o 00:15 Andrew Bartlett <abartlet at samba.org>
napisa?(a):
> Did you note the very second you saw the offending error message?
>
> That time would help a lot, I can't see the error packet.
>
> But all that said, I'm not the person who needs to look at this, I just
> hoped I could help get you the information they would need.
>
> The only other suggestion, other than the build on master in the hope
> this has already been fixed, is if you do find you can build Samba
> easily, is to do a full 'git bisect' between the last version that
> failed and the first version that worked, and pin it down that way.
>
> Sorry!
>
> Andrew Bartlett
>
> On Tue, 2020-05-05 at 00:04 +0200, pavlos via samba wrote:
> > Forgot to add the tcpdump command used:
> >
> > tcpdump -p -s 0 -n -w /root/tcpdump_all.txt
> >
> >
> > Pawel.
> >
> > wt., 5 maj 2020 o 00:01 pavlos <pavlos.audiofreak at gmail.com>
> > napisa?(a):
> >
> > > OK, have repeated the test:
> > >
> > >    - samba logs:
> > >    http://www.mediafire.com/file/jy4qcyvqg6z39zu/logs.tar.gz/file
> > >    - tcpdump log:
> > >
> > >
http://www.mediafire.com/file/qzex3kxf2vzd87n/tcpdump_all.tar.gz/file
> > >
> > >
> > > The tcpdump log covers the whole period of testing time, from
> > > starting to
> > > stopping smb and nmb services.
> > >
> > > Previous logs deleted from mediafire.
> > >
> > > Now waiting for any advice from you, or let's start doing a
clean
> > > compile.
> > >
> > > Pawel.
> > >
> > > pon., 4 maj 2020 o 23:31 pavlos <pavlos.audiofreak at
gmail.com>
> > > napisa?(a):
> > >
> > > > This is the smbstatus with 4.11.x. No problems on clients.
> > > > Using LC_ALL=C because have Polish locale set up.
> > > >
> > > > [root at arch1 ~]# LC_ALL=C smbstatus
> > > >
> > > > Samba version 4.11.3
> > > > PID     Username     Group        Machine
> > > >                                   Protocol Version 
Encryption
> > > >           Signing
> > > >
---------------------------------------------------------------
> > > >
---------------------------------------------------------------
> > > > ----------
> > > >
> > > > 490     ala          users        192.168.1.108
> > > > (ipv4:192.168.1.108:50462)
> > > >  SMB2_10           -                    -
> > > > 493     nobody       nobody       192.168.1.103
> > > > (ipv4:192.168.1.103:60458)
> > > >  SMB2_10           -                    -
> > > >
> > > > Service      pid     Machine       Connected at
> > > >                     Encryption   Signing
> > > >
---------------------------------------------------------------
> > > > ------------------------------
> > > >
> > > > nas          493     192.168.1.103 Mon May  4 23:24:23 2020
> > > > CEST    -
> > > >            -
> > > > nas          490     192.168.1.108 Mon May  4 23:23:47 2020
> > > > CEST    -
> > > >            -
> > > >
> > > > Locked files:
> > > > Pid          User(ID)   DenyMode   Access      R/W       
Oplock
> > > >           SharePath   Name   Time
> > > >
---------------------------------------------------------------
> > > > -----------------------------------
> > > >
> > > > 490          1000       DENY_ALL   0x100080    RDONLY    
NONE
> > > >             /nas   .   Mon May  4 23:23:47 2020
> > > > 490          1000       DENY_NONE  0x100081    RDONLY    
NONE
> > > >             /nas   .   Mon May  4 23:23:54 2020
> > > > 493          65534      DENY_NONE  0x100081    RDONLY    
NONE
> > > >             /nas   .   Mon May  4 23:24:22 2020
> > > >
> > > >
> > > >
> > > > pon., 4 maj 2020 o 23:25 Andrew Bartlett <abartlet at
samba.org>
> > > > napisa?(a):
> > > >
> > > > > On Mon, 2020-05-04 at 23:18 +0200, pavlos wrote:
> > > > > > @Andrew,
> > > > > >
> > > > > > Samba master - what do you mean? A clean compile
and install
> > > > > > from
> > > > > > genuine samba.org sources? As a matter of fact, I
was
> > > > > > thinking about
> > > > > > it....
> > > > >
> > > > > Yes.
> > > > >
> > > > > > Let me do the tests now, with tcpdump. Will send
some logs
> > > > > > and outputs  later on.
> > > > > >
> > > > > > You've asked previously - this is the
smbstatus, already on
> > > > > > the
> > > > > > latest kernel/samba versions.
> > > > > > The problem is persistent, no wonder...
> > > > >
> > > > > I'm not an expert on SMB share modes (I mostly do
strange AD DC
> > > > > stuff),
> > > > > but a DENY_ALL over the root of the share might be
> > > > > problematic.  I
> > > > > normally thought deny modes were for files, not
folders...
> > > > >
> > > > > > > [root at arch1 /]# LC_ALL=C smbstatus
> > > > > > >
> > > > > > > Locked files:
> > > > > > > Pid          User(ID)   DenyMode   Access    
R/W        O
> > > > > > > plock
> > > > > > >           SharePath   Name   Time
> > > > > > >
---------------------------------------------------------
> > > > > > > --------
> > > > > > > ---------------------------------
> > > > > > > 386          65534      DENY_ALL   0x100080  
RDONLY     N
> > > > > > > ONE
> > > > > > >             /nas   .   Mon May  4 23:05:16
2020
> > > > > > > 369          1000       DENY_NONE  0x100081  
RDONLY     N
> > > > > > > ONE
> > > > > > >             /nas   .   Mon May  4 23:04:49
2020
> > > > >
> > > > > Hopefully someone with more FS expertise can help
clarify on
> > > > > this one
> > > > > way or another.  Is it any different on a
'good' version?
> > > > >
> > > > > Andrew Bartlett
> > > > >
> > > > > > Pawel.
> > > > > >
> > > > > > pon., 4 maj 2020 o 23:03 Andrew Bartlett
<abartlet at samba.org>
> > > > > > napisa?(a):
> > > > > > > Thanks for getting back to us,
> > > > > > >
> > > > > > > On Mon, 2020-05-04 at 22:57 +0200, pavlos via
samba wrote:
> > > > > > > > @Rowland,
> > > > > > > > Installing and changing to LTS kernel
5.4.38-1-lts
> > > > > > > > doesn't help.
> > > > > > > > Downgrading samba and smbclient to
4.11.3 DOES help.
> > > > > > > >
> > > > > > > > @Andrew, let me first clean up with
kernel/samba
> > > > > > > > versions, set
> > > > > > >
> > > > > > > log
> > > > > > > > level to
> > > > > > > > 10 and log limit to 0, delete all logs
and repeat the
> > > > > > > > whole
> > > > > > >
> > > > > > > exercise.
> > > > > > > > Please advice how to get a network
capture, I am not that
> > > > > > >
> > > > > > > expert...
> > > > > > >
> > > > > > > Our standard advise on network packet capture
is here:
> > > > > > >
> > > > > > >
https://wiki.samba.org/index.php/Capture_Packets
> > > > > > >
> > > > > > > Finally, as you are on a rolling distro, can
you try Samba
> > > > > > > master,
> > > > > > > if
> > > > > > > only for giggles?
> > > > > > >
> > > > > > > Andrew Bartlett
> > > > > > >
> > > > > > > > Regards, Pawel.
> > > > > > > >
> > > > > > > > pon., 4 maj 2020 o 22:50 Andrew Bartlett
<
> > > > > > > > abartlet at samba.org>
> > > > > > > > napisa?(a):
> > > > > > > >
> > > > > > > > > Can you get a network capture, and
also ensure the logs
> > > > > > > > > have
> > > > > > >
> > > > > > > the
> > > > > > > > > exact
> > > > > > > > > time you see the failure in it (and
tell us that time,
> > > > > > > > > to the
> > > > > > > > > highest
> > > > > > > > > resolution you can)?  I can't
see anything suspicious,
> > > > > > > > > but at
> > > > > > >
> > > > > > > log
> > > > > > > > > level
> > > > > > > > > 10 the 'log limit = 50'
will cut in very fast, so it
> > > > > > > > > might have
> > > > > > > > > been
> > > > > > > > > lost.  Set that to 0.
> > > > > > > > >
> > > > > > > > > Andrew Bartlett
> > > > > > > > >
> > > > > > > > > On Mon, 2020-05-04 at 22:28 +0200,
pavlos via samba
> > > > > > > > > wrote:
> > > > > > > > > > Logs are here:
> > > > > > > > > >
> > > > > > >
> > > > > > >
> http://www.mediafire.com/file/qqwfkf3rsuv081k/logs.tar.gz/file
> > > > > > > > > >
> > > > > > > > > > 1. I have stopped smb and nmb
services,
> > > > > > > > > > 2. deleted all samba logs,
> > > > > > > > > > 3. addied 'log level -
10' to smb.conf,
> > > > > > > > > > 4. started services,
> > > > > > > > > > 5. performed few tests until
some problems appeared,
> > > > > > > > > > 6. stopped smb and nmb
services
> > > > > > > > > > 7. and grabbed all logs.
> > > > > > > > > > As this is test environment -
nothing to sanitize.
> > > > > > > > > > All logs
> > > > > > > > > > unchanged.
> > > > > > > > > > For better understanding:
> > > > > > > > > >
> > > > > > > > > >    - 192.168.1.103, netbios
name 'saloon' - Win7x32
> > > > > > > > > > client
> > > > > > > > > >    - 192.168.1.108, netbios
name 'attic' - win7x32
> > > > > > > > > > client
> > > > > > > > > >    - 192.168.1.115, hostname
'arch1' - the test Samba
> > > > > > > > > > server,
> > > > > > > > > > Arch
> > > > > > > > > > Linux
> > > > > > > > > >    5.6.10-arch1-1
> > > > > > > > > >
> > > > > > > > > > Regards, Pawel.
> > > > > > > > > >
> > > > > > > > > > pon., 4 maj 2020 o 21:45
Rowland penny via samba <
> > > > > > > > > > samba at lists.samba.org>
> > > > > > > > > > napisa?(a):
> > > > > > > > > >
> > > > > > > > > > > On 04/05/2020 20:21,
pavlos wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > All clients are
connecting either anonymously
> > > > > > > > > > > > read-only
> > > > > > >
> > > > > > > which
> > > > > > > > > > > > leverages the
'nobody' samba user, or using 'ala'
> > > > > > >
> > > > > > > user/pwd.
> > > > > > > > > > > > Both
> > > > > > > > > > > > have
> > > > > > > > > > > > same problems.
> > > > > > > > > > > >
> > > > > > > > > > > > No local
'ala' users on any client machine. In
> > > > > > > > > > > > Windows
> > > > > > >
> > > > > > > login
> > > > > > > > > > > > dialog I
> > > > > > > > > > > > am supplying
username as '192.168.1.115\ala' and
> > > > > > > > > > > > the
> > > > > > >
> > > > > > > defined
> > > > > > > > > > > > password
> > > > > > > > > > > > (....115 is the IP
address of the test samba
> > > > > > > > > > > > server).
> > > > > > > > > > >
> > > > > > > > > > > Try it without the the IP
i.e. just 'ala'
> > > > > > > > > > > >
> > > > > > > > > > > > I have also put
'log level = 10' to your
> > > > > > > > > > > > smb.conf, but
> > > > > > >
> > > > > > > the
> > > > > > > > > > > > hiccup
> > > > > > > > > > > > on
> > > > > > > > > > > > Win7 remains. As I
said - the stupid message box
> > > > > > > > > > > > in
> > > > > > >
> > > > > > > Windows
> > > > > > > > > > > > pops
> > > > > > > > > > > > up
> > > > > > > > > > > > randomly.
> > > > > > > > > > >
> > > > > > > > > > > This could be a Windows
error, but something might
> > > > > > > > > > > have
> > > > > > >
> > > > > > > changed
> > > > > > > > > > > in
> > > > > > > > > > > Samba
> > > > > > > > > > > that is now causing it.
> > > > > > > > > > > > This short exercise
has produced about 100 KB of
> > > > > > >
> > > > > > > compressed
> > > > > > > > > > > > logs.
> > > > > > > > > > > > How
> > > > > > > > > > > > might I share them
with you and all in the
> > > > > > > > > > > > mailing list?
> > > > > > > > > > >
> > > > > > > > > > > Post them somewhere
(after sanitising them) and
> > > > > > > > > > > then
> > > > > > >
> > > > > > > provide a
> > > > > > > > > > > link
> > > > > > > > > > > to it.
> > > > > > > > > > >
> > > > > > > > > > > Rowland
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > To unsubscribe from this
list go to the following
> > > > > > > > > > > URL and
> > > > > > >
> > > > > > > read
> > > > > > > > > > > the
> > > > > > > > > > > instructions:
> > > > > > >
> > > > > > > https://lists.samba.org/mailman/options/samba
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Andrew Bartlett
> > > > > > >
> > > > > > > https://samba.org/~abartlet/
> > > > > > > > > Authentication Developer, Samba
Team  https://samba.org
> > > > > > > > > Samba Developer, Catalyst IT
> > > > > > > > >
https://catalyst.net.nz/services/samba
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > >
> > > > > --
> > > > > Andrew Bartlett
> > > > > https://samba.org/~abartlet/
> > > > > Authentication Developer, Samba Team  https://samba.org
> > > > > Samba Developer, Catalyst IT
> > > > > https://catalyst.net.nz/services/samba
> > > > >
> > > > >
> > > > >
> > > > >
> --
> Andrew Bartlett                       https://samba.org/~abartlet/
> Authentication Developer, Samba Team  https://samba.org
> Samba Developer, Catalyst IT
> https://catalyst.net.nz/services/samba
>
>
>
>

On Tue, 2020-05-05 at 03:05 +0200, pavlos wrote:
> @Andrew
> Unfortunately I didn't make notes on exact time of errors when doing
> my tests...
> 
> But, I've spend last couple of hours on building samba from master:
> Have made a clone of the virtual machine, with new name and network
> interface mac address, and a new dynamic IP assigned by my router.
> Uninstalled Arch packages: samba, smbclient and cifs-tools (depending
> on smbclient)
> As 'ala' downloaded 
> https://download.samba.org/pub/samba/samba-latest.tar.gz and unpacked
> it, obtaining 'samba-4.12.2' directory
Thanks.  This gives you Samba 4.12.2.  However our development branch,
master is newer.  Could you try that?

git clone https://gitlab.com/samba-team/samba/

That can be downloaded from
https://gitlab.com/samba-team/samba/-/archive/master/samba-master.tar.bz2 
but if you use git, then you can really help us with the bisect.

But the rest of your steps look good.

The next step, assuming master still fails, is to do a bisect. 
http://webchick.net/node/99

You may need to run 'git clean -x -f -d' between steps.

This will be slow, particularly as you can't automate it (need test
with the client), but it is a very good way to find what the issue is.

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       https://samba.org/~abartlet/
Authentication Developer, Samba Team  https://samba.org
Samba Developer, Catalyst IT          
https://catalyst.net.nz/services/samba

My plans are as follows, step by step:
- clone the VM, clean and build the newest from gitlab, I can see it is
4.13.0pre1
- clone the VM. clean and build the latest supported 4.11.8
- create a new fresh VM with another distro, not Arch or Arch-based, build
the official 4.12.2 - to check/confirm that it is not distro-related
I will start playing in couple of hours, now I am at work...

Pawel.

wt., 5 maj 2020 o 03:33 Andrew Bartlett <abartlet at samba.org>
napisa?(a):
> On Tue, 2020-05-05 at 03:05 +0200, pavlos wrote:
> > @Andrew
> > Unfortunately I didn't make notes on exact time of errors when
doing
> > my tests...
> >
> > But, I've spend last couple of hours on building samba from
master:
> > Have made a clone of the virtual machine, with new name and network
> > interface mac address, and a new dynamic IP assigned by my router.
> > Uninstalled Arch packages: samba, smbclient and cifs-tools (depending
> > on smbclient)
> > As 'ala' downloaded
> > https://download.samba.org/pub/samba/samba-latest.tar.gz and unpacked
> > it, obtaining 'samba-4.12.2' directory
>
> Thanks.  This gives you Samba 4.12.2.  However our development branch,
> master is newer.  Could you try that?
>
> git clone https://gitlab.com/samba-team/samba/
>
> That can be downloaded from
> https://gitlab.com/samba-team/samba/-/archive/master/samba-master.tar.bz2
> but if you use git, then you can really help us with the bisect.
>
> But the rest of your steps look good.
>
> The next step, assuming master still fails, is to do a bisect.
> http://webchick.net/node/99
>
> You may need to run 'git clean -x -f -d' between steps.
>
> This will be slow, particularly as you can't automate it (need test
> with the client), but it is a very good way to find what the issue is.
>
> Thanks,
>
> Andrew Bartlett
>
> --
> Andrew Bartlett                       https://samba.org/~abartlet/
> Authentication Developer, Samba Team  https://samba.org
> Samba Developer, Catalyst IT
> https://catalyst.net.nz/services/samba
>
>
>
>