samba - Aug 2020 - smbclient mask command seems not to work the same way with recurse ON for mget and mput

Dear fellows.

Another piece of information. The issue reprduces on RHEL 7.7, Samba 4.9.1

[root at vnhprerhds01 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.7 (Maipo)
[root at vnhprerhds01 ~]# smbclient -V
Version 4.9.1

[root at vnhprerhds01 ~]# smbclient -W "${d}" -U "${u}"
"${s}" "${p}"
Try "help" to get a list of possible commands.
smb: \> !mkdir -p /tmp/borrame/a
smb: \> !mkdir -p /tmp/borrame/b
smb: \> !touch /tmp/borrame/a/AM.xls
smb: \> !touch /tmp/borrame/a/AT.xls
smb: \> !touch /tmp/borrame/b/BT.xls
smb: \> lcd /tmp/borrame
smb: \> cd borrar
smb: \borrar\> recurse
smb: \borrar\> prompt
smb: \borrar\> mask *M.xls
smb: \borrar\> mput *
putting file a/AT.xls as \borrar\a\AT.xls (0,0 kb/s) (average 0,0 kb/s)
putting file a/AM.xls as \borrar\a\AM.xls (0,0 kb/s) (average 0,0 kb/s)
putting file b/BT.xls as \borrar\b\BT.xls (0,0 kb/s) (average 0,0 kb/s)

I'll be trying next compiling 4.12.6 on OEL 6, which will take some time to
resolve dependencies, and if I am not able in a reasonable amount of time,
will do it on RHEL 7.7. It is really hard to believe this has not been
detected by noone else along the years. So I'm sure I'm doing something
wrong, seem so clumsy though to detect it myself.

Let you know about the progress. Best regards.


El lun., 24 ago. 2020 a las 20:16, LPC DPG (<lpcdpg at gmail.com>)
escribi?:
> Dear Rowland.
>
> So sorry, I did not get the responses to this e-mail address, got them by
> the purest change googling again.
>
> I'm sorry to disagree with your approach:
>
> 1-. In my most recent updates you should have seen I am compiling Samba4
> from source. Began with packaged distribution, but considering it might be
> a bug, I decided to test myself. 4 different versions, same behaviour. This
> means in my humble opinion it has nothing to do with the operating system
> version or its "commercial" support.
>
> 2-. This is a basic feature; there is nothing strange in how the DC or the
> Linux server are configured. So a basic issue with a basic feature
> suggests, again in my humble opinion, there is a kind of misunderstanding
> or regression bug here.
>
> Will you please confirm it should works as described by man? I have look
> into client.c, and the way cmd_mget and cmd_mput are programmed differs.
>
> Many thanks. Best regards.
>
> El lun., 24 ago. 2020 a las 14:48, LPC DPG (<lpcdpg at gmail.com>)
escribi?:
>
>> A new update. Same behaviour with 4.6.16. Exact issue as Bug 1249.
>>
>> Many thanks. Kind regards.
>>
>> El lun., 24 ago. 2020 a las 12:23, LPC DPG (<lpcdpg at
gmail.com>) escribi?:
>>
>>> Dear folks.
>>>
>>> Was hoping it had to do with the release, but have also tested in
4.4.16
>>> and the issue is also there. I am aware a RHEL/CEntOS based upon 6
>>> distribution is not the most up to date version I should test this
on, as
>>> it also limits how easy is compiling most recent versions of Samba,
but I
>>> assume though this is not a tricky test, and should work as
documented in
>>> both 4.4 and 4.2 versions.
>>>
>>> Looking forward to reading your remarks.
>>>
>>> Many thanks in advance. Best regards.
>>>
>>> El vie., 21 ago. 2020 a las 12:25, LPC DPG (<lpcdpg at
gmail.com>)
>>> escribi?:
>>>
>>>> Dear folks.
>>>>
>>>> I am using smbclient 4.2.10 version (the one packaged for OEL
6,
>>>> samba4-client-4.2.10-15.el6.x86_64), and observing a weird
behaviour of the
>>>> tool.
>>>>
>>>> I've got a Linux local directorio, say /tmp/eraseme, with a
couple of
>>>> subdirectories in it, say a and b, each one containing a dummy
file, say
>>>> a/AM.xls and b/AT.xls
>>>>
>>>> If I run:
>>>> smbclient -d 30 -l /var/log/samba -m SMB3 -W "${d}"
-U "${u}" "${s}"
>>>> "${p}"
>>>> smb: \> lcd /tmp/eraseme
>>>> smb: \> recurse
>>>> smb: \> prompt
>>>> smb: \> mask *M.xls
>>>> smb: \> mput *
>>>>
>>>> it transfer both AM.xls an AT.xls files, which seems not to
agree what
>>>> man page states:
>>>>
>>>> "       mask <mask>
>>>>            This command allows the user to set up a mask which
will be
>>>> used during recursive operation of the mget and mput commands.
>>>>
>>>>            The masks specified to the mget and mput commands
act as
>>>> filters for directories rather than files when recursion is
toggled ON.
>>>>
>>>>            The mask specified with the mask command is
necessary to
>>>> filter files within those directories. For example, if the mask
specified
>>>> in an
>>>>            mget command is "source*" and the mask
specified with the
>>>> mask command is "*.c" and recursion is toggled ON,
the mget command will
>>>> retrieve
>>>>            all files matching "*.c" in all
directories below and
>>>> including all directories matching "source*" in the
current working
>>>> directory.
>>>>
>>>>            Note that the value for mask defaults to blank
(equivalent
>>>> to "*") and remains so until the mask command is used
to change it. It
>>>> retains
>>>>            the most recently specified value indefinitely. To
avoid
>>>> unexpected results it would be wise to change the value of mask
back to "*"
>>>> after
>>>>            using the mget or mput commands."
>>>>
>>>> Nonetheless, if I do it the other way around,
>>>>
>>>> smbclient -d 30 -l /var/log/samba -m SMB3 -W "${d}"
-U "${u}" "${s}"
>>>> "${p}"
>>>> smb: \> lcd /tmp/eraseme
>>>> smb: \> recurse
>>>> smb: \> prompt
>>>> smb: \> mask *M.xls
>>>> smb: \> mget *
>>>>
>>>> , mget does seem to honour the referred description,
downloading only
>>>> AM.xls file to Linux.
>>>>
>>>> I have been looking in Google, but have seen no reference to
similar
>>>> problems. I do assume hence whether I am doing something wrong,
or
>>>> misinterpreting the manual, then I can't get it working.
>>>>
>>>> So I'd be really grateful if any of you who might have ever
faced
>>>> something similar could share the experience, as I am really
lost with it.
>>>>
>>>> Thanks so much in advance. Best regards.
>>>>
>>>

On Mon, Aug 24, 2020 at 09:15:48PM +0200, LPC DPG via samba
wrote:
> Dear fellows.
> 
> Another piece of information. The issue reprduces on RHEL 7.7, Samba 4.9.1
> 
> [root at vnhprerhds01 ~]# cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.7 (Maipo)
> [root at vnhprerhds01 ~]# smbclient -V
> Version 4.9.1
> 
> [root at vnhprerhds01 ~]# smbclient -W "${d}" -U "${u}"
"${s}" "${p}"
> Try "help" to get a list of possible commands.
> smb: \> !mkdir -p /tmp/borrame/a
> smb: \> !mkdir -p /tmp/borrame/b
> smb: \> !touch /tmp/borrame/a/AM.xls
> smb: \> !touch /tmp/borrame/a/AT.xls
> smb: \> !touch /tmp/borrame/b/BT.xls
> smb: \> lcd /tmp/borrame
> smb: \> cd borrar
> smb: \borrar\> recurse
> smb: \borrar\> prompt
> smb: \borrar\> mask *M.xls
> smb: \borrar\> mput *
> putting file a/AT.xls as \borrar\a\AT.xls (0,0 kb/s) (average 0,0 kb/s)
> putting file a/AM.xls as \borrar\a\AM.xls (0,0 kb/s) (average 0,0 kb/s)
> putting file b/BT.xls as \borrar\b\BT.xls (0,0 kb/s) (average 0,0 kb/s)
All the 'mask' command does is set a global string that
is returned by the smbclient internal function: client_get_fileselection().

Here is the code that decides if smbclient should do anything
with a given filename:

/*******************************************************************
 Decide if a file should be operated on.
********************************************************************/

static bool do_this_one(struct file_info *finfo)
{
        if (!finfo->name) {
                return false;
        }

        if (finfo->attr & FILE_ATTRIBUTE_DIRECTORY) {
                return true;
        }

        if (*client_get_fileselection() &&
            !mask_match(finfo->name,client_get_fileselection(),false)) {
                DEBUG(3,("mask_match %s failed\n", finfo->name));
                return false;
        }
.....

Note, all it's doing is a mask_match() call against the
returned name and the string set via the 'mask' parameter.

Note that when you're doing 'recurse' and the operation
you set you're providing a second mask parameter '*'
in this case.

Looks like the function do_list_helper() only looks
at the mask set via the 'mask' command if it's not
recursing.

Looks like this might be a bug that no one ever ran
into before. Normally, people expect the mask given
to the mXXXX functions to take priority. I'm guessing
almost no one uses the 'mask' command.

Dear Jeremy.

Really appreciate you took your time to answer. I had already reviewed
source4/client/client.c looking for mput (cmd_mput) command, but unluckily
saying that my C programming skills are poor woud be really overrating them.

I'm sorry to disagree for two reasons

1-. It works for mget,  true that the source code is completely different.

2-. If taking a look into smbclient man latest release:

"
mask <mask>

This command allows the user to set up a mask which will be used during
recursive operation of the mget and mput commands.

The masks specified to the mget and mput commands act as filters for
directories rather than files when recursion is toggled ON.

The mask specified with the mask command is necessary to filter files
within those directories\&. For example, if the mask specified in an mget
command is "source*" and the mask specified with the mask command is
"*.c"
and recursion is toggled ON, the mget command will retrieve all files
matching "*.c" in all directories below and including all directories
matching "source*" in the current working directory.

Note that the value for mask defaults to blank (equivalent to "*") and
remains so until the mask command is used to change it\&. It retains the
most recently specified value indefinitely. To avoid unexpected results it
would be wise to change the value of mask back to "*" after using the
mget
or mput commands.
"
I am no native English speaker, but as far as my understanding of your
language goes, it confirms that both * are necessary when recurse is on.
This is my interpretation.
a) recurse OFF; mask *.lsx; mput *.xls -> will transfer all (MMultiple) the
files ending with .xls, but only in the current directory. The mask *.lsx
is ignored, as recurse is off.
b)  a- recurse ON; mask *.lsx; mput *.xls -> should only transfer
directories existing in the current directory whose name end with .xls,
very unlikely, probably none; should they exist, it would transfer files
whose name end with .lsx.
c) a- recurse ON; mask *.lsx; mput * -> should transfer any file ending
with .lsx (mask *.lsx) in any (*) directory contained in the current
directory.
d) a- recurse ON; mask *.lsx; mput -> the very same es c).

So, my interpretation is to be wrong for sure, but I think it is what the
man states. In this case, there should be a documentation bug. Otherwise, I
really can't explain no one (but Ruslan, nearly 16 years ago) has ever
happened to run such filter. What's more, after testing with versions
4.2.10, 4.2.14, 4.4.16, 4.6.16 and 4.9.1 in two differente Fedora based
distributions, and having taken a look (with my referred limitations) to
client.c, I'd say mput has never been able to work with mask, unlike mget.

Still trying to compile Samba 4.12.6 in OEL 6.8, though it's a bit tricky
as I must manually compile GMP, Nettle, P11, LibFFI, GNUTLS, etc.
dependencies with versions much newer versions than the ones installable
from RPM, but I am beginning to assume I will find the very same behaviour.

Again, a ton of thanks for your time to review this issue. Best regards.

El lun., 24 ago. 2020 a las 22:33, Jeremy Allison (<jra at samba.org>)
escribi?:
> On Mon, Aug 24, 2020 at 09:15:48PM +0200, LPC DPG via samba wrote:
> > Dear fellows.
> >
> > Another piece of information. The issue reprduces on RHEL 7.7, Samba
> 4.9.1
> >
> > [root at vnhprerhds01 ~]# cat /etc/redhat-release
> > Red Hat Enterprise Linux Server release 7.7 (Maipo)
> > [root at vnhprerhds01 ~]# smbclient -V
> > Version 4.9.1
> >
> > [root at vnhprerhds01 ~]# smbclient -W "${d}" -U
"${u}" "${s}" "${p}"
> > Try "help" to get a list of possible commands.
> > smb: \> !mkdir -p /tmp/borrame/a
> > smb: \> !mkdir -p /tmp/borrame/b
> > smb: \> !touch /tmp/borrame/a/AM.xls
> > smb: \> !touch /tmp/borrame/a/AT.xls
> > smb: \> !touch /tmp/borrame/b/BT.xls
> > smb: \> lcd /tmp/borrame
> > smb: \> cd borrar
> > smb: \borrar\> recurse
> > smb: \borrar\> prompt
> > smb: \borrar\> mask *M.xls
> > smb: \borrar\> mput *
> > putting file a/AT.xls as \borrar\a\AT.xls (0,0 kb/s) (average 0,0
kb/s)
> > putting file a/AM.xls as \borrar\a\AM.xls (0,0 kb/s) (average 0,0
kb/s)
> > putting file b/BT.xls as \borrar\b\BT.xls (0,0 kb/s) (average 0,0
kb/s)
>
> All the 'mask' command does is set a global string that
> is returned by the smbclient internal function: client_get_fileselection().
>
> Here is the code that decides if smbclient should do anything
> with a given filename:
>
> /*******************************************************************
>  Decide if a file should be operated on.
> ********************************************************************/
>
> static bool do_this_one(struct file_info *finfo)
> {
>         if (!finfo->name) {
>                 return false;
>         }
>
>         if (finfo->attr & FILE_ATTRIBUTE_DIRECTORY) {
>                 return true;
>         }
>
>         if (*client_get_fileselection() &&
>             !mask_match(finfo->name,client_get_fileselection(),false)) {
>                 DEBUG(3,("mask_match %s failed\n",
finfo->name));
>                 return false;
>         }
> .....
>
> Note, all it's doing is a mask_match() call against the
> returned name and the string set via the 'mask' parameter.
>
> Note that when you're doing 'recurse' and the operation
> you set you're providing a second mask parameter '*'
> in this case.
>
> Looks like the function do_list_helper() only looks
> at the mask set via the 'mask' command if it's not
> recursing.
>
> Looks like this might be a bug that no one ever ran
> into before. Normally, people expect the mask given
> to the mXXXX functions to take priority. I'm guessing
> almost no one uses the 'mask' command.
>