samba - Jun 2017 - Successful compile / i686 "LARGEFILE64" question (was: Lots of RPC-related compile errors trying to update Samba from 3.5 to 4.6)

On Thu, 2017-06-08 at 11:28 +0200, awl1 via samba wrote:
> Hello Andrew,
> 
> and many thanks for your fast reply! :-)
> 
> Am 08.06.2017 um 11:05 schrieb Andrew Bartlett:
> > I think the key to avoiding the issue is that you have listed rpc 
> > modules which are not modules. The only rpc server module is 
> > rpc_mdssvc_module.
> 
> Ah, OK, I see - so I seem to have been wrong when assuming that I could 
> take the output of Samba 3.5.16 "smbd -b" command and use it to
feed the
> list of "builtin modules" it returns as the value for 
> "--with-static-modules"...
> 
> ==> Builtin modules:
>      pdb_ldap pdb_smbpasswd pdb_tdbsam pdb_wbc_sam rpc_lsarpc rpc_winreg 
> rpc_initshutdown rpc_dssetup rpc_wkssvc rpc_svcctl rpc_ntsvcs 
> rpc_netlogon rpc_netdfs rpc_srvsvc rpc_spoolss rpc_eventlog rpc_samr 
> idmap_ldap idmap_tdb idmap_passdb idmap_nss nss_info_template auth_sam 
> auth_unix auth_winbind auth_wbc auth_server auth_domain auth_builtin 
> auth_netlogond vfs_default vfs_posixacl
> ==> 
> So is there any documentation available, and/or can you please advise 
> how to determine the appropriate values for
> 
> --with-static-modules=...
> --with-shared-modules=...
The defaults suit most folks.  Otherwise, take a look at
script/autobuild.py for some tested combinations, including some of the
magic incantations for all and none.
> and possibly other related 4.6.x configure parameters from an old 3.5.x 
> installin order to compile a Samba 4.6.5 service with (at least) the 
> exact same (or equivalent) capabilities as the original Thecus 3.5.16 
> version?
I would focus on these:

 --with Options:
   WITH_ADS
   WITH_AIO
   WITH_CIFSMOUNT
   WITH_PAM
   WITH_PAM_MODULES
   WITH_PROFILE
   WITH_QUOTAS
   WITH_SENDFILE
   WITH_SYSLOG
   WITH_UTMP
   WITH_WINBIND

Of these, AIO is now on by default (no option any more) and the rest
are i think on by default (if found) but you may wish to ensure you
have the right headers for them to build.

The default is now --with-ads-support and it will whine if you don't
have the things that needs.  This, and quotas is the main thing I would
worry about, in terms of user-visible features.

I trust this helps,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba

Hello Andrew, hello Samba experts,

first of all, good news: So I have indeed managed to compile a set co 
cross-compiled binaries and installed them successfully onto my NAS. 
Many thanks for your help in getting there!

I have now only used

                 --bundled-libraries=ALL \

and left "--with-static-modules" unset (default).

 From cursorily testing so far, indeed everything seems to be working 
fine in terms of functionality - so far, I also have not made any 
changes to the 3.5.16 smb.conf, but simply used the exact same old 
config file for 4.6.5.

But I need to raise a performance-related issue about a severe 
performance degradation when using SMB version 3.1.1 dialect (as opposed 
to SMB 1.5) with a huge number of small files - I will do so shortly in 
a separate mail to this alias - and I have one more question regarding 
the comparison of "smbd -b" output between Thecus 3.5.16 and my new
4.6.5:


I have noticed that on my 32-bit NAS (Intel Atom), the "smbd -b"
output
of the old Thecus version explicitly lists several file handling system 
calls as 64-bit capable versions:

$ grep 64 smbd-b_thecus.txt
    HAVE_AIOCB64
    HAVE_AIO_CANCEL64
    HAVE_AIO_ERROR64
    HAVE_AIO_FSYNC64
    HAVE_AIO_READ64
    HAVE_AIO_RETURN64
    HAVE_AIO_SUSPEND64
    HAVE_AIO_WRITE64
    HAVE_CREAT64
    HAVE_FOPEN64
    HAVE_FSEEKO64
    HAVE_FSTAT64
    HAVE_FTELLO64
    HAVE_FTRUNCATE64
    HAVE_LSEEK64
    HAVE_LSTAT64
    HAVE_OPEN64
    HAVE_POSIX_FALLOCATE64
    HAVE_PREAD64
    HAVE_PWRITE64
    HAVE_READDIR64
    HAVE_SENDFILE64
    HAVE_STAT64
    HAVE_STRUCT_DIRENT64
    HAVE___OPEN64
    HAVE___PREAD64
    HAVE___PWRITE64
    _LARGEFILE64_SOURCE

while the output of my freshly cross-compiled version does not have any 
single of them:

$ grep 64 smbd-b_new.txt
    HAVE_INT64_T
    HAVE_UINT64_T
    SIZEOF_INT64_T
    SIZEOF_UINT64_T

Does this mean that my new version will handle large files (with a 
length of more than fits into 32 bits) inefficiently (or even not at 
all)? Or is this difference in output expected and I simply don't need 
to worry.

Note that in my compile options, I have explicitly used

-D_LARGE_FILES -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64

as part of my CCFLAGS, so I am really wondering why those 64-bits 
versions do not show up with 4.6.5 "smbd -b" output...!?

Can you please throw some light on this?

Thanks again & best regards
Andreas


Am 08.06.2017 um 11:57 schrieb Andrew Bartlett via samba:
>> So is there any documentation available, and/or can you please advise
>> how to determine the appropriate values for
>>
>> --with-static-modules=...
>> --with-shared-modules=...
> The defaults suit most folks.  Otherwise, take a look at
> script/autobuild.py for some tested combinations, including some of the
> magic incantations for all and none.
>
>> and possibly other related 4.6.x configure parameters from an old 3.5.x
>> installin order to compile a Samba 4.6.5 service with (at least) the
>> exact same (or equivalent) capabilities as the original Thecus 3.5.16
>> version?
> I would focus on these:
>
>   --with Options:
>     WITH_ADS
>     WITH_AIO
>     WITH_CIFSMOUNT
>     WITH_PAM
>     WITH_PAM_MODULES
>     WITH_PROFILE
>     WITH_QUOTAS
>     WITH_SENDFILE
>     WITH_SYSLOG
>     WITH_UTMP
>     WITH_WINBIND
>
> Of these, AIO is now on by default (no option any more) and the rest
> are i think on by default (if found) but you may wish to ensure you
> have the right headers for them to build.
>
> The default is now --with-ads-support and it will whine if you don't
> have the things that needs.  This, and quotas is the main thing I would
> worry about, in terms of user-visible features.
>
> I trust this helps,
>
> Andrew Bartlett

On Tue, 2017-06-13 at 11:03 +0200, awl1 wrote:
> Hello Andrew, hello Samba experts,
> 
> first of all, good news: So I have indeed managed to compile a set co 
> cross-compiled binaries and installed them successfully onto my NAS. 
> Many thanks for your help in getting there!
> while the output of my freshly cross-compiled version does not have any 
> single of them:
> 
> $ grep 64 smbd-b_new.txt
>     HAVE_INT64_T
>     HAVE_UINT64_T
>     SIZEOF_INT64_T
>     SIZEOF_UINT64_T
> 
> Does this mean that my new version will handle large files (with a 
> length of more than fits into 32 bits) inefficiently (or even not at 
> all)? Or is this difference in output expected and I simply don't need 
> to worry.
Don't read too much into the full set of defines - there has been a
total rewrite of the build system between the versions you indicate.
> Note that in my compile options, I have explicitly used
> 
> -D_LARGE_FILES -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
> 
> as part of my CCFLAGS, so I am really wondering why those 64-bits 
> versions do not show up with 4.6.5 "smbd -b" output...!?
These do not need to be forced, and in general should not be set.  We
set the right things in our waf configure system, and overriding them
is just going to confuse things. 

In regard to your particular concerns we no longer use the transitional
read64() etc functions.

Samba is by default and has been 64-bit clean for file access for most
of two decades. 

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba