samba - Nov 2013 - Samba 4 DC and member server, rfc3207, winbind, printing, asynchronous I/O - Problems and Fixes

Dear all,

After 4 days of sleepless nights, I have manged to rebuild the samba farm.  I
believe the following discovery might interest our samba community.

------------------------------------------------
System setting:
I have deployed samba 4.1.0 system for my working organisation.  It comprised of
2 DCs and 1 member server.
2 DCs maintains AD for login and the member server host files for user access.

The installation of DCs and member server follows the samba corresponding
official how-tos.  For flawless file access, the domain provision was done with
RFC2307 in DCs.
------------------------------------------------
Note:
1.  Effective GID of AD users:  It is a must that all users are added through
ADUC in way that Unix attributes like UID and GID are added also.  I have to
repeat that the effective GID of the user follow the user?s primary *AD* group. 
Merely changing group setting in the tab Unix Attributes will not work!  (This
should be added to the member server how-to!).

2.  GID range suggestion:  The default group of AD user is Domain User whose GID
should be setup through ADUC.  I recommend the GID should be more than 1000 in
order not to clash with the system group in unix side.

3.  Printing bug report:  In order to access files in the member server, it is a
must for me to assign UID to administrator and its group Domain Admin with
another GID.  However, I discover, when adding print driver following the Samba
4 Printing how-to, there is always an error of 0x0000001f error.  After digging
in the log level 10, the print driver upload involves access to a LDB file
situated in /usr/local/samba/private/sam.ldb.d.  The user should be
Administrator (as I login as administration in windows client).  Through mapping
uid and gid through rfc2307, the effective uid is 6000 and its gid is 3085. 
This in turn create problem in access the directory and cannot edit the LDB
file.  This cause failure in adding print driver.  Is it a bug?

In fact, there is a bug report about it:
https://bugzilla.samba.org/show_bug.cgi?id=10089

Now, there is no other bug but do a dirty fix:
chmod 755 /usr/local/samba/private/sam.ldb.d

The relevant log:
[2013/11/19 12:00:05.530215,  2, pid=13968, effective(6000, 3085), real(6000,
0), class=ldb] ../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
  ldb: ltdb:
tdb(/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb):
tdb_open_ex: could not open file
/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb:
Permission denied
[2013/11/19 12:00:05.530236, 10, pid=13968, effective(6000, 3085), real(6000,
0), class=ldb] ../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
  ldb: ldb_asprintf/set_errstring: Unable to open tdb
'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
[2013/11/19 12:00:05.530248,  1, pid=13968, effective(6000, 3085), real(6000,
0), class=ldb] ../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
  ldb: Unable to open tdb
'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
[2013/11/19 12:00:05.530260,  1, pid=13968, effective(6000, 3085), real(6000,
0), class=ldb] ../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
  ldb: Failed to connect to
'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
with backend 'tdb': Unable to open tdb
'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
[2013/11/19 12:00:05.530281,  0, pid=13968, effective(6000, 3085), real(6000,
0), class=ldb] ../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
  ldb: module partition initialization failed : Operations error


4.  Asynchronous I/O - should update how-to:
Reading ?The Evolution of I/O in samba? by Mr. Jeremy Allison has been
enjoyable.  As an system administrator, I am tempted to enable aio in my samba
system.  When trying to do this, I found out less information can be found how
to enable aio in samba 4.  Initially, I would love to enable vfs_aio_linux. 
However, I cannot turn on the module and found out that the relevant .so is not
built even I have tried "apt-get install libaio-dev" in my debian box.
I have no way but turn to enable vfs_aio_pthread instead by the following
smb.conf in the member server:

[global]
   vfs objects = acl_xattr, aio_pthread
   aio read size = 1024
   aio write size = 1024

The reading performance increases 30% in my test.  I think it is worthwhile to
amend it to the official how-to!  And please tell how to build vfs_aio_linux in
samba 4 in debian.

Hope it helps.

Kinglok, Fong


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL:
<http://lists.samba.org/pipermail/samba/attachments/20131119/92ce83eb/attachment.pgp>

On Tue, 2013-11-19 at 14:53 +0800, Kinglok, Fong wrote:
> Dear all,
> 
> After 4 days of sleepless nights, I have manged to rebuild the samba farm. 
I believe the following discovery might interest our samba community.
> Now, there is no other bug but do a dirty fix:
> chmod 755 /usr/local/samba/private/sam.ldb.d
NEVER. EVER do this.

Quick, dirty or otherwise, NEVER do this.  You have totally compromised
the security of the whole domain, because all the private (secret) keys
are not accessible to any user or process on that host. 

Indeed, as this has now been suggested publicly, I may have to add code
to Samba to refuse to start up in this situation. 

I realise you are in a bind, but all I can suggest is that you follow
the Samba Team's recommendation to use a member server for file and
print server tasks, not to combine these with the DC, until we can get
to the bottom of this particular issue. 

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org

Try using the sernet samba packages. 
im using debian and ubuntu and im using this version.

smbd -V   :  Version 4.1.1-SerNet-Ubuntu-7.precise 

these do contain the aio 
/usr/lib/x86_64-linux-gnu/samba/vfs/aio_fork.so
/usr/lib/x86_64-linux-gnu/samba/vfs/aio_posix.so
/usr/lib/x86_64-linux-gnu/samba/vfs/aio_pthread.so


Louis


>-----Oorspronkelijk bericht-----
>Van: busywater at gmail.com 
>[mailto:samba-bounces at lists.samba.org] Namens Kinglok, Fong
>Verzonden: dinsdag 19 november 2013 7:53
>Aan: samba at lists.samba.org
>Onderwerp: [Samba] Samba 4 DC and member server, rfc3207, 
>winbind, printing, asynchronous I/O - Problems and Fixes
>
>Dear all,
>
>After 4 days of sleepless nights, I have manged to rebuild the 
>samba farm.  I believe the following discovery might interest 
>our samba community.
>
>------------------------------------------------
>System setting:
>I have deployed samba 4.1.0 system for my working 
>organisation.  It comprised of 2 DCs and 1 member server.
>2 DCs maintains AD for login and the member server host files 
>for user access.
>
>The installation of DCs and member server follows the samba 
>corresponding official how-tos.  For flawless file access, the 
>domain provision was done with RFC2307 in DCs. 
>------------------------------------------------
>Note:
>1.  Effective GID of AD users:  It is a must that all users 
>are added through ADUC in way that Unix attributes like UID 
>and GID are added also.  I have to repeat that the effective 
>GID of the user follow the user?s primary *AD* group.  Merely 
>changing group setting in the tab Unix Attributes will not 
>work!  (This should be added to the member server how-to!).
>
>2.  GID range suggestion:  The default group of AD user is 
>Domain User whose GID should be setup through ADUC.  I 
>recommend the GID should be more than 1000 in order not to 
>clash with the system group in unix side.
>
>3.  Printing bug report:  In order to access files in the 
>member server, it is a must for me to assign UID to 
>administrator and its group Domain Admin with another GID.  
>However, I discover, when adding print driver following the 
>Samba 4 Printing how-to, there is always an error of 
>0x0000001f error.  After digging in the log level 10, the 
>print driver upload involves access to a LDB file situated in 
>/usr/local/samba/private/sam.ldb.d.  The user should be 
>Administrator (as I login as administration in windows 
>client).  Through mapping uid and gid through rfc2307, the 
>effective uid is 6000 and its gid is 3085.  This in turn 
>create problem in access the directory and cannot edit the LDB 
>file.  This cause failure in adding print driver.  Is it a bug?
>
>In fact, there is a bug report about it:
>https://bugzilla.samba.org/show_bug.cgi?id=10089
>
>Now, there is no other bug but do a dirty fix:
>chmod 755 /usr/local/samba/private/sam.ldb.d
>
>The relevant log:
>[2013/11/19 12:00:05.530215,  2, pid=13968, effective(6000, 
>3085), real(6000, 0), class=ldb] 
>../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>  ldb: ltdb: 
>tdb(/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATI
>ON,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb): tdb_open_ex: could not 
>open file 
>/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,D
>C=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb: Permission denied
>[2013/11/19 12:00:05.530236, 10, pid=13968, effective(6000, 
>3085), real(6000, 0), class=ldb] 
>../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>  ldb: ldb_asprintf/set_errstring: Unable to open tdb 
>'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,
>DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
>[2013/11/19 12:00:05.530248,  1, pid=13968, effective(6000, 
>3085), real(6000, 0), class=ldb] 
>../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>  ldb: Unable to open tdb 
>'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,
>DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
>[2013/11/19 12:00:05.530260,  1, pid=13968, effective(6000, 
>3085), real(6000, 0), class=ldb] 
>../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>  ldb: Failed to connect to 
>'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,
>DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb' with backend 'tdb': Unable 
>to open tdb 
>'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,
>DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
>[2013/11/19 12:00:05.530281,  0, pid=13968, effective(6000, 
>3085), real(6000, 0), class=ldb] 
>../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>  ldb: module partition initialization failed : Operations error
>
>
>4.  Asynchronous I/O - should update how-to:
>Reading ?The Evolution of I/O in samba? by Mr. Jeremy Allison 
>has been enjoyable.  As an system administrator, I am tempted 
>to enable aio in my samba system.  When trying to do this, I 
>found out less information can be found how to enable aio in 
>samba 4.  Initially, I would love to enable vfs_aio_linux.  
>However, I cannot turn on the module and found out that the 
>relevant .so is not built even I have tried "apt-get install 
>libaio-dev" in my debian box.  I have no way but turn to 
>enable vfs_aio_pthread instead by the following smb.conf in 
>the member server:
>
>[global]
>   vfs objects = acl_xattr, aio_pthread
>   aio read size = 1024
>   aio write size = 1024
>
>The reading performance increases 30% in my test.  I think it 
>is worthwhile to amend it to the official how-to!  And please 
>tell how to build vfs_aio_linux in samba 4 in debian.
>
>Hope it helps.
>
>Kinglok, Fong
>
>
>-- 
>To unsubscribe from this list go to the following URL and read the
>instructions:  https://lists.samba.org/mailman/options/samba
>

On 19 Nov, 2013, at 2:53 pm, Kinglok, Fong <busywater at gmail.com> wrote:
> Dear all,
> 
> After 4 days of sleepless nights, I have manged to rebuild the samba farm. 
I believe the following discovery might interest our samba community.
> 
> ------------------------------------------------
> System setting:
> I have deployed samba 4.1.0 system for my working organisation.  It
comprised of 2 DCs and 1 member server.
> 2 DCs maintains AD for login and the member server host files for user
access.
> 
> The installation of DCs and member server follows the samba corresponding
official how-tos.  For flawless file access, the domain provision was done with
RFC2307 in DCs.
> ------------------------------------------------
> Note:
> 1.  Effective GID of AD users:  It is a must that all users are added
through ADUC in way that Unix attributes like UID and GID are added also.  I
have to repeat that the effective GID of the user follow the user?s primary *AD*
group.  Merely changing group setting in the tab Unix Attributes will not work! 
(This should be added to the member server how-to!).
> 
> 2.  GID range suggestion:  The default group of AD user is Domain User
whose GID should be setup through ADUC.  I recommend the GID should be more than
1000 in order not to clash with the system group in unix side.
> 
> 3.  Printing bug report:  In order to access files in the member server, it
is a must for me to assign UID to administrator and its group Domain Admin with
another GID.  However, I discover, when adding print driver following the Samba
4 Printing how-to, there is always an error of 0x0000001f error.  After digging
in the log level 10, the print driver upload involves access to a LDB file
situated in /usr/local/samba/private/sam.ldb.d.  The user should be
Administrator (as I login as administration in windows client).  Through mapping
uid and gid through rfc2307, the effective uid is 6000 and its gid is 3085. 
This in turn create problem in access the directory and cannot edit the LDB
file.  This cause failure in adding print driver.  Is it a bug?
> 
> In fact, there is a bug report about it:
> https://bugzilla.samba.org/show_bug.cgi?id=10089
> 
> Now, there is no other bug but do a dirty fix:
> chmod 755 /usr/local/samba/private/sam.ldb.d
> 
> The relevant log:
> [2013/11/19 12:00:05.530215,  2, pid=13968, effective(6000, 3085),
real(6000, 0), class=ldb] ../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>  ldb: ltdb:
tdb(/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb):
tdb_open_ex: could not open file
/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb:
Permission denied
> [2013/11/19 12:00:05.530236, 10, pid=13968, effective(6000, 3085),
real(6000, 0), class=ldb] ../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>  ldb: ldb_asprintf/set_errstring: Unable to open tdb
'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
> [2013/11/19 12:00:05.530248,  1, pid=13968, effective(6000, 3085),
real(6000, 0), class=ldb] ../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>  ldb: Unable to open tdb
'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
> [2013/11/19 12:00:05.530260,  1, pid=13968, effective(6000, 3085),
real(6000, 0), class=ldb] ../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>  ldb: Failed to connect to
'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
with backend 'tdb': Unable to open tdb
'/usr/local/samba/private/sam.ldb.d/CN=SCHEMA,CN=CONFIGURATION,DC=SAMBA,DC=FOO,DC=EDU,DC=HK.ldb'
> [2013/11/19 12:00:05.530281,  0, pid=13968, effective(6000, 3085),
real(6000, 0), class=ldb] ../lib/ldb-samba/ldb_wrap.c:71(ldb_wrap_debug)
>  ldb: module partition initialization failed : Operations error
> 
> 
> 4.  Asynchronous I/O - should update how-to:
> Reading ?The Evolution of I/O in samba? by Mr. Jeremy Allison has been
enjoyable.  As an system administrator, I am tempted to enable aio in my samba
system.  When trying to do this, I found out less information can be found how
to enable aio in samba 4.  Initially, I would love to enable vfs_aio_linux. 
However, I cannot turn on the module and found out that the relevant .so is not
built even I have tried "apt-get install libaio-dev" in my debian box.
I have no way but turn to enable vfs_aio_pthread instead by the following
smb.conf in the member server:
> 
> [global]
>   vfs objects = acl_xattr, aio_pthread
>   aio read size = 1024
>   aio write size = 1024
> 
It turns out that after installing libaio-dev, I can have aio_linux.so in
/usr/local/samba/lib/vas

I still think that it is worthwhile to have a section call Asynchronous I/O in
samba 4 wiki!

Please tell how I can contribute.

Kinglok, Fong
> The reading performance increases 30% in my test.  I think it is worthwhile
to amend it to the official how-to!  And please tell how to build vfs_aio_linux
in samba 4 in debian.
> 
> Hope it helps.
> 
> Kinglok, Fong
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL:
<http://lists.samba.org/pipermail/samba/attachments/20131119/798c7067/attachment.pgp>

On Tue, Nov 19, 2013 at 02:53:11PM +0800, Kinglok, Fong
wrote:
> 
> 4.  Asynchronous I/O - should update how-to:
> Reading ?The Evolution of I/O in samba? by Mr. Jeremy Allison has been
enjoyable.  As an system administrator, I am tempted to enable aio in my samba
system.  When trying to do this, I found out less information can be found how
to enable aio in samba 4.  Initially, I would love to enable vfs_aio_linux. 
However, I cannot turn on the module and found out that the relevant .so is not
built even I have tried "apt-get install libaio-dev" in my debian box.
I have no way but turn to enable vfs_aio_pthread instead by the following
smb.conf in the member server:
> 
> [global]
>    vfs objects = acl_xattr, aio_pthread
>    aio read size = 1024
>    aio write size = 1024
> 
> The reading performance increases 30% in my test.  I think it is worthwhile
to amend it to the official how-to!  And please tell how to build vfs_aio_linux
in samba 4 in debian.
Thanks for that info ! Turning on vfs_aio_linux is not
suggested, as it doesn't really help performance (for various
reasons). Consider it a failed experiment.

aio_pthread should be considered the (working) replacement.

Cheers,

	Jeremy.