samba - Mar 2016 - The sad state of samba 4 adaption for home/small business routers.

Hi,

i would like to inspire a discussion on why samba4 has nearly no adaption on
home/small business routers firmware.

I recently switched from my old Asus-N16 running tomato to a WRT-1200AC and
was researching potential firmware's that i could run. We have a new
synology disk-station at work that runs Samba 4.x and it works great. So i
was surprised to notice that 3 years after the 4.0 release i could not find
a single home/small business oriented router firmware that was supporting
samba 4.x? I delved deeper into this and after several weeks toying around
with the tomato/dd-wrt/openWRT/buildroot/openembedded/optware2/entware
toolchains and the samba 4.3 source + waf, i understand why.


Some problems i encountered, while trying to build a samba 4 package for any
of the most popular router firmwares.

1) Waf's cross-compile approach/requirements are simply not feasible,
clearly understandable for most package maintainers.

The problem is that most maintainers can not test/create the required
qemu/crossanswer files for all the targets, that Waf relies on. As far as i
know samba4 is the only package in openWRT/buildroot that requires such a
special cross compile handling. So while i have a arm32 version running on
my AC1200, thats based on a fork of openWRT. We can not create/submit a
general package, since we lack the confidence to create crossanswer files
for all the different target platforms and creating a valid qemu host
package per target is also problematic, since no package did ever require
such a workflow.


2) The size of the final waf bin/lib install, even if stripped down to the
bare minimum of smbd/nmbd/smbpasswd is unsuitable for most older routers
that have just 8MB nvram and problematic for 16/32 MB more recent routers,
depending on what other packages are added to the firmware.

The smallest samba 4.3 package we could manage to build is around 9-12MB or
4.9MB lzma compressed using this makefile + some multicall/static linking
hacks/patches:
https://github.com/wongsyrone/openwrt-1/tree/master/package/external
/samba43related/samba43

In contrast Samba 3.6 can be stripped down to around 900kb (lzma), which is
not a problem for 8MB nvram targets.

I tried to base my build on the buildroot example and create a normal shared
lib package, since i don't like to maintain all the multicall patches and
the size gain over the shared version is just about 20%, which don't really
brings us this much closer.

So is there a way to further strip down samba4 to create a home environment
based "minimal fileserver" package? What most home users want is just
a
working smb3 fileserver (s3fs) to reach windows8/10 clients and netbios so
they can browse, find the file-server. Most will not even go through the
hassle and setup user profiles/restrictions and use guest mode.
I personally also found it very intriguing to have the btrfs vfs module and
the associated transparent compression option.


Here is the current status of the big firmwares regarding samba 4:
Tomato: don't even try to get this working, since they are based on a old
kernel and lack the toolchain to even build samba 4
dd-wrt: follows openWRT
openWRT: no official package exists and the versions that where created in
forks can not be submitted, since only arm target crossanswers are used in
the makefile.
Buildroot: made some progress in the recent weeks to add more cross answer
files, but its basically a full install that is too big.
openembedded: seems to haven given up after the 4.0 experimental support
optware2/entware-ng2: no real efforts are made


So i welcome a discussion on how to help fixing this situation, so also
home/sbu can enjoy a up-to-date samba 4 version.

thx
Andy

Am 11.03.2016 um 18:04 schrieb Andy Walsh:
> i would like to inspire a discussion on why samba4 has nearly no adaption
on
> home/small business routers firmware.
>
> I recently switched from my old Asus-N16 running tomato to a WRT-1200AC and
> was researching potential firmware's that i could run. We have a new
> synology disk-station at work that runs Samba 4.x and it works great. So i
> was surprised to notice that 3 years after the 4.0 release i could not find
> a single home/small business oriented router firmware that was supporting
> samba 4.x? I delved deeper into this and after several weeks toying around
> with the tomato/dd-wrt/openWRT/buildroot/openembedded/optware2/entware
> toolchains and the samba 4.3 source + waf, i understand why
most likely because there is no serious market

if i want a NAS i buy a NAS
if i want a router i buy a router

both are bad things if it comes to cheap consumer crap and combine two 
bad things makes it really a terrible frankenstein box

in my case i buy a box, install linux with some iptables-rules and 
whatever other services i need and that includes even wlan-accesspoints 
for the last 6 years

no struggle with updates and security holes
no struggle with software versions
no struggle with "i can have this and this but not combined with that"
no struggle with "cool features but terrible slow" or otherside round
no bloatware

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/samba/attachments/20160311/e538fbe9/signature.sig>

On Fri, Mar 11, 2016 at 05:04:50PM +0000, Andy Walsh
wrote:
> 
> 1) Waf's cross-compile approach/requirements are simply not feasible,
> clearly understandable for most package maintainers.
> 
> The problem is that most maintainers can not test/create the required
> qemu/crossanswer files for all the targets, that Waf relies on. As far as i
> know samba4 is the only package in openWRT/buildroot that requires such a
> special cross compile handling. So while i have a arm32 version running on
> my AC1200, thats based on a fork of openWRT. We can not create/submit a
> general package, since we lack the confidence to create crossanswer files
> for all the different target platforms and creating a valid qemu host
> package per target is also problematic, since no package did ever require
> such a workflow.
I've helped some OEMs create waf x-compilation files for S4.

It's a little tricky, but not so bad and doesn't require qemu hosts.
Hopefully
more experience will make this easier with time.
> 2) The size of the final waf bin/lib install, even if stripped down to the
> bare minimum of smbd/nmbd/smbpasswd is unsuitable for most older routers
> that have just 8MB nvram and problematic for 16/32 MB more recent routers,
> depending on what other packages are added to the firmware.
> 
> The smallest samba 4.3 package we could manage to build is around 9-12MB or
> 4.9MB lzma compressed using this makefile + some multicall/static linking
> hacks/patches:
> https://github.com/wongsyrone/openwrt-1/tree/master/package/external
> /samba43related/samba43
> 
> In contrast Samba 3.6 can be stripped down to around 900kb (lzma), which is
> not a problem for 8MB nvram targets.
Trouble is, S4 does more than S3 did - mainly the SMB2 engine,
which will soon be required for real-world useage.

I'm not unsympathetic, just that cutting this down for
router usage is a lower priority than adding the features
that enterprise, NAS and cloud-gateway vendors need.

I'm happy to review any changes you might suggest.

Reindl Harald <h.reindl <at> thelounge.net> writes:
> most likely because there is no serious market
> 
> if i want a NAS i buy a NAS
> if i want a router i buy a router
> 
> 
> no struggle with updates and security holes
> no struggle with software versions
> no struggle with "i can have this and this but not combined with
that"
> no struggle with "cool features but terrible slow" or otherside
round
> no bloatware
> 
Thats a strange argument, since up until recently home/sbu routers did not
come with 256/512MB ram, had USB3/sata3 ports and dual core arm cpu's.

All the above points are basically what i have now on my WRT-1200AC running
openWRT.
I use the latest 4.4.4 kernel + btrfs and can max out the GB ports using
smb3.1, while the system is stable and i can pick exactly what runs on the
system.

All this runs at 3-5 watts energy in a small form factor, at a very
affordable price.


bye
Andy


PS: I'm also curious what consumer router/nas combo would you buy to meet
your own requirements? There are security problems on almost all routers,
thats the reason why so many switch to openWRT/tomato/dd-wrt based
firmwares. That was also the main reason why the Asus N-16 and WRT54G became
famous.

On Fri, 2016-03-11 at 17:04 +0000, Andy Walsh wrote:
> 2) The size of the final waf bin/lib install, even if stripped down
> to the
> bare minimum of smbd/nmbd/smbpasswd is unsuitable for most older
> routers
> that have just 8MB nvram and problematic for 16/32 MB more recent
> routers,
> depending on what other packages are added to the firmware.
> 
> The smallest samba 4.3 package we could manage to build is around 9
> -12MB or
> 4.9MB lzma compressed using this makefile + some multicall/static
> linking
> hacks/patches:
> https://github.com/wongsyrone/openwrt-1/tree/master/package/external
> /samba43related/samba43
> 
> In contrast Samba 3.6 can be stripped down to around 900kb (lzma),
> which is
> not a problem for 8MB nvram targets.
The big difference between 3.6 and the 4.x series is that far more IDL
is generated and compiled.  Much of that probably could be skipped,
with some care and waf work.  It is harder because we have started to
use DCE/RPC services internally that we might not need for clients, but
I would give that a shot.

Patches to make Samba more modular (and to then allow those modules to
be disabled) will be considered.  Ideally we contain that to waf rules,
rather than #ifdef in the code.

Having a configure option to build just a standalone (auth against
passdb) file server exposing a similar set to what Apple exposes
(because that will give us a reasonable way to manage expectations)
would be one approach. 

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba