Martin Clemons
2026-Feb-18 09:29 UTC
[Samba] Problems encountered when building on SmartOS/Illumos
Hi everyone, I spent some time yesterday trying to get samba 4.23.4 to build on SmartOS (image base-64-lts 25.4.0). In the end I managed to get a working binary but not before working through some bugs blocking the build with the help of Codex. I?ve attached a summary of the issues below (also available as a gist <https://gist.github.com/martin-c/fae6934b129edde02468d1df7427f3dd>). Would it be worth it to submit each of these issues to the bug tracker? Is anyone else building for SmartOS/Illumos these days? Thanks! Martin Samba 4.23.4 on SmartOS: Bug Report Drafts Bug 1: auxv_t redefinition in embedded Heimdal on SmartOS <https://gist.github.com/martin-c/fae6934b129edde02468d1df7427f3dd#bug-1-auxv_t-redefinition-in-embedded-heimdal-on-smartos> Component: embedded Heimdal (roken) Affected file: third_party/heimdal/lib/roken/getauxval.h Build failure signal: error: conflicting types for 'auxv_t'; have 'struct rk_auxv' previous declaration from /usr/include/sys/auxv.h Observed impact: build stops in Heimdal getauxval path. Expected behavior: build should detect system auxv_t and avoid redefining it. Local workaround used: guard/restrict local auxv_t typedef on Solaris. Suggested fix direction: add robust configure/header guard for Solaris auxv_t. Bug 2: msghdr.msg_flags portability break in messages_dgm.c <https://gist.github.com/martin-c/fae6934b129edde02468d1df7427f3dd#bug-2-msghdrmsg_flags-portability-break-in-messages_dgmc> Component: messaging Affected file: lib/messaging/messages_dgm.c Build failure signal: error: 'struct msghdr' has no member named 'msg_flags' Observed impact: build fails in messaging_dgm_read_handler. Expected behavior: code should be gated by platform capability detection. Local workaround used: conditionally skip msg.msg_flags |= MSG_CMSG_CLOEXEC; on SmartOS. Suggested fix direction: gate on HAVE_STRUCT_MSGHDR_MSG_FLAGS (or equivalent autoconf result). Bug 3: macro collision on s_addr identifier in tsocket_bsd.c <https://gist.github.com/martin-c/fae6934b129edde02468d1df7427f3dd#bug-3-macro-collision-on-s_addr-identifier-in-tsocket_bsdc> Component: tsocket Affected file: lib/tsocket/tsocket_bsd.c Build failure signal: error: expected ... before '.' token error: 'S_un' undeclared failure at local declaration char *s_addr = NULL; Observed impact: SmartOS macro expansion collides with local variable name. Expected behavior: avoid variable names that collide with system networking macros. Local workaround used: rename local variable from s_addr to addr_part only within the function scope. Suggested fix direction: rename local identifier upstream in _tsocket_address_inet_from_hostport_strings. Bug 4: Solaris quota RPC include/GSS type break <https://gist.github.com/martin-c/fae6934b129edde02468d1df7427f3dd#bug-4-solaris-quota-rpc-includegss-type-break> Component: source3 quota support Affected file: source3/smbd/quotas.c (through RPC includes) Build failure signal: from /usr/include/rpc/rpcsec_gss.h: unknown types gss_cred_id_t, gss_ctx_id_t, etc. Observed impact: quota-related build path fails on SmartOS headers/toolchain. Expected behavior: quota path should compile or be auto-disabled when required RPC/GSS types are unavailable. Local workaround used: configure with --without-quotas. Suggested fix direction: tighten configure detection / conditional compilation for Solaris RPC/GSS variants. Bug 5: vfs_solarisacl.c declaration/signature/typo regressions <https://gist.github.com/martin-c/fae6934b129edde02468d1df7427f3dd#bug-5-vfs_solarisaclc-declarationsignaturetypo-regressions> Component: vfs_solarisacl Affected file: source3/modules/vfs_solarisacl.c (and declaration in vfs_solarisacl.h) Build failure signal: error: static declaration ... follows non-static declaration incompatible pointer argument (char * passed where const struct smb_filename * expected) typo call DBG_DEBG (implicit declaration warning) Observed impact: module fails to compile on SmartOS. Expected behavior: declaration/definition linkage and signatures should match; debug macro should be valid. Local workaround used: remove static mismatch, pass fsp->fsp_name instead of fsp->fsp_name->base_name, DBG_DEBG -> DBG_DEBUG. Suggested fix direction: upstream consistency fix in solarisacl module. Bug 6: ctime_r arity mismatch in libsmbclient example on SmartOS <https://gist.github.com/martin-c/fae6934b129edde02468d1df7427f3dd#bug-6-ctime_r-arity-mismatch-in-libsmbclient-example-on-smartos> Component: examples/libsmbclient Affected file: examples/libsmbclient/teststat.c Build failure signal: error: too few arguments to function 'ctime_r' SmartOS prototype expects 3 args. Observed impact: examples target fails to compile. Expected behavior: use portable wrapper or platform-specific ctime_r call signature. Local workaround used: changed calls to include buffer length (sizeof(...)). Suggested fix direction: portability macro for 2-arg vs 3-arg ctime_r. Bug 7: smbtorture link failure with system Heimdal on Solaris linker <https://gist.github.com/martin-c/fae6934b129edde02468d1df7427f3dd#bug-7-smbtorture-link-failure-with-system-heimdal-on-solaris-linker> Component: source4/torture (smbtorture) Affected target: source4/torture/smbtorture Link failure signal: undefined krb5_* symbols (e.g., krb5_get_creds, krb5_c_decrypt) Solaris ld reports symbols belong to implicit dependency /opt/local/heimdal/lib/libkrb5.so.26 Observed impact: install/build blocked when smbtorture is included in install graph. Expected behavior: target should link direct Kerberos libs when required (not rely on implicit deps). Local workaround used: disable smbtorture target to complete installation. Suggested fix direction: add explicit krb5/hx509/etc link dependencies to smbtorture (or its dependent modules) for Solaris linkers. Bug 8: Embedded Heimdal libhcrypto-private-samba.so unresolved PKCS#11 symbols on SmartOS <https://gist.github.com/martin-c/fae6934b129edde02468d1df7427f3dd#bug-8-embedded-heimdal-libhcrypto-private-sambaso-unresolved-pkcs11-symbols-on-smartos> Component: embedded Heimdal / hcrypto in Samba build graph Affected artifact: bin/default/third_party/heimdal_build/libhcrypto-private-samba.so Link failure signal: undefined symbols hc_EVP_pkcs11_hcrypto_* (e.g. hc_EVP_pkcs11_hcrypto_sha1, ..._aes_256_cbc, ..._md5, etc.) breaks linking of core targets including source3/smbd/smbd and source3/utils/smbpasswd Observed impact: file-server binaries fail to link, so build cannot complete with embedded Heimdal. Expected behavior: embedded Heimdal should either: compile/link all required PKCS#11 provider symbols, or disable those call sites when provider symbols are not available. Local workaround used: built/installed Heimdal separately and switched Samba to system Heimdal via --with-system-heimdalkrb5. Suggested fix direction: fix embedded Heimdal feature detection and symbol export/link ordering for SmartOS/Solaris, and ensure libhcrypto-private-samba.so does not retain unresolved hc_EVP_pkcs11_hcrypto_* references in this configuration. Optional Global Context <https://gist.github.com/martin-c/fae6934b129edde02468d1df7427f3dd#optional-global-context-attach-once-across-reports> Build profile included: --without-ad-dc --without-ads --without-ldap --without-quotas --without-gpgme --with-system-heimdalkrb5 --with-shared-modules=vfs_fruit,vfs_streams_xattr,vfs_catia,vfs_zfsacl Platform: SmartOS / Solaris userland.