Hello,
Please CC replies I'm not subscribed.
Performance with samba and only samba degrades terribly when we use
the bonding driver to aggregate two ethernet cards. Instead of a steady
file copy it seems to go in spurts. If I pull out one of the network
cables (doesn't matter which) performance resumes to full speed. I can
pull the cable in the middle of a transfer and it will go to half speed,
or I can restart the transfer. It is *ALWAYS* slower when both cards are
active. An 80MB file might take approx 50 seconds with both active and
15 with only one active. The
switch is a DGS-1224T which supports trunking/channel bonding and other
servers haven't
show problems. Again, NFS, scp, ssh, netperf all remain unaffected
regardless of 1 or 2 cards active. Samba IS bound to the bond0 interface
and only that interface. I haven't been able to find much on
google/forums about this since it seems to be a very specific
interaction between bonding and samba.
Configuration details below. Very minor editing to remove identifying IP
info. Debian Samba 3.0.23d-4 using Debian testing. Hardware is a stock
hp proliant ml110 with a d-link gigabit ethernet card added (for the bond).
Any ideas?
===========================================sage:~# ifconfig
bond0 Link encap:Ethernet HWaddr 00:18:71:77:94:3D
inet addr:xxx.xxx.17.9 Bcast:xxx.xxx.17.255 Mask:255.255.255.0
inet6 addr: fe80::218:71ff:fe77:943d/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:140981609 errors:0 dropped:0 overruns:0 frame:0
TX packets:128574116 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2043177198 (1.9 GiB) TX bytes:3171479660 (2.9 GiB)
eth0 Link encap:Ethernet HWaddr 00:18:71:77:94:3D
inet6 addr: fe80::218:71ff:fe77:943d/64 Scope:Link
UP BROADCAST SLAVE MULTICAST MTU:1500 Metric:1
RX packets:97516848 errors:0 dropped:0 overruns:0 frame:0
TX packets:61109354 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4018776902 (3.7 GiB) TX bytes:1263090953 (1.1 GiB)
Interrupt:169
eth1 Link encap:Ethernet HWaddr 00:18:71:77:94:3D
inet6 addr: fe80::218:71ff:fe77:943d/64 Scope:Link
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:43464760 errors:0 dropped:0 overruns:0 frame:0
TX packets:67464762 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2319367442 (2.1 GiB) TX bytes:1908388707 (1.7 GiB)
Interrupt:177
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1705911 errors:0 dropped:0 overruns:0 frame:0
TX packets:1705911 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4002868337 (3.7 GiB) TX bytes:4002868337 (3.7 GiB)
===============================================Shares excluded because it doesnt
seem to be share dependant
sage:/etc/samba# cat smb.conf
[global]
# identity
server string = File Server
netbios name = SAGE
workgroup = MYDOMAIN
# Networking
interfaces = 127.0.0.1/8 bond0
bind interfaces only = yes
hosts allow = xxx.xxx.17.0/24 10.0.18.0/24
socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
name resolve order = wins bcast hosts
# Domain Setup
security = DOMAIN
password server = JUNIOR
domain master = no
preferred master = yes
os level = 45
# WINS
wins support = yes
# Time support
time server = Yes
ea support = yes
map acl inherit = yes
#
logon drive = H
#Logging. Default Level 1
#log file = /var/log/samba/log.%m
log level = all:3 smb:3 auth:4 sam:3
# vfs:10 #smb:6
#2G
max log size = 100000
#Currently known debug classes:
# all
# tdb
# printdrivers
# lanman
# smb
# rpc_parse
# rpc_srv
# rpc_cli
# passdb
# sam
# auth
# winbind
# vfs
# idmap
# quota
# acls
# Ignore AppleTalk crap
veto files = /.AppleDB/.AppleDouble/.AppleDesktop/Network Trash
Folder/TheVolumeSettingsFolder/TheFindByContentFolder/Temporary Items/
# Printers
load printers = yes
# use client driver = yes
printing = cups
printcap name = cups
===============================================sage:/etc/samba# smbd -V
Version 3.0.23d
sage:/etc/samba# nmbd -V
Version 3.0.23d
sage:/etc/samba#
===============================================sage:/etc/samba# uname -a
Linux sage 2.6.18-3-686 #1 SMP Mon Dec 4 16:41:14 UTC 2006 i686 GNU/Linux
===============================================sage:/etc/samba# cat
/proc/interrupts
CPU0 CPU1
0: 2568 278337684 IO-APIC-edge timer
1: 0 0 IO-APIC-edge i8042
8: 1 0 IO-APIC-edge rtc
9: 1 0 IO-APIC-level acpi
14: 64 0 IO-APIC-edge ide0
50: 32132768 33720795 PCI-MSI libata
58: 83077157 33865392 PCI-MSI eth0
177: 19833549 70085995 IO-APIC-level uhci_hcd:usb4, skge
217: 2 0 IO-APIC-level uhci_hcd:usb1, ehci_hcd:usb5
225: 0 0 IO-APIC-level uhci_hcd:usb2
233: 0 0 IO-APIC-level uhci_hcd:usb3
NMI: 0 0
LOC: 278339349 278338330
ERR: 0
MIS: 0
===============================================sage:/etc/samba# lsmod
Module Size Used by
ppdev 8676 0
parport_pc 32164 0
lp 11012 0
parport 33256 3 ppdev,parport_pc,lp
quota_v2 8864 4
reiserfs 212640 1
raid456 115472 1
xor 14216 1 raid456
md_mod 70388 2 raid456
nfs 202828 0
nfsd 197840 17
exportfs 5600 1 nfsd
lockd 54344 3 nfs,nfsd
nfs_acl 3584 2 nfs,nfsd
sunrpc 138780 13 nfs,nfsd,lockd,nfs_acl
button 6672 0
ac 5188 0
battery 9636 0
ipv6 226016 28
dm_snapshot 15520 0
dm_mirror 19152 0
dm_mod 50232 2 dm_snapshot,dm_mirror
bonding 71528 0
loop 15048 0
serio_raw 6660 0
tsdev 7520 0
psmouse 35016 0
evdev 9088 0
rtc 12372 0
pcspkr 3072 0
ext3 119208 1
jbd 52456 1 ext3
mbcache 8356 1 ext3
sd_mod 19040 8
ide_cd 36064 0
cdrom 32544 1 ide_cd
generic 5028 0 [permanent]
usbhid 37248 0
ahci 17636 4
libata 89332 1 ahci
scsi_mod 124168 3 sd_mod,ahci,libata
ehci_hcd 28136 0
tg3 94948 0
skge 34544 0
piix 9444 0 [permanent]
ide_core 110504 3 ide_cd,generic,piix
uhci_hcd 21032 0
usbcore 112676 4 usbhid,ehci_hcd,uhci_hcd
thermal 13608 0
processor 28840 1 thermal
fan 4804 0
Have you tried changing 'socket options' in your smb.conf file? You may need to tune that for your setup, it does make a difference at least in a system with 1 NIC in it. However, I have no experience with bonding and Samba. Eric Bambach wrote:> Hello, > > Please CC replies I'm not subscribed. > > Performance with samba and only samba degrades terribly when we use > the bonding driver to aggregate two ethernet cards. Instead of a steady > file copy it seems to go in spurts. If I pull out one of the network > cables (doesn't matter which) performance resumes to full speed. I can > pull the cable in the middle of a transfer and it will go to half > speed, or I can restart the transfer. It is *ALWAYS* slower when both > cards are active. An 80MB file might take approx 50 seconds with both > active and 15 with only one active. The > switch is a DGS-1224T which supports trunking/channel bonding and > other servers haven't > show problems. Again, NFS, scp, ssh, netperf all remain unaffected > regardless of 1 or 2 cards active. Samba IS bound to the bond0 interface > and only that interface. I haven't been able to find much on > google/forums about this since it seems to be a very specific > interaction between bonding and samba. > > Configuration details below. Very minor editing to remove identifying IP > info. Debian Samba 3.0.23d-4 using Debian testing. Hardware is a stock > hp proliant ml110 with a d-link gigabit ethernet card added (for the > bond). > > Any ideas? > > ===========================================> sage:~# ifconfig > bond0 Link encap:Ethernet HWaddr 00:18:71:77:94:3D > inet addr:xxx.xxx.17.9 Bcast:xxx.xxx.17.255 Mask:255.255.255.0 > inet6 addr: fe80::218:71ff:fe77:943d/64 Scope:Link > UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 > RX packets:140981609 errors:0 dropped:0 overruns:0 frame:0 > TX packets:128574116 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:2043177198 (1.9 GiB) TX bytes:3171479660 (2.9 GiB) > > eth0 Link encap:Ethernet HWaddr 00:18:71:77:94:3D > inet6 addr: fe80::218:71ff:fe77:943d/64 Scope:Link > UP BROADCAST SLAVE MULTICAST MTU:1500 Metric:1 > RX packets:97516848 errors:0 dropped:0 overruns:0 frame:0 > TX packets:61109354 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:4018776902 (3.7 GiB) TX bytes:1263090953 (1.1 GiB) > Interrupt:169 > > eth1 Link encap:Ethernet HWaddr 00:18:71:77:94:3D > inet6 addr: fe80::218:71ff:fe77:943d/64 Scope:Link > UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 > RX packets:43464760 errors:0 dropped:0 overruns:0 frame:0 > TX packets:67464762 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:2319367442 (2.1 GiB) TX bytes:1908388707 (1.7 GiB) > Interrupt:177 > > lo Link encap:Local Loopback > inet addr:127.0.0.1 Mask:255.0.0.0 > inet6 addr: ::1/128 Scope:Host > UP LOOPBACK RUNNING MTU:16436 Metric:1 > RX packets:1705911 errors:0 dropped:0 overruns:0 frame:0 > TX packets:1705911 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:4002868337 (3.7 GiB) TX bytes:4002868337 (3.7 GiB) > ===============================================> Shares excluded because it doesnt seem to be share dependant > > sage:/etc/samba# cat smb.conf > [global] > # identity > server string = File Server > netbios name = SAGE > workgroup = MYDOMAIN > > # Networking > interfaces = 127.0.0.1/8 bond0 > bind interfaces only = yes > hosts allow = xxx.xxx.17.0/24 10.0.18.0/24 > socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192 > name resolve order = wins bcast hosts > > # Domain Setup > security = DOMAIN > password server = JUNIOR > domain master = no > preferred master = yes > os level = 45 > > # WINS > wins support = yes > > # Time support > time server = Yes > > ea support = yes > map acl inherit = yes > > # > logon drive = H > > #Logging. Default Level 1 > #log file = /var/log/samba/log.%m > log level = all:3 smb:3 auth:4 sam:3 > # vfs:10 #smb:6 > #2G > max log size = 100000 > #Currently known debug classes: > # all > # tdb > # printdrivers > # lanman > # smb > # rpc_parse > # rpc_srv > # rpc_cli > # passdb > # sam > # auth > # winbind > # vfs > # idmap > # quota > # acls > > # Ignore AppleTalk crap > veto files = /.AppleDB/.AppleDouble/.AppleDesktop/Network Trash > Folder/TheVolumeSettingsFolder/TheFindByContentFolder/Temporary Items/ > > # Printers > load printers = yes > # use client driver = yes > printing = cups > printcap name = cups > > ===============================================> sage:/etc/samba# smbd -V > Version 3.0.23d > sage:/etc/samba# nmbd -V > Version 3.0.23d > sage:/etc/samba# > ===============================================> sage:/etc/samba# uname -a > Linux sage 2.6.18-3-686 #1 SMP Mon Dec 4 16:41:14 UTC 2006 i686 GNU/Linux > ===============================================> sage:/etc/samba# cat /proc/interrupts > CPU0 CPU1 > 0: 2568 278337684 IO-APIC-edge timer > 1: 0 0 IO-APIC-edge i8042 > 8: 1 0 IO-APIC-edge rtc > 9: 1 0 IO-APIC-level acpi > 14: 64 0 IO-APIC-edge ide0 > 50: 32132768 33720795 PCI-MSI libata > 58: 83077157 33865392 PCI-MSI eth0 > 177: 19833549 70085995 IO-APIC-level uhci_hcd:usb4, skge > 217: 2 0 IO-APIC-level uhci_hcd:usb1, ehci_hcd:usb5 > 225: 0 0 IO-APIC-level uhci_hcd:usb2 > 233: 0 0 IO-APIC-level uhci_hcd:usb3 > NMI: 0 0 > LOC: 278339349 278338330 > ERR: 0 > MIS: 0 > ===============================================> sage:/etc/samba# lsmod > Module Size Used by > ppdev 8676 0 > parport_pc 32164 0 > lp 11012 0 > parport 33256 3 ppdev,parport_pc,lp > quota_v2 8864 4 > reiserfs 212640 1 > raid456 115472 1 > xor 14216 1 raid456 > md_mod 70388 2 raid456 > nfs 202828 0 > nfsd 197840 17 > exportfs 5600 1 nfsd > lockd 54344 3 nfs,nfsd > nfs_acl 3584 2 nfs,nfsd > sunrpc 138780 13 nfs,nfsd,lockd,nfs_acl > button 6672 0 > ac 5188 0 > battery 9636 0 > ipv6 226016 28 > dm_snapshot 15520 0 > dm_mirror 19152 0 > dm_mod 50232 2 dm_snapshot,dm_mirror > bonding 71528 0 > loop 15048 0 > serio_raw 6660 0 > tsdev 7520 0 > psmouse 35016 0 > evdev 9088 0 > rtc 12372 0 > pcspkr 3072 0 > ext3 119208 1 > jbd 52456 1 ext3 > mbcache 8356 1 ext3 > sd_mod 19040 8 > ide_cd 36064 0 > cdrom 32544 1 ide_cd > generic 5028 0 [permanent] > usbhid 37248 0 > ahci 17636 4 > libata 89332 1 ahci > scsi_mod 124168 3 sd_mod,ahci,libata > ehci_hcd 28136 0 > tg3 94948 0 > skge 34544 0 > piix 9444 0 [permanent] > ide_core 110504 3 ide_cd,generic,piix > uhci_hcd 21032 0 > usbcore 112676 4 usbhid,ehci_hcd,uhci_hcd > thermal 13608 0 > processor 28840 1 thermal > fan 4804 0 >