I believe I have some hardware or configuration related performance issues running samba 3.0.14a-3sarge. Our server is an Intel Celeron 2 Ghz with 512 MB of RAM and a 3ware card using SATA disks in a RAID 5 configuration (3ware controller card). We have a gigabit network and are using Intel Gigabit ethernet cards e1000). When copying large files to the samba shares on the system, the transfer rate maxes out near 100 mb/s. We tested with nttcp and were able to get speeds of nearly 800mb/s. So I think it is safe to conclude this is not a network issue. Various tools like top, xosview and mpstat convinced us that we are bound in the CPU. Stopping the samba file transfer and the cpu idle time exceeds 90%. We are convinced that our CPU is the bottleneck, but not sure why. #cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 2 model name : Intel(R) Celeron(R) CPU 2.00GHz stepping : 9 cpu MHz : 1996.920 cache size : 128 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid bogomips : 3956.73 Does anyone have any advice on how to speed up our file transfers? We regularly have to 18 GB worth of files to this system, and it would be very good if we could speed it up. At current speeds, we get no advantage at all from even having gigabit network cards! Please feel free to ask me any other questions about our system setup. Thanks in advance for any advice, Rohit
On Thu, Mar 30, 2006 at 01:52:29PM -0500, Rohit Kumar Mehta wrote:> > I believe I have some hardware or configuration related performance > issues running samba 3.0.14a-3sarge. > > Our server is an Intel Celeron 2 Ghz with 512 MB of RAM and a 3ware > card using SATA disks in a RAID 5 configuration (3ware controller card). > We have a gigabit network and are using Intel Gigabit ethernet cards > e1000). > > When copying large files to the samba shares on the system, the transfer > rate maxes out near 100 mb/s. We tested with nttcp and were able to get > speeds of nearly 800mb/s. So I think it is safe to conclude this is not > a network issue. > > Various tools like top, xosview and mpstat convinced us that we are > bound in the CPU. Stopping the samba file transfer and the cpu idle time > exceeds 90%. We are convinced that our CPU is the bottleneck, > but not sure why.Have you tried transferring the files using smbclient ? That will tell you if it's a Windows client issue. I'm assuming you're using Windows clients although you didn't give that information in your message. Jeremy.
On Thu, 2006-03-30 at 13:52 -0500, Rohit Kumar Mehta wrote:> I believe I have some hardware or configuration related performance > issues running samba 3.0.14a-3sarge. > > Our server is an Intel Celeron 2 Ghz with 512 MB of RAM and a 3ware > card using SATA disks in a RAID 5 configuration (3ware controller card). > We have a gigabit network and are using Intel Gigabit ethernet cards > e1000). > > When copying large files to the samba shares on the system, the transfer > rate maxes out near 100 mb/s. We tested with nttcp and were able to get > speeds of nearly 800mb/s. So I think it is safe to conclude this is not > a network issue. > > Various tools like top, xosview and mpstat convinced us that we are > bound in the CPU. Stopping the samba file transfer and the cpu idle time > exceeds 90%. We are convinced that our CPU is the bottleneck, > but not sure why. > > #cat /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 15 > model : 2 > model name : Intel(R) Celeron(R) CPU 2.00GHz > stepping : 9 > cpu MHz : 1996.920 > cache size : 128 KB > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 2 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge > mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid > bogomips : 3956.73 > > Does anyone have any advice on how to speed up our file transfers? We > regularly have to 18 GB worth of files to this system, and it would be > very good if we could speed it up. At current speeds, we get no > advantage at all from even having gigabit network cards! > > Please feel free to ask me any other questions about our system setup. > Thanks in advance for any advice,Have you done *ANY* system caching parameters, filesystem tuning, or Samba Config tuning? What have you done besides verify it is not the network itself? Have you tested throughput for the 3ware card? I can tell you this, if you have the RAID-5 setup not-optimally to work with the block sizing on your Filesystem you'll never get excellent throughput. I always tend to use largest blocking factors with the 3ware cards for RAID-5. This (for me at least) has proven the fastest and least latency ridden settings for me. But then I am using XFS on all of my 3ware RAID-5 setups. For Mirroring, I typically let the defaults work. Defaults have been by far the best setup for most filesystems. If you still believe you are suffering from CPU overload, I'd suggest sending it to the RAID-5 array with over compressed scp (with mild compression of 4 or 5) and then without compression. See what you get. I am betting the real problem comes from multiple bus-mastering cards conflicting or colliding. The Intel-E1000 and the 3ware card are definitely both bus-mastering. There are a couple of things on the Samba side you can do. Turn off Logging (you don't need it really), change the read and send buffer sizes, change the TCP setting it uses to be more in line with Gigabit, move to using Jumbo frames, get a TOE (TCP Offload Engine) NIC. Then if you still have issues, turn on logging for the stuff you are worried about (auth would be 0, etc...) and then add a sniffer to you connection. You'll definitely find something. My gut reaction is that since this is a Celeron Processor, you really need to goto 64-bit slots on the mother board. Getting a PCI-X capable motherboard would greatly help your problems. One last thing, any of the 95xx cards from 3ware are 3.3V only and are PCI2.3 compliant, they will function incorrectly possibly even be ruined or not recognized by a 5V or Auto-detect 5v/3.3v slot. The 9xxx, 8xxx and 7xxx cards can be used in either a 5V or 3.3V PCI slot. Good luck. -- greg, greg@gregfolkert.net The technology that is Stronger, Better, Faster: Linux Use Debian GNU/Linux, its a bazaar thing NOTICE: Due to Presidential Executive Orders, the National Security Agency may have read this email without warning, warrant, or notice, and certainly without probable cause. They may do this without any judicial or legislative oversight. You have no recourse nor protection. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 191 bytes Desc: This is a digitally signed message part Url : http://lists.samba.org/archive/samba/attachments/20060330/1205727a/attachment.bin
I believe I have some hardware related performance issues running samba 3.0.14a-3sarge. Our server is an Intel Celeron 2 Ghz with 512 MB of RAM and a 3ware card using SATA disks in a RAID 5 configuration (3ware controller card). We have a gigabit network and are using Intel Gigabit ethernet cards e1000). When copying large files to the samba shares on the system, the transfer rate maxes out near 100 mb/s. We tested with nttcp and were able to get speeds of nearly 800mb/s. So I think it is safe to conclude this is a software issue and not a network issue. So running top while doing this large copy, showed that smbd was the top process chewing on >80% of the CPU. Kill the copy, and the cpu idle time exceeds 90%. We are convinced that our CPU is the bottleneck, but not sure why. #cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 2 model name : Intel(R) Celeron(R) CPU 2.00GHz stepping : 9 cpu MHz : 1996.920 cache size : 128 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid bogomips : 3956.73 Does anyone have any advice on how to speed up our file transfers? We regularly have to 18 GB worth of files to this system, and it would be very good if we could speed it up. At current speeds, we get no advantage from even having gigabit network cards! Please feel free to ask me any other questions about our system setup. Thanks in advance for any advice, Rohit
What dows this mean in /var/log/samba/<machine> (apart from everything to do with printing from Windoze to a Linux printer taking such a long time)? error packet at smbd/ipc.c(97) cmd=37 (SMBtrans) STATUS_BUFFER_OVERFLOW Malcolm