On Fri, 14 Oct 2005, Michael VInce wrote:
> I been doing some network benchmarking using netperf and just simple
> 'fetch' on a new network setup to make sure I am getting the most
out of
> the router and servers, I thought I would post some results in case some
> one can help me with my problems or if others are just interested to see
> the results.
Until recently (or maybe still), netperf was compiled with -DHISTOGRAM by
our port/package, which resulted in a significant performance drop. I
believe that the port maintainer and others have agreed to change it, but
I'm not sure if it's been committed yet, or which packages have been
rebuilt. You may want to manually rebuild it to make sure -DHISTOGRAM
isn't set.
You may want to try setting net.isr.direct=1 and see what performance
impact that has for you.
Robert N M Watson
>
> The network is currently like this, where machines A and B are the Dell
1850s
> and C is the 2850 x 2 CPU (Server C has Apache2 worker MPM on it) and
server
> B is the gateway and A is acting as a client for fetch and netperf tests.
> A --- B --- C
> The 2 1850s are running AMD64 Freebsd 6.0rc1 (A and B) while C is running
> 5.4-stable i386 from Oct 12
>
> My main problem is that if I compile SMP into the machine C (5.4stable) the
> network speed goes down to a range between 6mbytes/sec to 15mbytes/sec on
> SMP.
> If I use GENERIC kernel the performance goes up to what I have show below
> which is around 65megabytes/sec for a 'fetch' get test from Apache
server and
> 933mbits/sec for netperf.
> Does any know why why network performance would be so bad on SMP?
>
> Does any one think that if I upgrade the i386 SMP server to 6.0RC1 the SMP
> network performance would improve? This server will be running java so I
need
> it to be stable and is the the reason I am using i386 and Java 1.4
>
> I am happy with performance of direct machine to machine (non SMP) which is
> pretty much full 1gigabit/sec speeds.
> Going through the gateway server-B seems to drop its speed down a bit for
in
> and out direction tcp speed tests using netperf I get around 266mbits/sec
> from server A through gateway Server-B to server-C which is quite adequate
> for the link I currently have for it.
>
> Doing a 'fetch' get for a 1gig file from the Apache server gives
good speeds
> of close to 600mbits/sec but netperf shows its weakness with 266mbits/sec.
> This is as fast as I need it to be but does any one know the weak points on
> the router gateway to make it faster? Is this the performance I should
expect
> for FreeBSD as a router with gigabit ethers?
>
> I have seen 'net.inet.ip.fastforwarding' in some peoples router
setups on the
> list but nothing about what it does or what it can affect.
> I haven't done any testing with polling yet but if I can get over
> 900mbits/sec on the interfaces does polling help with passing packets from
> one interface to the other?
> All machines have PF running other then that they don't really have any
> sysctls or special kernel options.
>
> Here are some speed benchmarks using netperf and 'fetch' gets.
>
> Server A to server C with server C using SMP kernel and just GENERIC kernel
> further below
>
> B# /usr/local/netperf/netperf -l 10 -H server-C -t TCP_STREAM -i 10,2 -I
99,5
> -- -m 4096 -s 57344 -S 57344
> TCP STREAM TEST to server-C : +/-2.5% @ 99% conf. : histogram
> Recv Send Send
> Socket Socket Message Elapsed
> Size Size Size Time Throughput
> bytes bytes bytes secs. 10^6bits/sec
>
> 57344 57344 4096 10.06 155.99
> tank# fetch -o - > /dev/null http://server-C/file1gig.iso
> - 100% of 1055 MB 13 MBps
> 00m00s
>
> ##### Using generic non SMP kernel
> Server A to server C with server C using GENERIC kernel.
> A# fetch -o - > /dev/null http://server-C/file1gig.iso
> - 100% of 1055 MB 59 MBps
> 00m00s
>
> A# ./tcp_stream_script server-C
>
> /usr/local/netperf/netperf -l 60 -H server-C -t TCP_STREAM -i 10,2 -I 99,5
--
> -m 4096 -s 57344 -S 57344
>
> Recv Send Send
> Socket Socket Message Elapsed
> Size Size Size Time Throughput
> bytes bytes bytes secs. 10^6bits/sec
>
> 57344 57344 4096 60.43 266.92
>
> ------------------------------------
> ###############################################
> Connecting from server-A to B (gateway)
> A# ./tcp_stream_script server-B
>
> ------------------------------------
>
> /usr/local/netperf/netperf -l 60 -H server-B -t TCP_STREAM -i 10,2 -I 99,5
--
> -m 4096 -s 57344 -S 57344
>
> TCP STREAM TEST to server-B : +/-2.5% @ 99% conf. : histogram
> Recv Send Send
> Socket Socket Message Elapsed
> Size Size Size Time Throughput
> bytes bytes bytes secs. 10^6bits/sec
>
> 57344 57344 4096 61.80 926.82
>
> ------------------------------------
> ##########################################
> Connecting from server B (gateway) to server C
> Fetch and Apache2 test
> B# fetch -o - > /dev/null http://server-C/file1gig.iso
> - 100% of 1055 MB 74 MBps
> 00m00s
>
> Netperf test
> B# /usr/local/netperf/tcp_stream_script server-C
>
> /usr/local/netperf/netperf -l 60 -H server-C -t TCP_STREAM -i 10,2 -I 99,5
--
> -m 4096 -s 57344 -S 57344
>
> TCP STREAM TEST to server-C : +/-2.5% @ 99% conf. : histogram
> Recv Send Send
> Socket Socket Message Elapsed
> Size Size Size Time Throughput
> bytes bytes bytes secs. 10^6bits/sec
>
> 57344 57344 4096 62.20 933.94
>
> ------------------------------------
>
> Cheers,
> Mike
>
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
"freebsd-net-unsubscribe@freebsd.org"
>