Linspeed
2008-Oct-23 12:54 UTC
[Xen-users] Fast write performance, slow read performance in NFS
Hello I need to setup shared storage for several virtual machines I''m running in Xen. I''ve gone for NFS. Native performance on the disk that is shared is as follows: Read: 199 MB/second Write: 158 MB/second Now I''ve done lots of testing with NFS options on the client (the virtual machine). I''ve managed to get the same write performance as I get natively, but read performance is way below half that of native. Around 70 MB/second. Current setup on the server: /disks/shared *(rw,async,no_root_squash,no_subtree_check) /etc/fstab on the virtual machine: host:/disks/shared /nfs nfs rsize=262144,wsize=262144,hard,intr,rw 0 0 I have tried lots of settings for rsize, but it makes no difference. wsize did - I got large performance increases by increasing the wsize. When I run mount -v: nfs (rw,rsize=262144,wsize=262144,hard,intr,addr=200.200.200.184,nfsvers=3,proto=tcp,mountproto=udp) I am using openSUSE 11.0. Kernel is 2.6.27. This problem also exists with kernel 2.6.25. Has anyone else had this problem, and if so how did you solve it? Thanks -- View this message in context: http://www.nabble.com/Fast-write-performance%2C-slow-read-performance-in-NFS-tp20130320p20130320.html Sent from the Xen - User mailing list archive at Nabble.com. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Nathan
2008-Oct-23 20:05 UTC
Re: [Xen-users] Fast write performance, slow read performance in NFS
On Thu, Oct 23, 2008 at 2:54 PM, Linspeed <lee@linspeed.com> wrote:> > Hello > > I need to setup shared storage for several virtual machines I''m running in > Xen. > I''ve gone for NFS. > > Native performance on the disk that is shared is as follows: > Read: 199 MB/second > Write: 158 MB/second > > Now I''ve done lots of testing with NFS options on the client (the virtual > machine). > I''ve managed to get the same write performance as I get natively, but read > performance is way below half that of native. Around 70 MB/second.158 MB/second, counting overhead that''s something like 1.5 Gbit/second . . . What kind of network is this? This is very off the cuff, but in your place I''d test by maximizing your MTU and varying - between tcp and udp (you''ve tested udp) - nfsv3 and nfsv4 (you''ve tested nfsv3) - [r|w]sizes lower than MTU size or much bigger (you''ve tested much bigger, and your default was probably 8192 with MTU of 1500 which is no good). With default size of 8192 octets you get a boost in performance when you push MTU to contain that size, MTU 9000 is good, I suppose that with your network you can do that :-) Don''t forget NFS server-side cache, I suppose you can tune that, it shouldn''t change your test values though(otherwise there''s something wrong with your tests) -- HTH, Nathan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users