I''ve written up a short review of the past few months experimenting with a Xen based Ubuntu NX thin client server. http://stateless.geek.nz/2005/08/29/xen-disk-performance/ I''m wondering if other people have experience similar stalling issues with NFS services in a domU. Is it a known issue? -- Nicholas Lee http://stateless.geek.nz gpg 8072 4F86 EDCD 4FC1 18EF 5BDD 07B0 9597 6D58 D70C _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 29 Aug 2005, at 10:29, Nicholas Lee wrote:> I''ve written up a short review of the past few months experimenting > with a Xen based Ubuntu NX thin client server. > > http://stateless.geek.nz/2005/08/29/xen-disk-performance/ > > I''m wondering if other people have experience similar stalling issues > with NFS services in a domU. Is it a known issue?No, I would expect that to work just fine. We certainly run our own servers with most networked services running in domUs, but I don''t think that includes an NFS server (i.e., that configuration is likely untested). What version of Xen are you using? Anything special about your network setup (or is it the default bridged setup)? Any special NFS server/client settings? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 8/29/05, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:> No, I would expect that to work just fine. We certainly run our own > servers with most networked services running in domUs, but I don''t > think that includes an NFS server (i.e., that configuration is likely > untested). > > What version of Xen are you using? Anything special about your network > setup (or is it the default bridged setup)? Any special NFS > server/client settings?Pretty standard Debian sarge installs. nic@bowl:~$ xm dmesg | head -15 | grep versio Xen version 2.0.6 (root@plumtree.co.nz) (gcc version 3.3.5 (Debian 1:3.3.5-13)) Fri Jun 24 01:18:07 NZST 2005 Xen bridged defaults for networking. Bulk standard NFS. rsize and wsize tuning don''t seem to make much difference with the IO stalling issue. In three configurations I''ve tested now with exactly the same domU. NFS home-dir from domU NFS home-dir from xen0 (host xen) LVM home-dir Only the first had these issues. I haven''t tested NFS home-dir from a dom0 (not the host xen). NFS home-dir from domU over a network seemed to work fine. NFS/domU on the same machine is really the only major issue I''ve faced. I have a couple very busy domUs. One with a disk active 4GL accounting application server on a LVM slice, and that has no problems. One of the things I''ve noted when using mutt is particular problems with ''c'' opening a new folder. I''m using courier-imap/maildir, and my mail spool has almost a 1000 folders. -- Nicholas Lee http://stateless.geek.nz gpg 8072 4F86 EDCD 4FC1 18EF 5BDD 07B0 9597 6D58 D70C _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 29 Aug 2005, at 11:23, Nicholas Lee wrote:> One of the things I''ve noted when using mutt is particular problems > with ''c'' opening a new folder. I''m using courier-imap/maildir, and my > mail spool has almost a 1000 folders.We certainly have seen inter-domain network problems in the past, but usually they crop up when running services in dom0, and moving them to domU fixes things. So this is odd because the behaviour appears to be reversed. A few things to try: Use TCP rather than UDP as the transport protocol. Also, try setting rsize/wsize to less than the MTU (e.g. 1024). If the latter fixes things, try setting just one of rsize/wsize to the lower value. UDP and IP fragmentation have been the biggest sources of problems for us. And I''m not totally sure how well Linux NFS implementation recovers from packet loss when it runs over UDP. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 8/29/05, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:> A few things to try: Use TCP rather than UDP as the transport protocol. > Also, try setting rsize/wsize to less than the MTU (e.g. 1024). If the > latter fixes things, try setting just one of rsize/wsize to the lower > value. > > UDP and IP fragmentation have been the biggest sources of problems for > us. And I''m not totally sure how well Linux NFS implementation recovers > from packet loss when it runs over UDP.Just quickly remounting in TCP mode for both the mail and shell server didn''t seem to help the mutt issue. I can reliably get this issue to reoccur by simply ''R'' entering the Drafts folder. Then ''e'' exiting back to the INBOX. I''ll see if I can put together a test account for this at the end of the week. I''ll look at trying NFS with TCP or smaller UDP on the desktop domU tomorrow. -- Nicholas Lee http://stateless.geek.nz gpg 8072 4F86 EDCD 4FC1 18EF 5BDD 07B0 9597 6D58 D70C _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Nicholas Lee wrote:>I''ve written up a short review of the past few months experimenting >with a Xen based Ubuntu NX thin client server. > >http://stateless.geek.nz/2005/08/29/xen-disk-performance/ > >I''m wondering if other people have experience similar stalling issues >with NFS services in a domU. Is it a known issue? > >Nicholas: I''m with Keir, this shouldn''t be happening. I''m with XenSource QA, this looks like an interesting situation. Could you provide us with some additional information (some you''ve already given). There''s a script I''ve written to gather system information called, oddly enough, system-info. Find a copy at http://linuxmafia.com/~karsten/Download/system-info (I''ll see about getting this to the Cambridge site, we''re looking at using this here). Try running that on both the DomU and physical system as this will give us an idea of your system state as well. You might also want to: - Provide specific NFS version info. - Check system logs on client and server for any possible related info. - Monitor traffic (tcpdump, ethereal, iptraf) to determine if latency is visible at network / protocol level. - Provide specific network hardware specs. We *are* seeing issues with specific cards, and have seen some quirks with hub/switch configurations. My own experience with mutt is that it can be notoriously laggy particularly dealing with large or frequently updated mailboxes. It may not be the best general purpose network performance test tool. Perhaps netperf (packaged for Debian) would be more useful. As Kier indicates, we''re largely dogfooding internally with services run via Xen DomUs, so this is a bit of a puzzler. Cheers. -- Karsten M. Self <karsten@xensource.com> XenSource, Inc. 2300 Geng Road #250 +1 650.798.5900 x259 Palo Alto, CA 94303 +1 650.493.1579 fax _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel