I''m using xen-3.0-testing from a week or two ago, and am finding that a recompile of xen-3.0-testing from hg now is all but killing the performance in the other domains. I am running an almost default setup, I''ve just made some small changes to bridging. Any suggestions as to where to start? Or is this a known and solved problem in the latest -testing? Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> I''m using xen-3.0-testing from a week or two ago, and am > finding that a recompile of xen-3.0-testing from hg now is > all but killing the performance in the other domains. > > I am running an almost default setup, I''ve just made some > small changes to bridging. > > Any suggestions as to where to start? Or is this a known and > solved problem in the latest -testing?There have been no changes in 3.0-testing that are likely relevant. I don''t think anyone else has reported this, so I''d look closely at your bridging changes. Also please report the test setup in more detail. You haven''t connected dom0 directly to the bridge rather than vif0.0 to the bridge have you? Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > I''m using xen-3.0-testing from a week or two ago, and am > > finding that a recompile of xen-3.0-testing from hg now is > > all but killing the performance in the other domains. > > > > I am running an almost default setup, I''ve just made some > > small changes to bridging. > > > > Any suggestions as to where to start? Or is this a known and > > solved problem in the latest -testing? > > There have been no changes in 3.0-testing that are likely relevant. I > don''t think anyone else has reported this, so I''d look closely at your > bridging changes. Also please report the test setup in more detail.You> haven''t connected dom0 directly to the bridge rather than vif0.0 tothe> bridge have you?Hmmm... I''m not quite sure what you mean here... I always thought that in dom0 you just connected eth0 (or whatever you might have renamed it to) into the bridge. Is this not the case? ''brctl show'' gives me: br0 8000.00508bea6159 no trunk br1 8000.00508bea6159 no br0.2 vif2.0 I have an Ethernet interface called ''trunk'', which is bridged to br0. br0 then has vlan''s on it (the vlan''s have to be on the bridge interface, not on the Ethernet interface, or it doesn''t work!), giving br0.2, which is in turn bridged into br1. The affected DomU is using br1. None of br0, br1, or trunk have an ip address on them in Dom0. I have another Ethernet interface called ''lan'', which is a gigabit interface and give''s dom0 its connection to the lan, including ATA over Ethernet. Anything hanging off of ''trunk'' is purely for the use of domU''s. Only Dom0 does AoE, All domU''s get their disk access via block devices (which are themselves AoE) exported from dom0. The slowdown is very very obvious, running a ''make world'' in dom0 brings domU to an almost standstill. Ctrl-Z on the make in dom0 instantly brings domU back to life. Even if I disable the Ethernet interface in domU (ifdown eth0), it still runs really really slowly. The system load goes up in domU too, if that means anything. Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > Any suggestions as to where to start? Or is this a known and > > solved problem in the latest -testing? > > There have been no changes in 3.0-testing that are likely relevant. I > don''t think anyone else has reported this, so I''d look closely at your > bridging changes. Also please report the test setup in more detail.You> haven''t connected dom0 directly to the bridge rather than vif0.0 tothe> bridge have you? >I just upgraded both my xen machines to the latest (yesterdays hg pull) and both appear to be doing the same thing now. One of them is SMP though, so the problem is less apparent, but if do a ''cpuburnP6'' on dom0, the domU''s slow down to a crawl. The problem really does appear to be scheduling related, not network, so I haven''t adjusted the network bridge configuration yet, but will do so if you tell me it''s worth a go. Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> I just upgraded both my xen machines to the latest > (yesterdays hg pull) and both appear to be doing the same > thing now. One of them is SMP though, so the problem is less > apparent, but if do a ''cpuburnP6'' on dom0, the domU''s slow > down to a crawl.are the guests SMP? What does ''xm list'' show about the relative CPU burn rates? Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> are the guests SMP?They are all SMP capable (default -xen kernel config), but except for one they are all running with 1 vcpu.> What does ''xm list'' show about the relative CPU burn rates?As I only rebooted recently I don''t have any actual figures, but Dom0 is highish (which makes sense as it was doing a compile), and DomU was low. Neither had unexpected values in them. If you can give me the syntax for rolling back to a specific changeset or time (or I can look it up, but I''m sure you''ll know it off the top of your head :) I can start a binary search on when it all went wrong, and definitely confirm that the problem didn''t exist earlier. Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ha! Look at that. I did: xm sched-sedf Domain-0 0 0 0 1 1 xm sched-sedf mail 0 0 0 1 1 and now I can do a cpuburn (burnP6) in dom0 and you barely notice it. So... questions... Sedf must be the default scheduler? I always thought it was bvt? What are the default settings for it? Obviously they aren''t optimal in my case! How can I query the current scheduler settings? Thanks James> -----Original Message----- > From: Mark Nijmeijer [mailto:mark@nijmeijer.com] > Sent: Friday, 13 January 2006 12:17 > To: James Harper > Subject: Re: [Xen-devel] performance problems... > > From the xen-users mailing list, can you try this ? > > 2) Scheduling issue > > The system was unusable with high CPU load in dom0, all other domains > were almost dead, and there''s apparantly no comprehensivedocumentation> of xen''s schedulers (please point out the relevant docs to me if I''m > wrong, all I found was sedf_scheduler_mini-HOWTO.txt, > xenwiki/Scheduling and `man xm`, none of which helps to understand > scheduling well enough to change its parameters). Thanks to TimFreeman> I found an easy setup that works for me: I set the schedulingparameters> for all domains via "xm sched-sedf <domID> 0 0 0 1 1" or something > similar corresponding to example "4 domains with weights 2:3:4:2" from > the sedf_scheduler_mini-HOWTO.txt with an additional 1 as theextratime> parameter if you want to allow a domain to allocate more CPU time if > other domains are idle. > > > > James Harper wrote: > > >>>Any suggestions as to where to start? Or is this a known and > >>>solved problem in the latest -testing? > >>> > >>> > >>There have been no changes in 3.0-testing that are likely relevant.I> >>don''t think anyone else has reported this, so I''d look closely atyour> >>bridging changes. Also please report the test setup in more detail. > >> > >> > >You > > > > > >>haven''t connected dom0 directly to the bridge rather than vif0.0 to > >> > >> > >the > > > > > >>bridge have you? > >> > >> > >> > > > >I just upgraded both my xen machines to the latest (yesterdays hgpull)> >and both appear to be doing the same thing now. One of them is SMP > >though, so the problem is less apparent, but if do a ''cpuburnP6'' on > >dom0, the domU''s slow down to a crawl. > > > >The problem really does appear to be scheduling related, not network,so> >I haven''t adjusted the network bridge configuration yet, but will doso> >if you tell me it''s worth a go. > > > >Thanks > > > >James > > > >_______________________________________________ > >Xen-devel mailing list > >Xen-devel@lists.xensource.com > >http://lists.xensource.com/xen-devel > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 13 Jan 2006, at 01:33, James Harper wrote:> Sedf must be the default scheduler? I always thought it was bvt? What > are the default settings for it? Obviously they aren''t optimal in my > case!The default is 75% CPU for domain0, which would explain the unfair sharing. :-) -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > The default is 75% CPU for domain0, which would explain the unfair > sharing. :-) >I would have said that DomU had less than 25%, based on its nonexistent performance. Is it a generally held belief that this arrangement is a good thing? Or does it just assume that I won''t do anything silly like xen recompiles under dom0? Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 13 Jan 2006, at 09:56, James Harper wrote:>> The default is 75% CPU for domain0, which would explain the unfair >> sharing. :-) >> > > I would have said that DomU had less than 25%, based on its nonexistent > performance. > > Is it a generally held belief that this arrangement is a good thing? Or > does it just assume that I won''t do anything silly like xen recompiles > under dom0?The default basically assumes that dom0 is a server for other domains, so you want to favour it if it''s runnable. There isn''t much science behind that choice of default, I can assure you. :-) -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel