Remus tries to go out of the tools directory and build in the kernel directory. This assumes that we''re actually building a kernel out of the xen build tree, and that kernel is actually being used. If Remus needs kernel modules, they should actually be part of the respective kernel trees, not grafted on post-facto. Disable the tools/remus directory until this is sorted out. Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> hg qnew -f didiff -r f45026ec8db5 tools/Makefile --- a/tools/Makefile Mon Aug 09 18:29:50 2010 +0100 +++ b/tools/Makefile Thu Aug 12 17:35:05 2010 -0700 @@ -33,7 +33,7 @@ SUBDIRS-$(CONFIG_IOEMU) += ioemu-dir SUBDIRS-y += xenpmd SUBDIRS-y += libxl -SUBDIRS-y += remus +#SUBDIRS-y += remus SUBDIRS-$(CONFIG_X86) += xenpaging SUBDIRS-$(CONFIG_X86) += debugger/gdbsx _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, 2010-08-13 at 01:38 +0100, Jeremy Fitzhardinge wrote:> Remus tries to go out of the tools directory and build in the kernel > directory. This assumes that we''re actually building a kernel out of > the xen build tree, and that kernel is actually being used. > > If Remus needs kernel modules, they should actually be part of the > respective kernel trees, not grafted on post-facto. > > Disable the tools/remus directory until this is sorted out. > > Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>Acked-by: Ian Campbell <ian.campbell@citrix.com> IanJ and I once spent a headscratching few hours over a kernel build failure which turned out to be because the remus modules were being buyilt before the kernel tree was configured (in a parallel "make world"). Took ages to figure out that was what was going on! Ian.> > hg qnew -f didiff -r f45026ec8db5 tools/Makefile > --- a/tools/Makefile Mon Aug 09 18:29:50 2010 +0100 > +++ b/tools/Makefile Thu Aug 12 17:35:05 2010 -0700 > @@ -33,7 +33,7 @@ > SUBDIRS-$(CONFIG_IOEMU) += ioemu-dir > SUBDIRS-y += xenpmd > SUBDIRS-y += libxl > -SUBDIRS-y += remus > +#SUBDIRS-y += remus > SUBDIRS-$(CONFIG_X86) += xenpaging > SUBDIRS-$(CONFIG_X86) += debugger/gdbsx > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, 13 Aug 2010, Ian Campbell wrote:> On Fri, 2010-08-13 at 01:38 +0100, Jeremy Fitzhardinge wrote: > > Remus tries to go out of the tools directory and build in the kernel > > directory. This assumes that we''re actually building a kernel out of > > the xen build tree, and that kernel is actually being used. > > > > If Remus needs kernel modules, they should actually be part of the > > respective kernel trees, not grafted on post-facto. > > > > Disable the tools/remus directory until this is sorted out. > > > > Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> > > Acked-by: Ian Campbell <ian.campbell@citrix.com> > > IanJ and I once spent a headscratching few hours over a kernel build > failure which turned out to be because the remus modules were being > buyilt before the kernel tree was configured (in a parallel "make > world"). Took ages to figure out that was what was going on! >I''ll apply the patch for now, of course I am more then willing to re-enable remus as soon as the problem is fixed. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini writes ("[Xen-devel] Re: [PATCH] Remus breaks the
build"):> I''ll apply the patch for now, of course I am more then willing to
> re-enable remus as soon as the problem is fixed.
Quite so. I have also removed remus from the nightly tests (since
otherwise this would be regarded as a regression in the cases where
remus still sort of worked; it hasn''t worked properly for some time).
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On Thursday, 12 August 2010 at 17:38, Jeremy Fitzhardinge wrote:> Remus tries to go out of the tools directory and build in the kernel > directory. This assumes that we''re actually building a kernel out of > the xen build tree, and that kernel is actually being used. > > If Remus needs kernel modules, they should actually be part of the > respective kernel trees, not grafted on post-facto. > > Disable the tools/remus directory until this is sorted out. > > Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>I assume you''re talking about this snippet of tools/remus/kmod/Makefile: $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd` modules which expects to find a Makefile in $KERNELDIR but does the actual building in place, in the tools/remus/kmod directory (unless the kernel build system has changed recently?). I thought this was a pretty standard way to build out-of-tree kernel modules. I''m not sure why this is causing you problems (is it?), but if you''re willing to carry sch_queue in the pvops tree, I''d be happy to drop tools/remus/kmod in the unstable tree. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Friday, 13 August 2010 at 13:52, Ian Jackson wrote:> Stefano Stabellini writes ("[Xen-devel] Re: [PATCH] Remus breaks the build"): > > I''ll apply the patch for now, of course I am more then willing to > > re-enable remus as soon as the problem is fixed. > > Quite so. I have also removed remus from the nightly tests (since > otherwise this would be regarded as a regression in the cases where > remus still sort of worked; it hasn''t worked properly for some time).As always, Ian, it would be better if you provided some kind of concrete bug report instead vaguely saying "it doesn''t work". I''ve never received one. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 08/13/2010 12:42 PM, Brendan Cully wrote:> I assume you''re talking about this snippet of tools/remus/kmod/Makefile: > > $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd` modules > > which expects to find a Makefile in $KERNELDIR but does the actual > building in place, in the tools/remus/kmod directory (unless the > kernel build system has changed recently?). I thought this was a > pretty standard way to build out-of-tree kernel modules.I don''t ever build the kernel out of the Xen tree. In general, it assumes the kernel tree has already been configured and built, which may not be true if you''re doing a parallel build, or if you''re building the Xen tree piecewise.> I''m not sure why this is causing you problems (is it?), but if you''re > willing to carry sch_queue in the pvops tree, I''d be happy to drop > tools/remus/kmod in the unstable tree.Yes, I''m happy to include it. Do you have a git reference I can merge from? J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, Aug 13, 2010 at 2:14 PM, Jeremy Fitzhardinge <jeremy@goop.org> wrote:> On 08/13/2010 12:42 PM, Brendan Cully wrote: >> I assume you''re talking about this snippet of tools/remus/kmod/Makefile: >> >> $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd` modules >> >> which expects to find a Makefile in $KERNELDIR but does the actual >> building in place, in the tools/remus/kmod directory (unless the >> kernel build system has changed recently?). I thought this was a >> pretty standard way to build out-of-tree kernel modules. > > I don''t ever build the kernel out of the Xen tree. In general, it > assumes the kernel tree has already been configured and built, which may > not be true if you''re doing a parallel build, or if you''re building the > Xen tree piecewise. > >> I''m not sure why this is causing you problems (is it?), but if you''re >> willing to carry sch_queue in the pvops tree, I''d be happy to drop >> tools/remus/kmod in the unstable tree. > > Yes, I''m happy to include it. Do you have a git reference I can merge from?My understanding is that we don''t need any out-of-tree modules, if linux is built with CONFIG_IFB. Also, that IFB isn''t something available on 2.6.18 (and maybe 2.6.27?), where we will need to build these modules. Is that right ? And, if thats the case, isn''t it better to fold these drivers into 2.6.18 (2.6.27 ?) and support Remus conditional on IFB for pvops tree ?> > J > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Friday, 13 August 2010 at 21:25, Dulloor wrote:> On Fri, Aug 13, 2010 at 2:14 PM, Jeremy Fitzhardinge <jeremy@goop.org> wrote: > > On 08/13/2010 12:42 PM, Brendan Cully wrote: > >> I assume you''re talking about this snippet of tools/remus/kmod/Makefile: > >> > >> $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd` modules > >> > >> which expects to find a Makefile in $KERNELDIR but does the actual > >> building in place, in the tools/remus/kmod directory (unless the > >> kernel build system has changed recently?). I thought this was a > >> pretty standard way to build out-of-tree kernel modules. > > > > I don''t ever build the kernel out of the Xen tree. In general, it > > assumes the kernel tree has already been configured and built, which may > > not be true if you''re doing a parallel build, or if you''re building the > > Xen tree piecewise. > > > >> I''m not sure why this is causing you problems (is it?), but if you''re > >> willing to carry sch_queue in the pvops tree, I''d be happy to drop > >> tools/remus/kmod in the unstable tree. > > > > Yes, I''m happy to include it. Do you have a git reference I can merge from? > > My understanding is that we don''t need any out-of-tree modules, if > linux is built with CONFIG_IFB. > Also, that IFB isn''t something available on 2.6.18 (and maybe > 2.6.27?), where we will need to build these modules. > > Is that right ? And, if thats the case, isn''t it better to fold these > drivers into 2.6.18 (2.6.27 ?) and > support Remus conditional on IFB for pvops tree ?Remus actually uses two modules. One is sch_queue, a Linux queueing discipline that queues outbound guest traffic until the machine state that produced it has been committed at the backup. This is the one that we''re talking about moving into the pvops tree (after I''ve tested it -- I''m having the customary day-long fight getting Xen running smoothly after having not updated for a while). We need a second module (IFB or IMQ, depending on the kernel version) because Linux queueing disciplines only operate on a device''s outbound traffic. Since Remus runs in dom0, it sees the guest''s outbound traffic as _inbound_ traffic on a VIF device. So IMQ/IFB is used to redirect that incoming VIF traffic to a virtual intermediate device with the sch_queue queueing discipline installed on it. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Brendan Cully writes ("Re: [Xen-devel] Re: [PATCH] Remus breaks the
build"):> We need a second module (IFB or IMQ, depending on the kernel version)
> because Linux queueing disciplines only operate on a device''s
outbound
> traffic. Since Remus runs in dom0, it sees the guest''s outbound
> traffic as _inbound_ traffic on a VIF device. So IMQ/IFB is used to
> redirect that incoming VIF traffic to a virtual intermediate device
> with the sch_queue queueing discipline installed on it.
Couldn''t this be achieved simply by putting a dummy bridge or tap
device in the way or something ? You seem to be describing a device
driver whose sole purpose is plumbing ...
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On Wednesday, 18 August 2010 at 14:40, Ian Jackson wrote:> Brendan Cully writes ("Re: [Xen-devel] Re: [PATCH] Remus breaks the build"): > > We need a second module (IFB or IMQ, depending on the kernel version) > > because Linux queueing disciplines only operate on a device''s outbound > > traffic. Since Remus runs in dom0, it sees the guest''s outbound > > traffic as _inbound_ traffic on a VIF device. So IMQ/IFB is used to > > redirect that incoming VIF traffic to a virtual intermediate device > > with the sch_queue queueing discipline installed on it. > > Couldn''t this be achieved simply by putting a dummy bridge or tap > device in the way or something ? You seem to be describing a device > driver whose sole purpose is plumbing ...You are correct, the device is just plumbing. But it isn''t any extra code, at least for pvops -- IFB is carried in upstream linux. You could probably use a second-level bridge or tap or policy routing or something to get the same effect, but I don''t think I see the advantage. You''re certainly welcome to put together an alternative approach for comparison. What would be simpler and more efficient would be to move the queueing directly into netback. That''s on the todo list but hasn''t gotten attention yet. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Friday, 13 August 2010 at 12:44, Brendan Cully wrote:> On Friday, 13 August 2010 at 13:52, Ian Jackson wrote: > > Stefano Stabellini writes ("[Xen-devel] Re: [PATCH] Remus breaks the build"): > > > I''ll apply the patch for now, of course I am more then willing to > > > re-enable remus as soon as the problem is fixed. > > > > Quite so. I have also removed remus from the nightly tests (since > > otherwise this would be regarded as a regression in the cases where > > remus still sort of worked; it hasn''t worked properly for some time). > > As always, Ian, it would be better if you provided some kind of > concrete bug report instead vaguely saying "it doesn''t work". I''ve > never received one.It turns out it was broken... by you, in 21488:dd6bbdc42033 (I missed this because I''ve been running the 4.0 tree until recently). That patch of yours only appeared to be a no-op, because it missed several calls to read_exact. I''ve patchbombed the fix. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Friday, 13 August 2010 at 14:14, Jeremy Fitzhardinge wrote:> On 08/13/2010 12:42 PM, Brendan Cully wrote: > > I assume you''re talking about this snippet of tools/remus/kmod/Makefile: > > > > $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd` modules > > > > which expects to find a Makefile in $KERNELDIR but does the actual > > building in place, in the tools/remus/kmod directory (unless the > > kernel build system has changed recently?). I thought this was a > > pretty standard way to build out-of-tree kernel modules. > > I don''t ever build the kernel out of the Xen tree. In general, it > assumes the kernel tree has already been configured and built, which may > not be true if you''re doing a parallel build, or if you''re building the > Xen tree piecewise. > > > I''m not sure why this is causing you problems (is it?), but if you''re > > willing to carry sch_queue in the pvops tree, I''d be happy to drop > > tools/remus/kmod in the unstable tree. > > Yes, I''m happy to include it. Do you have a git reference I can merge from?That''s more git than I''ve learned. Would a patch suffice? (even producing the diff was non-obvious. It turns out that git add foo; git diff doesn''t include the diff for foo!) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Aug 18, 2010, at 4:26 PM, Brendan Cully wrote:> On Friday, 13 August 2010 at 14:14, Jeremy Fitzhardinge wrote: >> On 08/13/2010 12:42 PM, Brendan Cully wrote: >>> I assume you''re talking about this snippet of tools/remus/kmod/Makefile: >>> >>> $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd` modules >>> >>> which expects to find a Makefile in $KERNELDIR but does the actual >>> building in place, in the tools/remus/kmod directory (unless the >>> kernel build system has changed recently?). I thought this was a >>> pretty standard way to build out-of-tree kernel modules. >> >> I don''t ever build the kernel out of the Xen tree. In general, it >> assumes the kernel tree has already been configured and built, which may >> not be true if you''re doing a parallel build, or if you''re building the >> Xen tree piecewise. >> >>> I''m not sure why this is causing you problems (is it?), but if you''re >>> willing to carry sch_queue in the pvops tree, I''d be happy to drop >>> tools/remus/kmod in the unstable tree. >> >> Yes, I''m happy to include it. Do you have a git reference I can merge from? > > That''s more git than I''ve learned. Would a patch suffice? (even > producing the diff was non-obvious. It turns out that git add foo; git > diff doesn''t include the diff for foo!)Because you staged it. git-diff(1) works before you stage the changes. You probably want to commit and use git-format-patch(1). http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html Regards, Jed Smith Systems Administrator Linode, LLC +1 (609) 593-7103 x1209 jed@linode.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wednesday, 18 August 2010 at 16:34, Jed Smith wrote:> On Aug 18, 2010, at 4:26 PM, Brendan Cully wrote: > > > On Friday, 13 August 2010 at 14:14, Jeremy Fitzhardinge wrote: > >> On 08/13/2010 12:42 PM, Brendan Cully wrote: > >>> I assume you''re talking about this snippet of tools/remus/kmod/Makefile: > >>> > >>> $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd` modules > >>> > >>> which expects to find a Makefile in $KERNELDIR but does the actual > >>> building in place, in the tools/remus/kmod directory (unless the > >>> kernel build system has changed recently?). I thought this was a > >>> pretty standard way to build out-of-tree kernel modules. > >> > >> I don''t ever build the kernel out of the Xen tree. In general, it > >> assumes the kernel tree has already been configured and built, which may > >> not be true if you''re doing a parallel build, or if you''re building the > >> Xen tree piecewise. > >> > >>> I''m not sure why this is causing you problems (is it?), but if you''re > >>> willing to carry sch_queue in the pvops tree, I''d be happy to drop > >>> tools/remus/kmod in the unstable tree. > >> > >> Yes, I''m happy to include it. Do you have a git reference I can merge from? > > > > That''s more git than I''ve learned. Would a patch suffice? (even > > producing the diff was non-obvious. It turns out that git add foo; git > > diff doesn''t include the diff for foo!) > > Because you staged it. git-diff(1) works before you stage the changes. You > probably want to commit and use git-format-patch(1). > > http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.htmlNot that it really matters (I was just sneaking in a rant, and I did manage to produce the diff), but git diff also didn''t include the new file before I added it. It parts ways here with every other version control system I can remember, not to mention intuition. But hey, that''s their problem. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Aug-18 23:54 UTC
Re: [Xen-devel] Re: [PATCH] Remus breaks the build
On 08/18/2010 01:26 PM, Brendan Cully wrote:> That''s more git than I''ve learned. Would a patch suffice? (even > producing the diff was non-obvious. It turns out that git add foo; git > diff doesn''t include the diff for foo!)What''s the origin of the code? Do they have a git tree (I seem to remember one). I can pull from that, then apply any local patches you may have. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wednesday, 18 August 2010 at 16:54, Jeremy Fitzhardinge wrote:> On 08/18/2010 01:26 PM, Brendan Cully wrote: > > That''s more git than I''ve learned. Would a patch suffice? (even > > producing the diff was non-obvious. It turns out that git add foo; git > > diff doesn''t include the diff for foo!) > > What''s the origin of the code? Do they have a git tree (I seem to > remember one). I can pull from that, then apply any local patches you > may have.You''re probably remembering IMQ, which was a big ugly third-party module but is no longer necessary, since we can now use IFB. The little patch I just sent (attached to the last message) was written by me. It''s just a little queueing disciple for buffering outbound traffic until an rtnetlink message releases it. It''s against whatever pvops tree xen-unstable pulls down (xen/master, I guess). Does that patch suffice? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, Aug 18, 2010 at 05:03:48PM -0700, Brendan Cully wrote:> On Wednesday, 18 August 2010 at 16:54, Jeremy Fitzhardinge wrote: > > On 08/18/2010 01:26 PM, Brendan Cully wrote: > > > That''s more git than I''ve learned. Would a patch suffice? (even > > > producing the diff was non-obvious. It turns out that git add foo; git > > > diff doesn''t include the diff for foo!) > > > > What''s the origin of the code? Do they have a git tree (I seem to > > remember one). I can pull from that, then apply any local patches you > > may have. > > You''re probably remembering IMQ, which was a big ugly third-party > module but is no longer necessary, since we can now use IFB. The > little patch I just sent (attached to the last message) was written by > me. It''s just a little queueing disciple for buffering outbound > traffic until an rtnetlink message releases it. It''s against whatever > pvops tree xen-unstable pulls down (xen/master, I guess). Does that > patch suffice? >Current xen-unstable uses xen/stable-2.6.32.x as a default. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Brendan Cully writes ("Re: [Xen-devel] Re: [PATCH] Remus breaks the
build"):> You are correct, the device is just plumbing. But it isn''t any
extra
> code, at least for pvops -- IFB is carried in upstream linux. You
> could probably use a second-level bridge or tap or policy routing or
> something to get the same effect, but I don''t think I see the
> advantage. You''re certainly welcome to put together an alternative
> approach for comparison.
Ah, I see. I hadn''t realised it was in upstream. Fine, then.
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel