Xen devel - May 2009 - 3.4.0 wants to download Linux despite its presence

I tried at first to build the integrated sources, but couldn''t get it
to
work.  I saw a list message that said it was broken, so I tried building 
the separate xen 3.4.0 and Linux tarballs.

However, no matter what I try, the build hits the Mercurial server to 
download the Linux kernel sources.  I have tried a number of things, but 
I am completely unable to prevent the download.

May I respectfully suggest that it''s A REALLY BAD IDEA to automatically
download dependencies of you find them missing?

Much better would be to just list what the dependencies are in the 
README, as well as where to get them.  If the build finds them missing, 
then it should just say so and stop.

I''ve been trying to get a working build of Xen for WEEKS now, with very
little success.  I am able to boot Xen and a pv_ops Dom0, but that''s as
far as I get.

Please help me.

I really, really want to be able to help out by contributing useful code 
to the project, but the troubles I keep having with the build have had 
me stymied.

Mike
-- 
Michael David Crawford
mdc@prgmr.com

    prgmr.com - We Don''t Assume You Are Stupid.

       Xen-Powered Virtual Private Servers: http://prgmr.com/xen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Michael David Crawford writes ("[Xen-devel] 3.4.0 wants to download Linux
despite its presence"):
> However, no matter what I try, the build hits the Mercurial server to 
> download the Linux kernel sources.  I have tried a number of things, but 
> I am completely unable to prevent the download.
The build system is rather unfortunate.  You can completely prevent
the Xen build from building a kernel by setting XKERNELS=''''
You''ll have to build the kernel yourself then of course.
> May I respectfully suggest that it''s A REALLY BAD IDEA to
automatically
> download dependencies of you find them missing?
Sadly it''s not just the kernel which is done this way.  There are a
handful of tarballs of various kinds too.  If you want to deal with
those offline too you''ll find that you can set XEN_EXTFILES_URL.
> Much better would be to just list what the dependencies are in the 
> README, as well as where to get them.  If the build finds them missing, 
> then it should just say so and stop.
I would actually agree.  Having a part of the build system which
brings all the pieces together automatically for you would be good,
but it would be sensible for xen-unstable.hg to be less enthusiastic
about downloading stuff.

I''m not sure the community really agrees with me though.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 20/05/2009 07:57, "Ian Jackson" <Ian.Jackson@eu.citrix.com>
wrote:
>> Much better would be to just list what the dependencies are in the
>> README, as well as where to get them.  If the build finds them missing,
>> then it should just say so and stop.
> 
> I would actually agree.  Having a part of the build system which
> brings all the pieces together automatically for you would be good,
> but it would be sensible for xen-unstable.hg to be less enthusiastic
> about downloading stuff.
Perhaps we should more prominently advertise ''make install-xen
install-tools''. This should generally result in no downloads apart from
qemu, and that''s a special-case dependency since it ties quite closely
to
xen-unstable -- we can''t just use any qemu tree. And of course you can
define XEN_EXTFILES_URL so as to make any download attempt fail, and then
you can go grab the packages yourself as you see fit and assemble your
requirements list. Imagine the noise on the mailing list if xen-unstable
didn''t try to assemble itself automatically out of the box.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
> On 20/05/2009 07:57, "Ian Jackson"
<Ian.Jackson@eu.citrix.com> wrote:
>
>   
>>> Much better would be to just list what the dependencies are in the
>>> README, as well as where to get them.  If the build finds them
missing,
>>> then it should just say so and stop.
>>>       
>> I would actually agree.  Having a part of the build system which
>> brings all the pieces together automatically for you would be good,
>> but it would be sensible for xen-unstable.hg to be less enthusiastic
>> about downloading stuff.
>>     
>
> Perhaps we should more prominently advertise ''make install-xen
> install-tools''. This should generally result in no downloads apart
from
> qemu, and that''s a special-case dependency since it ties quite
closely to
> xen-unstable -- we can''t just use any qemu tree. And of course you
can
> define XEN_EXTFILES_URL so as to make any download attempt fail, and then
> you can go grab the packages yourself as you see fit and assemble your
> requirements list. Imagine the noise on the mailing list if xen-unstable
> didn''t try to assemble itself automatically out of the box.
>   
What about adding "make fetch-dependencies", and have the default
build
say something like "X is missing.  Type "make
fetch-dependencies"."?

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 20/05/2009 14:12, "Jeremy Fitzhardinge" <jeremy@goop.org>
wrote:
>> Perhaps we should more prominently advertise ''make install-xen
>> install-tools''. This should generally result in no downloads
apart from
>> qemu, and that''s a special-case dependency since it ties quite
closely to
>> xen-unstable -- we can''t just use any qemu tree. And of course
you can
>> define XEN_EXTFILES_URL so as to make any download attempt fail, and
then
>> you can go grab the packages yourself as you see fit and assemble your
>> requirements list. Imagine the noise on the mailing list if
xen-unstable
>> didn''t try to assemble itself automatically out of the box.
>>   
> 
> What about adding "make fetch-dependencies", and have the default
build
> say something like "X is missing.  Type "make
fetch-dependencies"."?
I could live with that, yes.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Jackson wrote:
> Michael David Crawford writes ("[Xen-devel] 3.4.0 wants to download
Linux despite its presence"):
>> However, no matter what I try, the build hits the Mercurial server to 
>> download the Linux kernel sources.  I have tried a number of things,
but
>> I am completely unable to prevent the download.
> 
> The build system is rather unfortunate.  You can completely prevent
> the Xen build from building a kernel by setting
XKERNELS=''''
> You''ll have to build the kernel yourself then of course.
I''m afraid that didn''t work - it still downloaded from
Mercurial, and is
building it now.

I''m just going to let it build, but use a kernel that I previously
built.

Jeremy''s suggestion to "make fetch-dependencies" is I think a
good one.

I''ve tried "make xen" and "make tools", rather than
"make world", but
the impression I have is that they leave the build incomplete somehow. 
I was never able to get a DomU to load and would get all sorts of arcane 
messages.

I promise to keep trying though - I really *want* Xen to work.

I don''t think it''s so bad to automatically fetch all the
dependencies
other that Linux.

One reason I wanted to have my own Linux kernel was so I could use a 
RAID driver that I backported to 2.6.18 from the current release.  I 
would imagine many users would have specific requirements for their 
Linux kernel that the default Xen Linux cannot satisfy.

Mike
-- 
Michael David Crawford
mdc@prgmr.com

    prgmr.com - We Don''t Assume You Are Stupid.

       Xen-Powered Virtual Private Servers: http://prgmr.com/xen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Doing make xen and make tools should be just fine (I do this regularly, as 
I don''t want or need Linux to be constantly rebuilt and reinstalled
when
I''m only changing things to Xen and/or tools).

Did you install them after building (make install-xen and make 
install-tools)? I think you need to be sudo for it to install properly.


Patrick


Michael David Crawford wrote:
> Ian Jackson wrote:
>> Michael David Crawford writes ("[Xen-devel] 3.4.0 wants to
download
>> Linux despite its presence"):
>>> However, no matter what I try, the build hits the Mercurial server
to
>>> download the Linux kernel sources.  I have tried a number of
things,
>>> but I am completely unable to prevent the download.
>>
>> The build system is rather unfortunate.  You can completely prevent
>> the Xen build from building a kernel by setting
XKERNELS=''''
>> You''ll have to build the kernel yourself then of course.
> 
> I''m afraid that didn''t work - it still downloaded from
Mercurial, and is
> building it now.
> 
> I''m just going to let it build, but use a kernel that I previously
built.
> 
> Jeremy''s suggestion to "make fetch-dependencies" is I
think a good one.
> 
> I''ve tried "make xen" and "make tools", rather
than "make world", but
> the impression I have is that they leave the build incomplete somehow. I 
> was never able to get a DomU to load and would get all sorts of arcane 
> messages.
> 
> I promise to keep trying though - I really *want* Xen to work.
> 
> I don''t think it''s so bad to automatically fetch all the
dependencies
> other that Linux.
> 
> One reason I wanted to have my own Linux kernel was so I could use a 
> RAID driver that I backported to 2.6.18 from the current release.  I 
> would imagine many users would have specific requirements for their 
> Linux kernel that the default Xen Linux cannot satisfy.
> 
> Mike
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Patrick Colp wrote:
> Doing make xen and make tools should be just fine (I do this regularly, 
> as I don''t want or need Linux to be constantly rebuilt and
reinstalled
> when I''m only changing things to Xen and/or tools).
I don''t know for sure, but I have the theory that "make
world" will
build things that "make xen" and "make tools"
don''t, which will then
persist in being installed when later tools builds miss some things.
> Did you install them after building (make install-xen and make 
> install-tools)? I think you need to be sudo for it to install properly.
I think I tried those just a couple times, but killed them when they 
started to build the Linux kernel again.

I''ve been using install.sh, but that turns out not to check that all
the
required files actually exist - it just installs whatever is in 
dist/install, even if something is missing.

I have a "make world" going right now, and will do a "make
install" when
that''s done.  I decided to just let it build Linux, just so I can be 
certain everything else builds.

Mike
-- 
Michael David Crawford
mdc@prgmr.com

    prgmr.com - We Don''t Assume You Are Stupid.

       Xen-Powered Virtual Private Servers: http://prgmr.com/xen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Michael David Crawford wrote:
> Patrick Colp wrote:
>> Doing make xen and make tools should be just fine (I do this regularly,
>> as I don''t want or need Linux to be constantly rebuilt and
reinstalled
>> when I''m only changing things to Xen and/or tools).
> 
> I don''t know for sure, but I have the theory that "make
world" will
> build things that "make xen" and "make tools"
don''t, which will then
> persist in being installed when later tools builds miss some things.
Hmm... I don''t know if I''ve ever used "make world",
actually. If I want to
build everything, I just do "make" (it seems the difference is that
world
will do a clean and kdelete first).

The only difference I can see between using world vs xen and tools is that 
world also builds the stubdom (which can be done separately with "make 
stubdom").

So perhaps the thing to do if you want stubdom support too is to do:

sudo make install-xen
sudo make install-tools
sudo make install-stubdom

>> Did you install them after building (make install-xen and make 
>> install-tools)? I think you need to be sudo for it to install properly.
> 
> I think I tried those just a couple times, but killed them when they 
> started to build the Linux kernel again.
I don''t think they should do that... but maybe they need to build the
linux
kernel once. I''m not sure, since I did build and install the linux
kernel
(but only once, ever after I just need to do xen and tools).

> I have a "make world" going right now, and will do a "make
install" when
> that''s done.  I decided to just let it build Linux, just so I can
be
> certain everything else builds.
Yes, this certainly seems like the safe thing to do.


Patrick

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Am Freitag, den 22.05.2009, 02:11 -0700 schrieb Michael David
Crawford:
> Patrick Colp wrote:
> > Doing make xen and make tools should be just fine (I do this
regularly,
> > as I don''t want or need Linux to be constantly rebuilt and
reinstalled
> > when I''m only changing things to Xen and/or tools).
> 
> I don''t know for sure, but I have the theory that "make
world" will
> build things that "make xen" and "make tools"
don''t, which will then
> persist in being installed when later tools builds miss some things.
> 
> > Did you install them after building (make install-xen and make 
> > install-tools)? I think you need to be sudo for it to install
properly.
> 
> I think I tried those just a couple times, but killed them when they 
> started to build the Linux kernel again.
> 
> I''ve been using install.sh, but that turns out not to check that
all the
> required files actually exist - it just installs whatever is in 
> dist/install, even if something is missing.
> 
> I have a "make world" going right now, and will do a "make
install" when
> that''s done.  I decided to just let it build Linux, just so I can
be
> certain everything else builds.
> 
> Mike
Hello Mike,

just don''t use "make world" as it includes a call to
"make clean" which
will delete the included linux tree before the actual build and it will
dowload the linux tree again during the build phase. Use "make dist"
instead.

Marc


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>> I don''t know for sure, but I have the theory that "make
world" will
>> build things that "make xen" and "make tools"
don''t, which will then
>> persist in being installed when later tools builds miss some things.
This seems to actually be the case: when I just did "make xen" and
"make
world", the only thing in the dist directory was the the install directory.

Just now I had a "make world" complete successfully, and now the dist 
directory *also* contains check, COPYING, install.sh and README.

Mike
-- 
Michael David Crawford
mdc@prgmr.com

    prgmr.com - We Don''t Assume You Are Stupid.

       Xen-Powered Virtual Private Servers: http://prgmr.com/xen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel