Xen users - Feb 2011 - how to reduce time of git pulling each time when you do a make world on Xen source

I am compiling xen from source and each time I do a `make world`
it basically gives some or the other error
my problem are not those errors ( I am trying to debug them)
but the problem is each time when I do a `make world`

Xen basically pulls things from git repository

   + rm -rf linux-2.6-pvops.git linux-2.6-pvops.git.tmp
   + mkdir linux-2.6-pvops.git.tmp
   + rmdir linux-2.6-pvops.git.tmp
   + git clone -o xen -n
git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
linux-2.6-pvops.git.tmp
   Initialized empty Git repository in
/usr/src/xen-4.0.1/linux-2.6-pvops.git.tmp/.git/
   remote: Counting objects: 1941611, done.
   remote: Compressing objects: 100% (319127/319127), done.
remote: Total 1941611 (delta 1614302), reused 1930655 (delta 1604595)
**Receiving objects: 20% (1941611/1941611), 98.17 MiB | 87 KiB/s, done.**


and if you notice the last line it is still consuming my bandwidth
pulling things from internet.How can I stop this step each time and
use existing git repository?

--


-- 
http://mightydreams.blogspot.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Maybe "make dist" is what you are looking for?

it builds whole xen into a directory "dist" where you will find an 
"install.sh" for installing xen.

Patrick
> I am compiling xen from source and each time I do a `make world`
> it basically gives some or the other error
> my problem are not those errors ( I am trying to debug them)
> but the problem is each time when I do a `make world`
> 
> Xen basically pulls things from git repository
> 
>    + rm -rf linux-2.6-pvops.git linux-2.6-pvops.git.tmp
>    + mkdir linux-2.6-pvops.git.tmp
>    + rmdir linux-2.6-pvops.git.tmp
>    + git clone -o xen -n
> git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
> linux-2.6-pvops.git.tmp
>    Initialized empty Git repository in
> /usr/src/xen-4.0.1/linux-2.6-pvops.git.tmp/.git/
>    remote: Counting objects: 1941611, done.
>    remote: Compressing objects: 100% (319127/319127), done.
> remote: Total 1941611 (delta 1614302), reused 1930655 (delta 1604595)
> **Receiving objects: 20% (1941611/1941611), 98.17 MiB | 87 KiB/s, done.**
> 
> 
> and if you notice the last line it is still consuming my bandwidth
> pulling things from internet.How can I stop this step each time and
> use existing git repository?
> 
> --
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

What is the difference between
make world
make dist
make xen ?

On Sat, Feb 26, 2011 at 7:54 PM, Patrick Scharrenberg
<patrick@scharrenberg.com> wrote:
> Maybe "make dist" is what you are looking for?
>
> it builds whole xen into a directory "dist" where you will find
an
> "install.sh" for installing xen.
>
> Patrick
>
>> I am compiling xen from source and each time I do a `make world`
>> it basically gives some or the other error
>> my problem are not those errors ( I am trying to debug them)
>> but the problem is each time when I do a `make world`
>>
>> Xen basically pulls things from git repository
>>
>>    + rm -rf linux-2.6-pvops.git linux-2.6-pvops.git.tmp
>>    + mkdir linux-2.6-pvops.git.tmp
>>    + rmdir linux-2.6-pvops.git.tmp
>>    + git clone -o xen -n
>> git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
>> linux-2.6-pvops.git.tmp
>>    Initialized empty Git repository in
>> /usr/src/xen-4.0.1/linux-2.6-pvops.git.tmp/.git/
>>    remote: Counting objects: 1941611, done.
>>    remote: Compressing objects: 100% (319127/319127), done.
>> remote: Total 1941611 (delta 1614302), reused 1930655 (delta 1604595)
>> **Receiving objects: 20% (1941611/1941611), 98.17 MiB | 87 KiB/s,
done.**
>>
>>
>> and if you notice the last line it is still consuming my bandwidth
>> pulling things from internet.How can I stop this step each time and
>> use existing git repository?
>>
>> --
>


--

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

You can take a look into the Makefile:
there you''ll find e.g.:

world: 
        $(MAKE) clean
        $(MAKE) kdelete
        $(MAKE) dist

so what a "make world" does is
   make clean
   make kdelete
and finally
   make dist

so if you solely do a "make dist" the build process skips the
deletition of
the compiled binary objects (make clean) and skips deleting your checked out 
kernel tree (make kdelete)

so a "make dist" after an upgraded working copy (hg pull -u) only
compiles the
files which have changed, which saves time.

Please have a look at the Makefile, do a "make help" and take a look
at the
README file for further information.

Patrick



> What is the difference between
> make world
> make dist
> make xen ?
> 
> On Sat, Feb 26, 2011 at 7:54 PM, Patrick Scharrenberg
> 
> <patrick@scharrenberg.com> wrote:
> > Maybe "make dist" is what you are looking for?
> > 
> > it builds whole xen into a directory "dist" where you will
find an
> > "install.sh" for installing xen.
> > 
> > Patrick
> > 
> >> I am compiling xen from source and each time I do a `make world`
> >> it basically gives some or the other error
> >> my problem are not those errors ( I am trying to debug them)
> >> but the problem is each time when I do a `make world`
> >> 
> >> Xen basically pulls things from git repository
> >> 
> >>    + rm -rf linux-2.6-pvops.git linux-2.6-pvops.git.tmp
> >>    + mkdir linux-2.6-pvops.git.tmp
> >>    + rmdir linux-2.6-pvops.git.tmp
> >>    + git clone -o xen -n
> >> git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
> >> linux-2.6-pvops.git.tmp
> >>    Initialized empty Git repository in
> >> /usr/src/xen-4.0.1/linux-2.6-pvops.git.tmp/.git/
> >>    remote: Counting objects: 1941611, done.
> >>    remote: Compressing objects: 100% (319127/319127), done.
> >> remote: Total 1941611 (delta 1614302), reused 1930655 (delta
1604595)
> >> **Receiving objects: 20% (1941611/1941611), 98.17 MiB | 87 KiB/s,
> >> done.**
> >> 
> >> 
> >> and if you notice the last line it is still consuming my bandwidth
> >> pulling things from internet.How can I stop this step each time
and
> >> use existing git repository?
> >> 
> >> --
> 
> --
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Wow thanks I was looking for this information only.

On Sat, Feb 26, 2011 at 9:59 PM, Patrick Scharrenberg <pittipatti@web.de>
wrote:
> You can take a look into the Makefile:
> there you''ll find e.g.:
>
> world:
>        $(MAKE) clean
>        $(MAKE) kdelete
>        $(MAKE) dist
>
> so what a "make world" does is
>   make clean
>   make kdelete
> and finally
>   make dist
>
> so if you solely do a "make dist" the build process skips the
deletition of
> the compiled binary objects (make clean) and skips deleting your checked
out
> kernel tree (make kdelete)
>
> so a "make dist" after an upgraded working copy (hg pull -u) only
compiles the
> files which have changed, which saves time.
>
> Please have a look at the Makefile, do a "make help" and take a
look at the
> README file for further information.
>
> Patrick
>
>
>
>
>> What is the difference between
>> make world
>> make dist
>> make xen ?
>>
>> On Sat, Feb 26, 2011 at 7:54 PM, Patrick Scharrenberg
>>
>> <patrick@scharrenberg.com> wrote:
>> > Maybe "make dist" is what you are looking for?
>> >
>> > it builds whole xen into a directory "dist" where you
will find an
>> > "install.sh" for installing xen.
>> >
>> > Patrick
>> >
>> >> I am compiling xen from source and each time I do a `make
world`
>> >> it basically gives some or the other error
>> >> my problem are not those errors ( I am trying to debug them)
>> >> but the problem is each time when I do a `make world`
>> >>
>> >> Xen basically pulls things from git repository
>> >>
>> >>    + rm -rf linux-2.6-pvops.git linux-2.6-pvops.git.tmp
>> >>    + mkdir linux-2.6-pvops.git.tmp
>> >>    + rmdir linux-2.6-pvops.git.tmp
>> >>    + git clone -o xen -n
>> >> git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
>> >> linux-2.6-pvops.git.tmp
>> >>    Initialized empty Git repository in
>> >> /usr/src/xen-4.0.1/linux-2.6-pvops.git.tmp/.git/
>> >>    remote: Counting objects: 1941611, done.
>> >>    remote: Compressing objects: 100% (319127/319127), done.
>> >> remote: Total 1941611 (delta 1614302), reused 1930655 (delta
1604595)
>> >> **Receiving objects: 20% (1941611/1941611), 98.17 MiB | 87
KiB/s,
>> >> done.**
>> >>
>> >>
>> >> and if you notice the last line it is still consuming my
bandwidth
>> >> pulling things from internet.How can I stop this step each
time and
>> >> use existing git repository?
>> >>
>> >> --
>>
>> --
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>


-- 
http://mightydreams.blogspot.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Sat, Feb 26, 2011 at 9:59 PM, Patrick Scharrenberg <pittipatti@web.de>
wrote:
> You can take a look into the Makefile:
> there you''ll find e.g.:
>
> world:
>        $(MAKE) clean
>        $(MAKE) kdelete
>        $(MAKE) dist
>
> so what a "make world" does is
>   make clean
>   make kdelete
> and finally
>   make dist
>
> so if you solely do a "make dist" the build process skips the
deletition of
> the compiled binary objects (make clean) and skips deleting your checked
out
> kernel tree (make kdelete)
>
> so a "make dist" after an upgraded working copy (hg pull -u) only
compiles the
> files which have changed, which saves time.
>
> Please have a look at the Makefile, do a "make help" and take a
look at the
> README file for further information.
>
> Patrick
>
>
>
>

Ok I checked a few more archives where  I got one
http://web.archiveorange.com/archive/v/ZvO4jYCfIzdpcmkWRz5w
it is Boris message where he mentions to clone git repository after
you have untarred Xen source and done
make xen etc.

Where he mentions

"Then clone JF''s Git Repo, checkout git branch as you want and
build
corresponding
kernel.That''s all i always do."

Which kernel is he referring to is it Dom0?
because when I ran make xen it worked without  any problem and I can see in my
/usr/src/xen-4.0.1/dist/install/boot

-rw-r--r-- 1 root tapas  572865 2011-02-26 07:24 xen-4.0.1.gz
lrwxrwxrwx 1 root tapas      12 2011-02-26 07:24 xen-4.0.gz -> xen-4.0.1.gz
lrwxrwxrwx 1 root tapas      12 2011-02-26 07:24 xen-4.gz -> xen-4.0.1.gz
lrwxrwxrwx 1 root tapas      12 2011-02-26 07:24 xen.gz -> xen-4.0.1.gz
-rw-r--r-- 1 root tapas 8005931 2011-02-26 07:24 xen-syms-4.0.1

So does that mean I have xen here.
So why am I doing make world what is the purpose of make world?
This part is not clear to me.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Sat, Feb 26, 2011 at 10:33 PM, Tapas Mishra <mightydreams@gmail.com>
wrote:
> On Sat, Feb 26, 2011 at 9:59 PM, Patrick Scharrenberg
<pittipatti@web.de> wrote:
>> You can take a look into the Makefile:
>> there you''ll find e.g.:
>>
>> world:
>>        $(MAKE) clean
>>        $(MAKE) kdelete
>>        $(MAKE) dist
>>
>> so what a "make world" does is
>>   make clean
>>   make kdelete
>> and finally
>>   make dist
>>
>> so if you solely do a "make dist" the build process skips the
deletition of
>> the compiled binary objects (make clean) and skips deleting your
checked out
>> kernel tree (make kdelete)
Thanks for this information Patrick.
>> so a "make dist" after an upgraded working copy (hg pull -u)
only compiles the
>> files which have changed, which saves time.
>>
>> Please have a look at the Makefile, do a "make help" and take
a look at the
>> README file for further information.
>>
>> Patrick
>>
>>
>>
>>
>
After digging this problem for some time I finally found a remedy for the same.

One needs to fix the src.git-clone file  for the same from the xen-4.0 tar


When you untar xen-4.0.1.tar then you see

xen-4.0.1/buildconfigs directory open file
src.git-clone and comment out following

[ -d $(LINUX_SRCDIR) ]; then \
               rm -rf $(LINUX_SRCDIR) $(LINUX_SRCDIR).tmp; \
               mkdir $(LINUX_SRCDIR).tmp; rmdir $(LINUX_SRCDIR).tmp; \
               $(GIT) clone -o $(XEN_GIT_ORIGIN) -n
$(XEN_LINUX_GIT_URL) $(LINUX_SRCDIR).tmp; \
               (cd $(LINUX_SRCDIR).tmp; git checkout -b
$(XEN_LINUX_GIT_LOCALBRANCH) $(XEN_LINUX_GITREV) ); \
               mv $(LINUX_SRCDIR).tmp $(LINUX_SRCDIR); \
fi

The above lines in src.git-clone are responsible for this time consuming pull.

So what I did was manually did a pull in the above directory location
and during make xen and make tools etc I had had uncommented above
lines.So that pull does not happen again and again.

Now when you need to compile again and again xen-tools you can simply
copy the $HOME/linux-2.6-pvops.git.tmp to /usr/src/xen-4.0.1/

and  begin the compile process again.

make install xen-tools
and it should work fine.

I feel with this approach I have been able to save a lot of expensive
bandwidth and download time as well.
Let me know if some one feels this is wrong.
----
Regards
Tapas

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users