libvirt users - Mar 2015 - Re: [virt-tools-list] libvirt/virt-manager source integration with xen

Thanks Cole, unfortunately I'm a n00b in this area so your response just
raised many more questions for me :/

* what are the config file names for libxl, libvirt and virt-manager
* where are they located in the respective source trees
* are the build flags defined in those config files?
* libxl is part of xen 4.5.0 distro (xen/tools/libxl after untarring). But
no config file is in xen/tools/libxl directory
* I do see CFLAGS definitions in the various makefiles within each
directory.  Are those the build flags you are referring to?  here's an
example from libxl/Makefile:

CFLAGS += -Werror -Wno-format-zero-length -Wmissing-declarations \
        -Wno-declaration-after-statement -Wformat-nonliteral
CFLAGS += -I. -fPIC

* by distro I assume you mean binary package, and that I can install
virt-manager binary package on top of libxl/libvirt/xen that I built and
installed locally from source.  if I have compiled libxl/libvirt/xen with
virt-manager compile flags.  Does that summarize what you are
recommending?  If so then how do I know virt-manager flags?  That was a
binary.  Do I have to download virt-manager source to find those?

* libxl was part of xen source tarball, and therefore has a source tree
position within xen (xen/tools/libxl as mentioned before).  Should I place
libvirt and virt-manager in some specific source tree spot relative to
xen?  Should libvirt and virt-manager be aware of libxl source location?
Is there any makefile engineering involved?

Trying to work with your answer but I have a serious background deficit to
deal with.  For this reason I love technical 'cookbooks' that provide
all
missing details :)

Thanks,
Wayne

On Mon, Mar 2, 2015 at 12:12 PM, Cole Robinson <crobinso@redhat.com>
wrote:
> On 03/02/2015 10:55 AM, Wayne Mills wrote:
> > Hi,
> >
> > I built and installed Xen 4.5.0 from source, on top of Ubuntu 14.04.2,
> using
> > "make world" and "make install" targets.  I then
installed latest
> virt-manager
> > from pre-built packages.  After bringing up virt-manager I attempt to
> connect
> > to Xen hypervisor and got:
> >
> >      unable to connect to libvirt
> >      Failed to connect socket to /var/run/libvirt/libvirt-sock' No
such
> file
> > or directory
> >
> > I then noticed libvirt-bin isn't running.  If I try to issue
'service
> > libvirt-bin start' it gives me back a process number, but it
apparently
> dies
> > quickly because the service still shows as down.  Three log files are
> touched
> > during my start attempt:
> >
> >      -rw-r--r-- 1 root      root  39757 Feb 27 06:00
> /var/log/xenstored-access.log
> >      -rw-r--r-- 1 root      root  44908 Feb 27 06:00
> > /var/log/libvirt/libxl/libxl-driver.log
> >      -rw------- 1 root      root  46396 Feb 27 06:00
> /var/log/libvirt/libvirtd.log
> >
> > * xenstored-access.log has 22 new entries, grouped into pairs that
> increment
> > an  "Axx" identifier and go from A63 to A73.  Here is the
A63 logs:
> >
> >      [20150227T11:00:29.478Z]  A63          newconn
> >      [20150227T11:00:29.479Z]  A63          endconn
> >
> > * libxl-driver.log also has 11 log groupings that are just the same
set
> of
> > logs repeated 11 times.  Here is the first group:
> >
> >      xc: detail: sysctl operation failed -- need to rebuild the
> user-space
> > tool set?
> >      libxl: error: libxl.c:4320:libxl_get_physinfo: getting physinfo:
> > Permission denied
> >      xc: debug: hypercall buffer: total allocations:7 total releases:7
> >      xc: debug: hypercall buffer: current allocations:0 maximum
> allocations:1
> >      xc: debug: hypercall buffer: cache current size:1
> >      xc: debug: hypercall buffer: cache hits:6 misses:1 toobig:0
> >
> > * libvirtd.log also has 11 log groupings, that are just the same set
of
> logs
> > repeated 11 times.  Here is that group:
> >
> >      2015-02-27 11:00:29.479+0000: 4842: info : libvirt version: 1.2.2
> >      2015-02-27 11:00:29.479+0000: 4842: error :
> libxlDriverConfigNew:1131 :
> > Unable to configure libxl's memory management parameters
> >      2015-02-27 11:00:29.479+0000: 4842: error :
virStateInitialize:749 :
> > Initialization of LIBXL state driver failed: Unknown problem
> >      2015-02-27 11:00:29.479+0000: 4842: error :
daemonRunStateInit:920 :
> > Driver state initialization failed
> >
> > When I google for libxl_get_physinfo I see other reported errors
during
> "xl
> > info" and other "xl" commands.    I tried "xl
info" and that works for
> me and
> > does not alter the log files:
> >
> > root@<server>:~# xl info
> > host                   : server
> > release                : 3.13.0-46-generic
> > version                : #75-Ubuntu SMP Tue Feb 10 15:24:04 UTC 2015
> > machine                : x86_64
> > nr_cpus                : 8
> > max_cpu_id             : 7
> > nr_nodes               : 2
> > cores_per_socket       : 4
> > threads_per_core       : 1
> > cpu_mhz                : 2400
> > hw_caps                :
> >
bfebfbff:2c100800:00000000:00003f00:17bee3ff:00000000:00000001:00000000
> > virt_caps              : hvm hvm_directio
> > total_memory           : 98168
> > free_memory            : 128
> > sharing_freed_memory   : 0
> > sharing_used_memory    : 0
> > outstanding_claims     : 0
> > free_cpus              : 0
> > xen_major              : 4
> > xen_minor              : 5
> > xen_extra              : .0
> > xen_version            : 4.5.0
> > xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
> > hvm-3.0-x86_32p hvm-3.0-x86_64
> > xen_scheduler          : credit
> > xen_pagesize           : 4096
> > platform_params        : virt_start=0xffff800000000000
> > xen_changeset          : Mon Jan 12 11:30:05 2015 -0500 git:a8ac229
> > xen_commandline        : placeholder
> > cc_compiler            : gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2
> > cc_compile_by          : root
> > cc_compile_domain      : cisco.com <http://cisco.com>
> > cc_compile_date        : Thu Feb 19 07:26:00 EST 2015
> > xend_config_format     : 4
> > root@<server>:~#
> >
> > After consulting with xen-users mailer, I was given this response by
> "Ian":
> >
> > "Did you install libvirt from source or from packages? If the
latter
> then you
> > may have a disconnect between the packaged version and your
source-built
> Xen.
> > You'll probably need to rebuild libvirt against your Xen
libraries."
> >
> > What are the virt-related steps to take to test out Ian's idea?  I
have
> root
> > access to one machine that serves both as a host to VM's as well
as a
> build
> > environment for source builds such as this.  I already built xen 4.5.0
> and
> > installed it.  Do I now need to separately download and build libvirt,
> then
> > download and build virt-manager?  Is it important to place the virt*
code
> > within the xen source tree, or do something else so that the virt*
code
> > compiles "against" an appropriate xen environment?
> >
> > In summary, I would like some guidance on how to build libvirt, and if
> needed,
> > virt-manager, source against a specific Xen version.
> >
>
> Distro packages of virt-manager should work fine, once you get libvirt
> building.
>
> I recommend you download the ubuntu .deb config files for libxl, and
> rebuild+install libxl with the exact same build flags your distro packages
> use. They've already figured out the necessary flags to make things
work
> together with the other distro packages. Then do the same with libvirt.
>
> - Cole
>

I downloaded and untarred the deb package and grepped for 'configure',
but
none of the hits were configure commands.  Is there a better way to find it?

Wayne


On Mon, Mar 2, 2015 at 3:08 PM, Cole Robinson <crobinso@redhat.com> wrote:
> On 03/02/2015 01:39 PM, Wayne Mills wrote:
> > Thanks Cole, unfortunately I'm a n00b in this area so your
response just
> > raised many more questions for me :/
> >
> > * what are the config file names for libxl, libvirt and virt-manager
> > * where are they located in the respective source trees
> > * are the build flags defined in those config files?
> > * libxl is part of xen 4.5.0 distro (xen/tools/libxl after untarring).
> But no
> > config file is in xen/tools/libxl directory
> > * I do see CFLAGS definitions in the various makefiles within each
> directory.
> > Are those the build flags you are referring to?  here's an example
from
> > libxl/Makefile:
> >
> > CFLAGS += -Werror -Wno-format-zero-length -Wmissing-declarations \
> >         -Wno-declaration-after-statement -Wformat-nonliteral
> > CFLAGS += -I. -fPIC
> >
> > * by distro I assume you mean binary package, and that I can install
> > virt-manager binary package on top of libxl/libvirt/xen that I built
and
> > installed locally from source.  if I have compiled libxl/libvirt/xen
with
> > virt-manager compile flags.  Does that summarize what you are
> recommending?
> > If so then how do I know virt-manager flags?  That was a binary.  Do I
> have to
> > download virt-manager source to find those?
> >
>
> I meant the files that are used to generate your distro's binary
packages.
> The
> bits that determine how software is compiled before wrapping it up in a
> distribution package.
>
> For example, I googled 'libvirt ubuntu package' and got to this:
>
> https://launchpad.net/ubuntu/+source/libvirt
>
> I clicked the top link for 1.2.12-0ubuntu7:
>
> https://launchpad.net/ubuntu/+source/libvirt/1.2.12-0ubuntu7
>
> On this page is a link to the .deb packaging files:
>
>
>
https://launchpad.net/ubuntu/+archive/primary/+files/libvirt_1.2.12-0ubuntu7.debian.tar.xz
>
> Somewhere in there ./configure command line that generates the build for
> ubuntu's libvirt package. You'll want to pull that out for libvirt
and for
> whatever provides libxl on ubuntu.
>
> At least that's the process I follow on Fedora when I want to compile
some
> random software package from source.
>
> > * libxl was part of xen source tarball, and therefore has a source
tree
> > position within xen (xen/tools/libxl as mentioned before).  Should I
> place
> > libvirt and virt-manager in some specific source tree spot relative to
> xen?
> > Should libvirt and virt-manager be aware of libxl source location?  Is
> there
> > any makefile engineering involved?
> >
>
> If you copy the distribution ./configure lines, you shouldn't have to
deal
> with specifying any explicit paths.
>
> > Trying to work with your answer but I have a serious background
deficit
> to
> > deal with.  For this reason I love technical 'cookbooks' that
provide all
> > missing details :)
> >
>
> I don't know of any explicit instructions for what you want, since
these
> types
> of things vary a lot distro to distro and between software. Building
> multiple
> dependent packages from source is often a non-trivial task. And if you are
> 'make install'ing various different build configurations and things
don't
> 'just work', it can be very difficult to debug things if they go
wrong.
>
> - Cole
>
>
> > On Mon, Mar 2, 2015 at 12:12 PM, Cole Robinson <crobinso@redhat.com
> > <mailto:crobinso@redhat.com>> wrote:
> >
> >     On 03/02/2015 10:55 AM, Wayne Mills wrote:
> >     > Hi,
> >     >
> >     > I built and installed Xen 4.5.0 from source, on top of Ubuntu
> 14.04.2, using
> >     > "make world" and "make install" targets. 
I then installed latest
> >     virt-manager
> >     > from pre-built packages.  After bringing up virt-manager I
attempt
> to
> >     connect
> >     > to Xen hypervisor and got:
> >     >
> >     >      unable to connect to libvirt
> >     >      Failed to connect socket to
/var/run/libvirt/libvirt-sock' No
> such file
> >     > or directory
> >     >
> >     > I then noticed libvirt-bin isn't running.  If I try to
issue
> 'service
> >     > libvirt-bin start' it gives me back a process number, but
it
> apparently dies
> >     > quickly because the service still shows as down.  Three log
files
> are
> >     touched
> >     > during my start attempt:
> >     >
> >     >      -rw-r--r-- 1 root      root  39757 Feb 27 06:00
> >     /var/log/xenstored-access.log
> >     >      -rw-r--r-- 1 root      root  44908 Feb 27 06:00
> >     > /var/log/libvirt/libxl/libxl-driver.log
> >     >      -rw------- 1 root      root  46396 Feb 27 06:00
> >     /var/log/libvirt/libvirtd.log
> >     >
> >     > * xenstored-access.log has 22 new entries, grouped into pairs
that
> increment
> >     > an  "Axx" identifier and go from A63 to A73.  Here
is the A63 logs:
> >     >
> >     >      [20150227T11:00:29.478Z]  A63          newconn
> >     >      [20150227T11:00:29.479Z]  A63          endconn
> >     >
> >     > * libxl-driver.log also has 11 log groupings that are just
the
> same set of
> >     > logs repeated 11 times.  Here is the first group:
> >     >
> >     >      xc: detail: sysctl operation failed -- need to rebuild
the
> user-space
> >     > tool set?
> >     >      libxl: error: libxl.c:4320:libxl_get_physinfo: getting
> physinfo:
> >     > Permission denied
> >     >      xc: debug: hypercall buffer: total allocations:7 total
> releases:7
> >     >      xc: debug: hypercall buffer: current allocations:0
maximum
> >     allocations:1
> >     >      xc: debug: hypercall buffer: cache current size:1
> >     >      xc: debug: hypercall buffer: cache hits:6 misses:1
toobig:0
> >     >
> >     > * libvirtd.log also has 11 log groupings, that are just the
same
> set of logs
> >     > repeated 11 times.  Here is that group:
> >     >
> >     >      2015-02-27 11:00:29.479+0000: 4842: info : libvirt
version:
> 1.2.2
> >     >      2015-02-27 11:00:29.479+0000: 4842: error :
> libxlDriverConfigNew:1131 :
> >     > Unable to configure libxl's memory management parameters
> >     >      2015-02-27 11:00:29.479+0000: 4842: error :
> virStateInitialize:749 :
> >     > Initialization of LIBXL state driver failed: Unknown problem
> >     >      2015-02-27 11:00:29.479+0000: 4842: error :
> daemonRunStateInit:920 :
> >     > Driver state initialization failed
> >     >
> >     > When I google for libxl_get_physinfo I see other reported
errors
> during "xl
> >     > info" and other "xl" commands.    I tried
"xl info" and that works
> for
> >     me and
> >     > does not alter the log files:
> >     >
> >     > root@<server>:~# xl info
> >     > host                   : server
> >     > release                : 3.13.0-46-generic
> >     > version                : #75-Ubuntu SMP Tue Feb 10 15:24:04
UTC
> 2015
> >     > machine                : x86_64
> >     > nr_cpus                : 8
> >     > max_cpu_id             : 7
> >     > nr_nodes               : 2
> >     > cores_per_socket       : 4
> >     > threads_per_core       : 1
> >     > cpu_mhz                : 2400
> >     > hw_caps                :
> >     >
> bfebfbff:2c100800:00000000:00003f00:17bee3ff:00000000:00000001:00000000
> >     > virt_caps              : hvm hvm_directio
> >     > total_memory           : 98168
> >     > free_memory            : 128
> >     > sharing_freed_memory   : 0
> >     > sharing_used_memory    : 0
> >     > outstanding_claims     : 0
> >     > free_cpus              : 0
> >     > xen_major              : 4
> >     > xen_minor              : 5
> >     > xen_extra              : .0
> >     > xen_version            : 4.5.0
> >     > xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p
> hvm-3.0-x86_32
> >     > hvm-3.0-x86_32p hvm-3.0-x86_64
> >     > xen_scheduler          : credit
> >     > xen_pagesize           : 4096
> >     > platform_params        : virt_start=0xffff800000000000
> >     > xen_changeset          : Mon Jan 12 11:30:05 2015 -0500
git:a8ac229
> >     > xen_commandline        : placeholder
> >     > cc_compiler            : gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2
> >     > cc_compile_by          : root
> >     > cc_compile_domain      : cisco.com <http://cisco.com>
<
> http://cisco.com>
> >     > cc_compile_date        : Thu Feb 19 07:26:00 EST 2015
> >     > xend_config_format     : 4
> >     > root@<server>:~#
> >     >
> >     > After consulting with xen-users mailer, I was given this
response
> by "Ian":
> >     >
> >     > "Did you install libvirt from source or from packages?
If the
> latter then you
> >     > may have a disconnect between the packaged version and your
> source-built Xen.
> >     > You'll probably need to rebuild libvirt against your Xen
> libraries."
> >     >
> >     > What are the virt-related steps to take to test out Ian's
idea?  I
> have root
> >     > access to one machine that serves both as a host to VM's
as well
> as a build
> >     > environment for source builds such as this.  I already built
xen
> 4.5.0 and
> >     > installed it.  Do I now need to separately download and build
> libvirt, then
> >     > download and build virt-manager?  Is it important to place
the
> virt* code
> >     > within the xen source tree, or do something else so that the
virt*
> code
> >     > compiles "against" an appropriate xen environment?
> >     >
> >     > In summary, I would like some guidance on how to build
libvirt,
> and if needed,
> >     > virt-manager, source against a specific Xen version.
> >     >
> >
> >     Distro packages of virt-manager should work fine, once you get
> libvirt
> >     building.
> >
> >     I recommend you download the ubuntu .deb config files for libxl,
and
> >     rebuild+install libxl with the exact same build flags your distro
> packages
> >     use. They've already figured out the necessary flags to make
things
> work
> >     together with the other distro packages. Then do the same with
> libvirt.
> >
> >     - Cole
> >
> >
>
>