Hi all, For the last couple weeks or so, I''ve been pounding away at building a version of the XCP host tools to run on a 64-bit dom0 (not just a 64-bit kernel, but a full 64-bit tool stack). I''m starting with the basic build setup of Centos 5.5 with arch x86_64. Naturally, I''m rebuilding a large number of RPM packages from source, and I''m running into package hell as I try to track down proper 64-bit source versions of packages, fix build and compile issues, and get up to the point of being able to build the xapi-libs and xapi packages. Has anybody else attempted this? I can''t find anything about XCP with a 64-bit dom0 environment being available for download. It seems like the kind of thing somebody would have tried to get going before. Anybody have advice, help, or thoughts to offer? Cheers. -kannan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, Apr 20, 2011 at 2:08 PM, Kannan Vijayan <kvijayan@gridcentric.ca> wrote:> Hi all, > > For the last couple weeks or so, I''ve been pounding away at building a > version of the XCP host tools to run on a 64-bit dom0 (not just a > 64-bit kernel, but a full 64-bit tool stack). I''m starting with the > basic build setup of Centos 5.5 with arch x86_64. Naturally, I''m > rebuilding a large number of RPM packages from source, and I''m running > into package hell as I try to track down proper 64-bit source versions > of packages, fix build and compile issues, and get up to the point of > being able to build the xapi-libs and xapi packages.I''ve never tried it. Is there a particular reason WHY you want to do it? I can''t think of any benefit, and the downside is your dom0 will use more RAM, which means less RAM available for your domUs. -Dustin _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I''m trying to integrate live-cloning (via the xen snowflock codebase) into the XCP control path, which requires me to run a 64-bit PV kernel, and also requires 64-bit dom0 tools to talk properly with Xen. Perhaps I should re-ask this question in Xen-devel. It may be a more appropriate venue. -kannan On Thu, Apr 21, 2011 at 1:07 AM, Dustin Marquess <dmarquess@gmail.com> wrote:> On Wed, Apr 20, 2011 at 2:08 PM, Kannan Vijayan <kvijayan@gridcentric.ca> wrote: >> Hi all, >> >> For the last couple weeks or so, I''ve been pounding away at building a >> version of the XCP host tools to run on a 64-bit dom0 (not just a >> 64-bit kernel, but a full 64-bit tool stack). I''m starting with the >> basic build setup of Centos 5.5 with arch x86_64. Naturally, I''m >> rebuilding a large number of RPM packages from source, and I''m running >> into package hell as I try to track down proper 64-bit source versions >> of packages, fix build and compile issues, and get up to the point of >> being able to build the xapi-libs and xapi packages. > > I''ve never tried it. Is there a particular reason WHY you want to do > it? I can''t think of any benefit, and the downside is your dom0 will > use more RAM, which means less RAM available for your domUs. > > -Dustin >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
>>> Hi all, >>> >>> For the last couple weeks or so, I''ve been pounding away at building a >>> version of the XCP host tools to run on a 64-bit dom0 (not just a >>> 64-bit kernel, but a full 64-bit tool stack). I''m starting with the >>> basic build setup of Centos 5.5 with arch x86_64. Naturally, I''m >>> rebuilding a large number of RPM packages from source, and I''m running >>> into package hell as I try to track down proper 64-bit source versions >>> of packages, fix build and compile issues, and get up to the point of >>> being able to build the xapi-libs and xapi packages. >> >> I''ve never tried it. Is there a particular reason WHY you want to do >> it? I can''t think of any benefit, and the downside is your dom0 will >> use more RAM, which means less RAM available for your domUs. >> >> -Dustin> I''m trying to integrate live-cloning (via the xen snowflock codebase) > into the XCP control path, which requires me to run a 64-bit PV > kernel, and also requires 64-bit dom0 tools to talk properly with Xen. > > Perhaps I should re-ask this question in Xen-devel. It may be a more > appropriate venue. > > -kannan Ok, i will reply directly to xen-devel to move the discussion. I''m very interested and i''m starting to recompile. I''m trying to integrate into an existing 64bit cluster a dom0 based on xpc. What packages are fundamental to have a stable environment? Thanks, Muriel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, Apr 22, 2011 at 6:36 AM, Muriel <mucawhite@gmail.com> wrote:>> I''m trying to integrate live-cloning (via the xen snowflock codebase) >> into the XCP control path, which requires me to run a 64-bit PV >> kernel, and also requires 64-bit dom0 tools to talk properly with Xen. >> >> Perhaps I should re-ask this question in Xen-devel. It may be a more >> appropriate venue. >> >> -kannan > > > Ok, i will reply directly to xen-devel to move the discussion. > I''m very interested and i''m starting to recompile. I''m trying to integrate > into an existing 64bit cluster a dom0 based on xpc. What packages are > fundamental to have a stable environment? > > Thanks, > Muriel >I''ve been working at it from the other end, with a goal of "shortest path to working". I''m coming in with relative unfamiliarity with the XCP architecture, and started working top-down. I can boot the XCP host with a 64-bit snowflock kernel. I subsequently ran into problems with a 32-bit blktap userspace toolset having problems talking to a 64-bit dom0 kernel, so I started rebuilding that and kept moving down as needed. I got the blktap issue resolved, but currently I''m running into an issue with xc_dom_boot_mem_init: /var/log/xensource.log:[20110418T16:56:55.946Z|debug|localhost|1003 unix-RPC||cli] Xapi_cli.exception_handler: Got exception INTERNAL_ERROR: [ XenguestHelper.Xc_dom_linux_build_failure(4, " xc_dom_boot_mem_init: can''t allocate low memory for dom\\\"") ] Once again, I think this is due to a mismatch between 32-bit/64-bit addresses. The call is made through a tool called ''xenguest'' which is built alongside xapi and is used by xapi. I had to go and dig pretty deep in the rebuild tree to generate the dependencies needed to build a 64-bit xapi and xenguest. So far, these are the high-level packages I''ve had to rebuild to target a 64-bit runtime (all as part of getting to a 64-bit build of xapi and xenguest): blktap-1.0.0-566.x86_64.rpm blktap-devel-1.0.0-566.x86_64.rpm ocaml-3.11.0-unknown.x86_64.rpm ocaml-camlp4-3.11.0-unknown.x86_64.rpm ocaml-findlib-1.1.2pl1-16.x86_64.rpm ocaml-findlib-devel-1.1.2pl1-16.x86_64.rpm ocaml-getopt-20040811-unknown.x86_64.rpm ocaml-getopt-devel-20040811-unknown.x86_64.rpm ocaml-type-conv-1.6.8-unknown.x86_64.rpm ocaml-xmlm-1.0.2-unknown.x86_64.rpm ocaml-xmlm-devel-1.0.2-unknown.x86_64.rpm omake-0.9.8.5-unknown.x86_64.rpm xapi-client-devel-0.2-unknown.x86_64.rpm xapi-core-0.2-unknown.x86_64.rpm xapi-datamodel-devel-0.2-unknown.x86_64.rpm xapi-docs-0.2-unknown.x86_64.rpm xapi-libs-devel-0-unknown.x86_64.rpm xapi-libs-fe-0-unknown.x86_64.rpm xapi-libs-utils-0-unknown.x86_64.rpm xapi-squeezed-0.2-unknown.x86_64.rpm xapi-tests-0.2-unknown.x86_64.rpm xapi-www-0.2-unknown.x86_64.rpm xapi-xe-0.2-unknown.x86_64.rpm xapi-xenops-0.2-unknown.x86_64.rpm xen-blktap-3.4.2-1.0.0.700.20051.x86_64.rpm xen-devel-3.4.2-1.0.0.700.20051.x86_64.rpm xen-firmware-3.4.2-1.0.0.700.20051.x86_64.rpm xen-hypervisor-3.4.2-1.0.0.700.20051.x86_64.rpm xen-tools-3.4.2-1.0.0.700.20051.x86_64.rpm Current problem: Installing these directly on a normal XCP deployment is infeasible. It wants to pull a TON of 64-bit support packages, and quickly descends into conflict hell. So now I''m looking for ways to build an XCP install image, and hoping I can replace all the base packages with 64-bit versions and go from there. I''ve looked around for instructions on building a full install image (found the instructions for building/modifying xapi.. those were very useful earlier in the process.. but not for this). If I can''t rebuild an XCP install disk, the other tactic I have in mind is to edit an existing install ISO and switch out the existing packages for 64-bit surrogates. I haven''t tried either of these approaches yet. Does anybody know of docs, or have thoughts, on either of these subjects? I''m picking up what I need to know as I go along, so my overall picture of the system is coalescing over time, but is almost definitely not complete at this time. Just a note: from what I _have_ seen of the system, it''s very nicely designed. I like the xapi object model and schema design. Simple, but complete, clear, and understandable. A short analysis was enough to figure out how best to approach the integration of live-cloning semantics into XCP. Kudos on the good work. Cheers. -kannan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, Apr 25, 2011 at 11:39 AM, Kannan Vijayan <kvijayan@gridcentric.ca> wrote:> On Fri, Apr 22, 2011 at 6:36 AM, Muriel <mucawhite@gmail.com> wrote: >>> I''m trying to integrate live-cloning (via the xen snowflock codebase) >>> into the XCP control path, which requires me to run a 64-bit PV >>> kernel, and also requires 64-bit dom0 tools to talk properly with Xen. >>>Have you considered running the 64 bit code in a VM? The dom0 in XCP is 32 bit because of performance. It is recommended for performance reasons to use 64bit HVM and then use the PVonHVM drivers. http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers>>> Perhaps I should re-ask this question in Xen-devel. It may be a more >>> appropriate venue. >>> >>> -kannan >> >> >> Ok, i will reply directly to xen-devel to move the discussion. >> I''m very interested and i''m starting to recompile. I''m trying to integrate >> into an existing 64bit cluster a dom0 based on xpc. What packages are >> fundamental to have a stable environment? >> >> Thanks, >> Muriel >> > > I''ve been working at it from the other end, with a goal of "shortest > path to working". I''m coming in with relative unfamiliarity with the > XCP architecture, and started working top-down. > > I can boot the XCP host with a 64-bit snowflock kernel. I > subsequently ran into problems with a 32-bit blktap userspace toolset > having problems talking to a 64-bit dom0 kernel, so I started > rebuilding that and kept moving down as needed. I got the blktap > issue resolved, but currently I''m running into an issue with > xc_dom_boot_mem_init: > > /var/log/xensource.log:[20110418T16:56:55.946Z|debug|localhost|1003 > unix-RPC||cli] Xapi_cli.exception_handler: Got exception > INTERNAL_ERROR: [ XenguestHelper.Xc_dom_linux_build_failure(4, " > xc_dom_boot_mem_init: can''t allocate low memory for dom\\\"") ] > > Once again, I think this is due to a mismatch between 32-bit/64-bit > addresses. The call is made through a tool called ''xenguest'' which is > built alongside xapi and is used by xapi. I had to go and dig pretty > deep in the rebuild tree to generate the dependencies needed to build > a 64-bit xapi and xenguest. > > So far, these are the high-level packages I''ve had to rebuild to > target a 64-bit runtime (all as part of getting to a 64-bit build of > xapi and xenguest): > > blktap-1.0.0-566.x86_64.rpm > blktap-devel-1.0.0-566.x86_64.rpm > ocaml-3.11.0-unknown.x86_64.rpm > ocaml-camlp4-3.11.0-unknown.x86_64.rpm > ocaml-findlib-1.1.2pl1-16.x86_64.rpm > ocaml-findlib-devel-1.1.2pl1-16.x86_64.rpm > ocaml-getopt-20040811-unknown.x86_64.rpm > ocaml-getopt-devel-20040811-unknown.x86_64.rpm > ocaml-type-conv-1.6.8-unknown.x86_64.rpm > ocaml-xmlm-1.0.2-unknown.x86_64.rpm > ocaml-xmlm-devel-1.0.2-unknown.x86_64.rpm > omake-0.9.8.5-unknown.x86_64.rpm > xapi-client-devel-0.2-unknown.x86_64.rpm > xapi-core-0.2-unknown.x86_64.rpm > xapi-datamodel-devel-0.2-unknown.x86_64.rpm > xapi-docs-0.2-unknown.x86_64.rpm > xapi-libs-devel-0-unknown.x86_64.rpm > xapi-libs-fe-0-unknown.x86_64.rpm > xapi-libs-utils-0-unknown.x86_64.rpm > xapi-squeezed-0.2-unknown.x86_64.rpm > xapi-tests-0.2-unknown.x86_64.rpm > xapi-www-0.2-unknown.x86_64.rpm > xapi-xe-0.2-unknown.x86_64.rpm > xapi-xenops-0.2-unknown.x86_64.rpm > xen-blktap-3.4.2-1.0.0.700.20051.x86_64.rpm > xen-devel-3.4.2-1.0.0.700.20051.x86_64.rpm > xen-firmware-3.4.2-1.0.0.700.20051.x86_64.rpm > xen-hypervisor-3.4.2-1.0.0.700.20051.x86_64.rpm > xen-tools-3.4.2-1.0.0.700.20051.x86_64.rpm > > > Current problem: Installing these directly on a normal XCP deployment > is infeasible. It wants to pull a TON of 64-bit support packages, and > quickly descends into conflict hell. > > So now I''m looking for ways to build an XCP install image, and hoping > I can replace all the base packages with 64-bit versions and go from > there. I''ve looked around for instructions on building a full install > image (found the instructions for building/modifying xapi.. those were > very useful earlier in the process.. but not for this). > > If I can''t rebuild an XCP install disk, the other tactic I have in > mind is to edit an existing install ISO and switch out the existing > packages for 64-bit surrogates. > > I haven''t tried either of these approaches yet. Does anybody know of > docs, or have thoughts, on either of these subjects? I''m picking up > what I need to know as I go along, so my overall picture of the system > is coalescing over time, but is almost definitely not complete at this > time. > > Just a note: from what I _have_ seen of the system, it''s very nicely > designed. I like the xapi object model and schema design. Simple, > but complete, clear, and understandable. A short analysis was enough > to figure out how best to approach the integration of live-cloning > semantics into XCP. Kudos on the good work. > > Cheers. > -kannan > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- Todd Deshane http://www.linkedin.com/in/deshantm http://www.xen.org/products/cloudxen.html _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Actually I know at least one reason to have 64-bits dom0: speed of network/disk operations. I test a pretty fast raid60 with about 50 disks in it connected via iscsi (over 10G) and found 64-bit OS (same config, same version) have about 5% gain in speed (about 1.15GB/s against 1.1GB/s for 32-bit) And I expect some speedup for openvswitch/netback (which one is bottleneck right now...) On 21.04.2011 09:07, Dustin Marquess wrote:> On Wed, Apr 20, 2011 at 2:08 PM, Kannan Vijayan<kvijayan@gridcentric.ca> wrote: >> Hi all, >> >> For the last couple weeks or so, I''ve been pounding away at building a >> version of the XCP host tools to run on a 64-bit dom0 (not just a >> 64-bit kernel, but a full 64-bit tool stack). I''m starting with the >> basic build setup of Centos 5.5 with arch x86_64. Naturally, I''m >> rebuilding a large number of RPM packages from source, and I''m running >> into package hell as I try to track down proper 64-bit source versions >> of packages, fix build and compile issues, and get up to the point of >> being able to build the xapi-libs and xapi packages. > I''ve never tried it. Is there a particular reason WHY you want to do > it? I can''t think of any benefit, and the downside is your dom0 will > use more RAM, which means less RAM available for your domUs. > > -Dustin > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users