Hi, I''m grass-green new to Xen. The only virtualization I''ve used is VMware (most recently server) on Linux, Windows and Mac, and Parallels on Mac. I''ve used VMware for years as a testing mechanism. I''m not a VMware expert, but have used it in all the ways I intend to use Xen. I''ve been reading through the Xen documentation and have some questions that don''t seem to be clear. I''m not afraid of the documentation. I haven''t waded through it all, but I have not yet found the answers to some burning questions and can''t concentrate on the topic at hand because of it. Some things just seem to be missing, and I don''t know if it''s because I just haven''t read that far or if that entire issue is handled by Dom0. If something I''m asking about is best described by the docs, I would love to see a link to that section rather than make you guys type in a description. First, my hardware: I have an Intel i7 920 on an Asus P6T with 12G RAM. The processor supports VT-x but not the vpro version. I''m still not clear on the differences between these. The drives are SATA, 4xWD750gx10000rpm (WD7500AADS) configured as Linux kernel-based RAID with LVM2 on top. Mostly RAID 1 in 2 volume groups, two drives per group. The /boot drive is 4 drives RAID 1, grub boot loader. I also have a removable SATA drive with 1.5 Tb as a backup drive. Currently all this space is mostly empty, but by the time I get done that probably won''t be the case. I have an nVidia GT200 video card with dual monitors, if that matters. I also have some other hardware (mostly sound and video hardware) but that''s not pertinent at this point I think. A full hardware profile is included at the end of my message in case it''s pertinent to one of the questions. My intent: This is my home workstation. I currently run Gentoo on it. I also want to work from it, and I''m a software developer. Here are the VMs I want: 1 Gentoo stable. This is probably going to be where I live most. It will have video and sound and will probably use the most CPU of everything. 2 Gentoo ~arch. This may be a chroot of stable, or maybe a full image. This is primarily for testing and experimentation with unreleased software. 3 Windows 2003 converted from a VMware disk. This has Microsoft SQL Server on it, and everything is using legitimate developer licenses. This is used as a database and as a web browser for testing, and has a couple other end-user tools on it, but very little human interaction. 4 Some random flavor of operating system. I sometimes try out a distro just to see what''s going on. 5 Possibly one or two other stand-alone server images, usually a minimal Linux. 6 I may also have Ubuntu on it. For some reason I''m fixated by this distro. Stable and Windows 2003 will probably start at boot. Everything else will probably be a manual start-as-needed arrangement. My questions: 1. Board issues. http://wiki.xensource.com/xenwiki/VTdHowTo shows a line which causes me concern: ASUS P6T Deluxe (Intel X58 chipset) requires (currently non-public) BIOS update to correct DMAR-table issue My board is P6T, not deluxe. It has the X58 chip set, the BIOS is set to allow VT-d, but I''m not sure if this board has the broken BIOS like the Deluxe, or if it has a good BIOS, or if it is missing the feature which requires the fix altogether. Can this board run VT-d in a proper way? Do I need to get a different board in order to make this work well? 2. Dom0 strategy: I''m a bit confused as to the entire role of dom0. Is this supposed to be just a minimal command-line distro for drivers and Xen admin, or is it a full-blown distro which also has Xen admin? Do I want flexibility or stability? Rolling release or conservative non-rolling release? Admin-only or do I live here? 3. Partitioning: I''m currently using RAID per linux-only schemes. Does Xen have its own requirements and abilities for that, or is that entirely handled by Dom0? Do I assign logical volumes directly to the DomUs with the proper partitioning scheme or do I store everything on XFS in a big file and let the DomU partition that file? Does it make sense still to segment the system out to different partition types for performance, or what? What''s the strategy? Thanks. -------------------------------------------------------------------- Hardware details: /proc/cpuinfo: processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz stepping : 5 cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 5344.67 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: --------------------- /proc/meminfo MemTotal: 12321128 kB MemFree: 9958988 kB Buffers: 316368 kB Cached: 604180 kB SwapCached: 0 kB Active: 1268860 kB Inactive: 552456 kB Active(anon): 907640 kB Inactive(anon): 16 kB Active(file): 361220 kB Inactive(file): 552440 kB Unevictable: 16 kB Mlocked: 16 kB SwapTotal: 31487368 kB SwapFree: 31487368 kB Dirty: 40 kB Writeback: 4 kB AnonPages: 900820 kB Mapped: 96912 kB Shmem: 6888 kB Slab: 428268 kB SReclaimable: 381632 kB SUnreclaim: 46636 kB KernelStack: 3032 kB PageTables: 18288 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 37647932 kB Committed_AS: 1260788 kB VmallocTotal: 34359738367 kB VmallocUsed: 356972 kB VmallocChunk: 34359358460 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 18944 kB DirectMap2M: 12554240 kB --------------------- lspci 00:00.0 Host bridge: Intel Corporation X58 I/O Hub to ESI Port (rev 12) 00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 12) 00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 12) 00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 12) 00:14.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub System Management Registers (rev 12) 00:14.1 PIC: Intel Corporation 5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 12) 00:14.2 PIC: Intel Corporation 5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 12) 00:14.3 PIC: Intel Corporation 5520/5500/X58 I/O Hub Throttle Registers (rev 12) 00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5 00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6 00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2 00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller 00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 1 00:1c.2 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 3 00:1c.3 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 4 00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 5 00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1 00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3 00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) 00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller 00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller 00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller 00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller 01:00.0 Multimedia video controller: Conexant Systems, Inc. Hauppauge Inc. HDPVR-1250 model 1196 (rev 04) 02:00.0 VGA compatible controller: nVidia Corporation GT200 [GeForce GTX 260] (rev a1) 04:00.0 SATA controller: JMicron Technology Corp. 20360/20363 Serial ATA Controller (rev 03) 04:00.1 IDE interface: JMicron Technology Corp. 20360/20363 Serial ATA Controller (rev 03) 05:00.0 FireWire (IEEE 1394): VIA Technologies, Inc. Device 3403 06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02) 08:00.0 Multimedia audio controller: Creative Labs SB X-Fi ----------------------- lsusb Bus 007 Device 002: ID 051d:0002 American Power Conversion Uninterruptible Power Supply Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 006 Device 003: ID 04b3:310b IBM Corp. Red Wheel Mouse Bus 006 Device 002: ID 058f:9410 Alcor Micro Corp. Keyboard Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 002 Device 006: ID 05e3:0608 Genesys Logic, Inc. USB-2.0 4-Port HUB Bus 002 Device 004: ID 05e3:0608 Genesys Logic, Inc. USB-2.0 4-Port HUB Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ken wrote:>2. Dom0 strategy: >I''m a bit confused as to the entire role of dom0. Is this supposed to >be just a minimal command-line distro for drivers and Xen admin, or is >it a full-blown distro which also has Xen admin? Do I want flexibility >or stability? Rolling release or conservative non-rolling release? >Admin-only or do I live here?Dom0 is to Xen a bit like user root is to Unix style systems. Xen loads first and sits on top of the hardware, but Xen itself doesn''t really have any way to interact with it. It loads Dom0 as the first guest OS (and yes, even Dom0 is a virtual machine I believe) - and gives it special privileges, such as being able to communicate with the hypervisor and control it. Dom0 also gets direct access to all hardware by default. It is customary that Dom0 is a "light" install - just containing what you need to run the machine. It doesn''t have to be, you can load up all your GUI shells, user apps etc - but it''s custom to keep Dom0 light because of it''s privileged role and the fact that if you compromise Dom0 then all the guests are compromised. So if you have a machine that is your desktop, then it''s quite OK to run Xen on it, use Dom0 as your "desktop machine" and fire up some other guests as required. You just have to accept that if you compromise your desktop machine, then the others are compromised too. it would be a good way to get started and experiment - just not good for running production services.>3. Partitioning: >I''m currently using RAID per linux-only schemes. Does Xen have its own >requirements and abilities for that, or is that entirely handled by >Dom0? > >Do I assign logical volumes directly to the DomUs with the proper >partitioning scheme or do I store everything on XFS in a big file and >let the DomU partition that file? > >Does it make sense still to segment the system out to different >partition types for performance, or what? What''s the strategy?Again, this is largely a matter of personal preference. In terms of performance, then unless you take steps to segregate stuff (eg keeping different bits of data on different drives), then the I/O from all your guests shares the same disk I/O bottlenecks. In some ways it could be said to be worse since you will typically have the virtual disks for different machines spread across the disks and thus ensure lots of head seeks. Xen does not handle any file systems on it''s own, whatever containers you use are transparent to the hypervisor. What type of container is again a matter of preference. At one extreme, you can build a big filesystem for Dom0, and use file to store the virtual disks for the guests. Personally I use block devices and LVM - I create on logical volume per filesystem in each DomU, and I don''t partition them in the DomUs. This has the advantage that each filesystem can be mounted in Dom0 without any hassles (ie you can just "mount /dev/vg0/guest1root /mnt") and have access to the file on the guests disks (but you must shut down the guest first). If you create a virtual disk and partition it inside the guest then filesystems can still be mounted elsewhere, but there''s an extra step or two involved. One LV per filesystem also makes resizing filesystems a doddle - shutdown guest, shrink filesystem if reducing size, resize LV, expand filesystem. There is talk of it being possible to resize (expand) the LV and trigger some signal to make the increased size visible to a running Dom0, and then live-expand the filesystem. As mentioned above, many of the same performance issues arise, but with some added complications because you are no longer considering one "machine". If you do have a heavy I/O application, then you may still want to take the usual steps of keeping that data on it''s own set of spindles and so on - Xen will let you do that, it really doesn''t care what you do. Dunno about the other questions. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Simon, That helps a lot. Much of what you said matches what my impression was, especially about Dom0. I think in that case I will make a minimal distro for that, and keep it simple. At least until I decide a DomU might not be adequate in some way, if ever. With regards to partitioning, I think then what you''re saying is that Dom0 decides what partitions are visible to the DomU, and whether those partitions are files on a Dom0 partition or if they''re partitions on the same "layer" so to speak. So just guessing, I''m thinking that Dom0 would be the only thing that needs to understand RAID/LVM and the DomUs could get by thinking it was a plain file system. Another thing: I have this existing Gentoo, that I''m typing on. Is there some way to preserve it, or should I just figure on starting over? Is there some way to wedge the hypervisor under it, in place? Is that a bad idea even if possible, or is it no big deal? Thanks a lot, you''ve really helped. -- Ken. On Tue, 2010-08-10 at 08:02 +0100, Simon Hobson wrote:> Ken wrote: > > >2. Dom0 strategy: > >I''m a bit confused as to the entire role of dom0. Is this supposed to > >be just a minimal command-line distro for drivers and Xen admin, or is > >it a full-blown distro which also has Xen admin? Do I want flexibility > >or stability? Rolling release or conservative non-rolling release? > >Admin-only or do I live here? > > Dom0 is to Xen a bit like user root is to Unix style systems. Xen > loads first and sits on top of the hardware, but Xen itself doesn''t > really have any way to interact with it. It loads Dom0 as the first > guest OS (and yes, even Dom0 is a virtual machine I believe) - and > gives it special privileges, such as being able to communicate with > the hypervisor and control it. Dom0 also gets direct access to all > hardware by default. > > It is customary that Dom0 is a "light" install - just containing what > you need to run the machine. It doesn''t have to be, you can load up > all your GUI shells, user apps etc - but it''s custom to keep Dom0 > light because of it''s privileged role and the fact that if you > compromise Dom0 then all the guests are compromised. > So if you have a machine that is your desktop, then it''s quite OK to > run Xen on it, use Dom0 as your "desktop machine" and fire up some > other guests as required. You just have to accept that if you > compromise your desktop machine, then the others are compromised too. > it would be a good way to get started and experiment - just not good > for running production services. > > >3. Partitioning: > >I''m currently using RAID per linux-only schemes. Does Xen have its own > >requirements and abilities for that, or is that entirely handled by > >Dom0? > > > >Do I assign logical volumes directly to the DomUs with the proper > >partitioning scheme or do I store everything on XFS in a big file and > >let the DomU partition that file? > > > >Does it make sense still to segment the system out to different > >partition types for performance, or what? What''s the strategy? > > Again, this is largely a matter of personal preference. In terms of > performance, then unless you take steps to segregate stuff (eg > keeping different bits of data on different drives), then the I/O > from all your guests shares the same disk I/O bottlenecks. In some > ways it could be said to be worse since you will typically have the > virtual disks for different machines spread across the disks and thus > ensure lots of head seeks. > > Xen does not handle any file systems on it''s own, whatever containers > you use are transparent to the hypervisor. What type of container is > again a matter of preference. > At one extreme, you can build a big filesystem for Dom0, and use file > to store the virtual disks for the guests. Personally I use block > devices and LVM - I create on logical volume per filesystem in each > DomU, and I don''t partition them in the DomUs. This has the advantage > that each filesystem can be mounted in Dom0 without any hassles (ie > you can just "mount /dev/vg0/guest1root /mnt") and have access to the > file on the guests disks (but you must shut down the guest first). > If you create a virtual disk and partition it inside the guest then > filesystems can still be mounted elsewhere, but there''s an extra step > or two involved. > One LV per filesystem also makes resizing filesystems a doddle - > shutdown guest, shrink filesystem if reducing size, resize LV, expand > filesystem. There is talk of it being possible to resize (expand) the > LV and trigger some signal to make the increased size visible to a > running Dom0, and then live-expand the filesystem. > > As mentioned above, many of the same performance issues arise, but > with some added complications because you are no longer considering > one "machine". If you do have a heavy I/O application, then you may > still want to take the usual steps of keeping that data on it''s own > set of spindles and so on - Xen will let you do that, it really > doesn''t care what you do. > > > > Dunno about the other questions. > > -- > Simon Hobson > > Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed > author Gladys Hobson. Novels - poetry - short stories - ideal as > Christmas stocking fillers. Some available as e-books. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ken wrote:>With regards to partitioning, I think then what you''re saying is that >Dom0 decides what partitions are visible to the DomU, and whether those >partitions are files on a Dom0 partition or if they''re partitions on the >same "layer" so to speak.Correct.>So just guessing, I''m thinking that Dom0 would be the only thing that >needs to understand RAID/LVM and the DomUs could get by thinking it was >a plain file system.Yes.>Another thing: I have this existing Gentoo, that I''m typing on. Is >there some way to preserve it, or should I just figure on starting over? >Is there some way to wedge the hypervisor under it, in place? Is that a >bad idea even if possible, or is it no big deal?In general, all you need to do is install a kernel package with Xen Dom0 support, install Xen, and configure Xen+Compatible kernel as a boot option. You should then have the option of booting : + Your existing kernel + The new kernel + Xen plus the new kernel Another option, but slightly harder as you need to re-arrange partitions a bit ... Install a kernel with Xen DomU support, put the whole disk/partitions to one side, build a new boot setup with a fresh Xen+Dom0, boot your existing Gentoo as a DomU. A convenient way of doing this is to install a new disk as the primary disk, and then you just configure the DomU to use the real partitions (rather than LVs). As long as you don''t mess up the disk, you then have the option of putting things back and booting up as you are now. I like options with a safety net ;) -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> My questions: > > 1. Board issues. > http://wiki.xensource.com/xenwiki/VTdHowTo shows a line which causes me > concern: > ASUS P6T Deluxe (Intel X58 chipset) requires (currently non-public) BIOS > update to correct DMAR-table issue > > My board is P6T, not deluxe. It has the X58 chip set, the BIOS is set > to allow VT-d, but I''m not sure if this board has the broken BIOS like > the Deluxe, or if it has a good BIOS, or if it is missing the feature > which requires the fix altogether. Can this board run VT-d in a proper > way? Do I need to get a different board in order to make this work > well? >I can''t answer this in relation to your board, but you don''t look like you need VT-d support for your requirements. VT-d allows you to pass off devices to your HVM domU''s, such as a dedicated eth or SAS card or the like. Regards, Mark _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Tue, 2010-08-10 at 17:35 +0100, Mark Adams wrote:> > My questions: > > > > 1. Board issues. > > http://wiki.xensource.com/xenwiki/VTdHowTo shows a line which causes me > > concern: > > ASUS P6T Deluxe (Intel X58 chipset) requires (currently non-public) BIOS > > update to correct DMAR-table issue > > > > My board is P6T, not deluxe. It has the X58 chip set, the BIOS is set > > to allow VT-d, but I''m not sure if this board has the broken BIOS like > > the Deluxe, or if it has a good BIOS, or if it is missing the feature > > which requires the fix altogether. Can this board run VT-d in a proper > > way? Do I need to get a different board in order to make this work > > well? > > > > I can''t answer this in relation to your board, but you don''t look like > you need VT-d support for your requirements. VT-d allows you to pass off > devices to your HVM domU''s, such as a dedicated eth or SAS card > or the like. > > Regards, > MarkMaybe I need to go back to the beginning of the docs and start over. I hate that. I always lose patience and try to skip forward to the good parts and then miss something important. Thanks. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users