I am having trouble using my Dom0 local ZFS datasets in my linux PV DomU. I have installed Xen4.3 from Debian wheezy with some unstable packages from sid and also set up ZFSonLinux. Both appear to be working well but I can''t work out the best way to use the storage in a Debian PV Dom0. I tried creating the Dom0 with: xen-create-image --hostname=web1-pv --memory=8gb --vcpus=2 --dhcp --pygrub --dist=wheezy --dir=/zpool1/vm2/ I suspect the --dir=/zpool1/vm2/ is incorrect but am unsure what to use instead (perhaps need to create the image manually?) The virtual machine fails to find the images when it starts. Is it necessary to have EXT3 images stored on the ZFS pool or can the Dom0 somehow access the ZFS datasets directly? So I need a wrapper like iSCSI or NFS to present files to the DomUs? If so, which would have the least overhead and preserve the benefits of ZFS? For those that are curious, the performace of ZFS on linux looks really good, 201MB/s read, 123MB/s write on a 2 disk mirror with 3TB 7200rpm 3.5" SAS drives, no fancy ZIF or L2ARC yet, no dedup or compression.
I am having trouble using my Dom0 local ZFS datasets in my linux PV DomU. I have installed Xen4.3 from Debian wheezy with some unstable packages from sid and also set up ZFSonLinux. Both appear to be working well but I can''t work out the best way to use the storage in a Debian PV Dom0. I tried creating the Dom0 with: xen-create-image --hostname=web1-pv --memory=8gb --vcpus=2 --dhcp --pygrub --dist=wheezy --dir=/zpool1/vm2/ I suspect the --dir=/zpool1/vm2/ is incorrect but am unsure what to use instead (perhaps need to create the image manually?) The virtual machine fails to find the images when it starts. Is it necessary to have EXT3 images stored on the ZFS pool or can the Dom0 somehow access the ZFS datasets directly? So I need a wrapper like iSCSI or NFS to present files to the DomUs? If so, which would have the least overhead and preserve the benefits of ZFS? For those that are curious, the performace of ZFS on linux looks really good, 201MB/s read, 123MB/s write on a 2 disk mirror with 3TB 7200rpm 3.5" SAS drives, no fancy ZIF or L2ARC yet, no dedup or compression.
Roger Pau Monné
2013-Sep-16 08:47 UTC
Re: How to install linux PV DomU on Debian ZFS Dom0 (Xen 4.3)
On 15/09/13 07:45, Frank wrote:> I am having trouble using my Dom0 local ZFS datasets in my linux PV DomU. > > I have installed Xen4.3 from Debian wheezy with some unstable packages from sid and also set up ZFSonLinux. > Both appear to be working well but I can''t work out the best way to use the storage in a Debian PV Dom0. > I tried creating the Dom0 with: > xen-create-image --hostname=web1-pv --memory=8gb --vcpus=2 --dhcp --pygrub --dist=wheezy --dir=/zpool1/vm2/ > I suspect the --dir=/zpool1/vm2/ is incorrect but am unsure what to use instead (perhaps need to create the image manually?) > The virtual machine fails to find the images when it starts.Could you please post the config file of the guest? And the path were your image is stored.> Is it necessary to have EXT3 images stored on the ZFS pool or can the Dom0 somehow access the ZFS datasets directly? > So I need a wrapper like iSCSI or NFS to present files to the DomUs? > If so, which would have the least overhead and preserve the benefits of ZFS?IMHO you should dd the image created by xen-create-image to a zvol (a block device) in order to obtain better performance. Using raw image files hosted on ZFS is probably not going to make much of a difference from other filesystems.
On 16/09/2013, at 6:47 PM, Roger Pau Monné <roger.pau@citrix.com> wrote:> On 15/09/13 07:45, Frank wrote: >> I am having trouble using my Dom0 local ZFS datasets in my linux PV DomU. >> >> I have installed Xen4.3 from Debian wheezy with some unstable packages from sid and also set up ZFSonLinux. >> Both appear to be working well but I can''t work out the best way to use the storage in a Debian PV Dom0. >> I tried creating the Dom0 with: >> xen-create-image --hostname=web1-pv --memory=8gb --vcpus=2 --dhcp --pygrub --dist=wheezy --dir=/zpool1/vm2/ >> I suspect the --dir=/zpool1/vm2/ is incorrect but am unsure what to use instead (perhaps need to create the image manually?) >> The virtual machine fails to find the images when it starts. > > Could you please post the config file of the guest? And the path were > your image is stored.I don''t think the images were created properly as they are not visible in the file tree where they should be: /zpool1/vm2/ is empty # # Configuration file for the Xen instance web1-pv, created # by xen-tools 4.4beta1 on Thu Sep 12 22:44:29 2013. # # # Kernel + memory size # bootloader = ''/usr/lib/xen-4.3/bin/pygrub'' vcpus = ''2'' memory = ''8192'' # # Disk device(s). # root = ''/dev/xvda2 ro'' disk = [ ''file:/zpool1/vm2/domains/web1-pv/disk.img,xvda2,w'', ''file:/zpool1/vm2/domains/web1-pv/swap.img,xvda1,w'', ] # # Physical volumes # # # Hostname # name = ''web1-pv'' # # Networking # dhcp = ''dhcp'' vif = [ <REMOVED>] # # Behaviour # on_poweroff = ''destroy'' on_reboot = ''restart'' on_crash = ''restart''> >> Is it necessary to have EXT3 images stored on the ZFS pool or can the Dom0 somehow access the ZFS datasets directly? >> So I need a wrapper like iSCSI or NFS to present files to the DomUs? >> If so, which would have the least overhead and preserve the benefits of ZFS? > > IMHO you should dd the image created by xen-create-image to a zvol (a > block device) in order to obtain better performance. Using raw image > files hosted on ZFS is probably not going to make much of a difference > from other filesystems. >That sounds like a good idea, thanks for the suggestion. I have a ext4 partition where I can create the images and then I will dd them to the zpool what would be the appropriate disk = [] specification once I have done this?
Niels Dettenbach
2013-Sep-16 13:43 UTC
Re: How to install linux PV DomU on Debian ZFS Dom0 (Xen 4.3)
Am Montag, 16. September 2013, 23:32:30 schrieb Frank:> what would be the appropriate disk = [] specification once I have done > this?should be something like [ ''phy:/dev/path/to/blockdevice,xvda1,w'', ... ] if you want to use ZFS block devices with xen you have to attach it with ''phy:'' - "xvda1" means the corresponding device / partition inside the DomU. good luck, Niels. -- --- Niels Dettenbach Syndicat IT & Internet http://www.syndicat.com PGP: https://syndicat.com/pub_key.asc ---
On 16/09/2013, at 11:43 PM, Niels Dettenbach <nd@syndicat.com> wrote:> Am Montag, 16. September 2013, 23:32:30 schrieb Frank: >> what would be the appropriate disk = [] specification once I have done >> this? > > should be something like > > [ ''phy:/dev/path/to/blockdevice,xvda1,w'', > ... ] > > if you want to use ZFS block devices with xen you have to attach it with > ''phy:'' - "xvda1" means the corresponding device / partition inside the DomU. >Thanks, it''s partially working with the following procedure but I think sparse images may be causing a problem as the DomU boots and uses the ZFS Zvol but I get a problem when writing with bonnie: Writing intelligently...Can''t write block.: No such file or directory Can''t write block 442958. - maybe bonnie isn''t the right tool for testing the speed within the DomU? The procedure I followed is: 0) set options in /etc/xen-tools/xen-tools.conf dir = /home/xen/ fs = ext3 image = sparse 1) create DomU eg: xen-create-image --hostname vitrualM --memory=8gb --dhcp --vcpus 2 --pygrub --dist wheezy this creates volumes in /home/xen/domains/virtualM/ disk.img swap.img 2) create zvol, eg: zfs create -V 100G zpool1/vitrualM 3) use dd to copy over image to zvol: dd if=/home/xen/domains/vitrualM/disk.img of=/dev/zvol/zpool1/vitrualM bs=1M 4) set DomU config to point to new location of image: ''phy:/dev/zvol/zpool1/vitrualM,xvda2,w'', I have 3 further queries: - is there any point putting the swap volumes on ZFS or should I just leave them on my ext3 disk? - I believe it will be better to use ext2 as the format for the images as I figure I don''t need journalling with zfs taking care of the data integrity, is this a wise choice? - the options are set to create sparse images currently, I presume there''s no problem with this as they are no longer sparse when copied over to the ZFS zvol with dd, or could this be the cause of my bonnie error?