Hi all, I thought I?d post this in case any one has issues similar to mine. First, my initial email to the list which I didn?t send; I?m trying to run a KVM based guest OS off of a mirrored ZFS dataset. It won?t run with errors "invalid argument..", but will run when on the root volume, or when that ZFS dataset has been removed in favor of an EXT4 volumes. Thanks in advance, PS I do see this thread which is not promising; http://www.redhat.com/archives/redhat-list/2014-July/msg00017.html Lastly, the solution; Set your cache policy on the KVM guest as write back. This allows it to run on a ZFS volume. My apology if this has been covered. I did spend some time on this so hopefully you can save time by setting the cache policy. - aurf "Janitorial Services" -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.centos.org/pipermail/centos-virt/attachments/20150115/fc9aff2e/attachment-0002.html>
On Fri, Jan 16, 2015 at 12:47 AM, aurfalien <aurfalien at gmail.com> wrote:> Hi all, > > I thought I?d post this in case any one has issues similar to mine. > > First, my initial email to the list which I didn?t send; > > I?m trying to run a KVM based guest OS off of a mirrored ZFS dataset. > > It won?t run with errors "invalid argument..", but will run when on the root > volume, or when that ZFS dataset has been removed in favor of an EXT4 > volumes. > > Thanks in advance, > > PS I do see this thread which is not promising; > > http://www.redhat.com/archives/redhat-list/2014-July/msg00017.html > > Lastly, the solution; > > Set your cache policy on the KVM guest as write back. > > This allows it to run on a ZFS volume. > > My apology if this has been covered. I did spend some time on this so > hopefully you can save time by setting the cache policy.The virt-manager / KVM angle is covered here: https://github.com/zfsonlinux/zfs/issues/224 It looks like virt-manager asks for "cached=none" by default, which causes qemu to pass the "O_DIRECT" flag when opening the file, which ZFS doesn't support. Be advised that if you use your work-around, you may be risking problems with disk corruption unless your OS understands how to use flush commands; see cached=writeback here: https://www.suse.com/documentation/sles11/book_kvm/data/sect1_1_chapter_book_kvm.html -George
On 01/15/2015 07:47 PM, aurfalien wrote:> > I?m trying to run a KVM based guest OS off of a mirrored ZFS dataset. > > It won?t run with errors "invalid argument..", but will run when on the > root volume, or when that ZFS dataset has been removed in favor of an > EXT4 volumes.KVM works fine on ZFS block devices. I haven't tried running an image file off of one (zvols are more appropriate to task). Try (approximately, from memory): zfs create pool/machine zfs create -V 2G pool/machine/boot zfs create -V 64G pool/machine/root etc. and in KVM, define the disks as physical devices: /dev/zvol/pool/machine/boot and /dev/zvol/pool/machine/root (etc.) I usually partition each 'disk' from within the VM, but you could pre-partition from the host. I like boot-part1 as ext3, for ultimate portability back to older virtualization software (and it's just '/boot', so who cares) on the off chance I have to move the VM at some point. If you're going to compress, do zfs set compress=lz4 pool/machine for better performance (need a new-ish ZoL with feature-flags for this). Setup zfs-auto-snapshot with --fast in the cron jobs for best performance. If you're using whole-disk encryption, do those on the disks _before_ they're added to the pool. I've been bitten by trying to encrypt on top of the ZFS mirror before. Put them in crypttab and you'll be all set at boot time. On newer Fedora-derived OS'es you'll need to 'systemctl enable zfs.target' to get the ordering right. -Bill -- Bill McGonigle, Owner BFC Computing, LLC http://bfccomputing.com/ Telephone: +1.855.SW.LIBRE Email, IM, VOIP: bill at bfccomputing.com VCard: http://bfccomputing.com/vcard/bill.vcf Social networks: bill_mcgonigle/bill.mcgonigle