Jelle de Jong
2016-Feb-01 21:52 UTC
[Libguestfs] virsh, virt-filesystems, guestmount, virt-install not working well with ceph rbd yet?
Hello everybody, This is a cross post to libvirt-users, libguestfs and ceph-users. I came back from FOSDEM 2016 and this was my 7th year or so and seen the awesome development around visualization going on and want to thank everybody for there contributions. I seen presentations from oVirt, OpenStack and quite a few great Redhat people, just like the last previous years. I personally been supporting Linux-HA (pacemaker) DRBD/iSCSI/KVM platform for years now and last year I started supporting Ceph RBD clusters with KVM hosts. But I keep hitting some ?pains? and I been wondering why they have not been solved and how I can best request help around this? Tools that don't seem to fully work with Ceph RBD yet: - virsh snapshot-create --quiesce --disk-only $quest - virt-filesystems ${array_volume[@]} - guestmount --ro ${array_volume[@]} -m $mount /mnt/$name-$disk - virt-install also doesn't have Ceph RBD and I currently use virsh edit to add the RBD storage disks. My request is to get these tools working as well with Ceph RBD and maybe not only integrate oVirt or OpenStack alike systems. I use several types of back-up strategies depending on the size on the data storage use. I am not sure how oVirt and alike create secure encrypted incremental off-site back-ups of large data sets but I use(d) combinations of rdiff-backup, duplicity and full guest dumps with dd, rbd export, xz. Are these issues known and important enough, roadmap, bug reports? Should I start working for Redhat as I don't have the resources myself, but I can help with some money towards a crowed fund or bounty. Maybe my software is outdated? virsh 1.2.9 ceph version 0.80.10 libguestfs-tools 1:1.28.1-1 3.16.7-ckt20-1+deb8u2 My current workarounds for full storage exports: virsh domfsfreeze $quest sleep 2 virsh domblklist $quest rbd snap create --snap snapshot $blkdevice virsh domfsthaw $quest rbd export $blkdevice at snapshot - | xz -1 | ssh -p 222 $user@$server "dd of=/$location/$blkdevice$snapshot-$daystamp.dd.disk.gz" rbd snap rm $blkdevice at snapshot Kind regards, Jelle de Jong irc: #tuxcrafter
Dominique Ramaekers
2016-Feb-02 05:07 UTC
[Libguestfs] [libvirt-users] virsh, virt-filesystems, guestmount, virt-install not working well with ceph rbd yet?
I can only contribute on one point: I only get the external snapshots working on Qemu 2.1. Not on a more recent version of Qemu... ________________________________________ Van: libvirt-users-bounces at redhat.com <libvirt-users-bounces at redhat.com> namens Jelle de Jong <jelledejong at powercraft.nl> Verzonden: maandag 1 februari 2016 22:52 Aan: libvirt-users at redhat.com; libguestfs at redhat.com; ceph-users at lists.ceph.com Onderwerp: [libvirt-users] virsh, virt-filesystems, guestmount, virt-install not working well with ceph rbd yet? Hello everybody, This is a cross post to libvirt-users, libguestfs and ceph-users. I came back from FOSDEM 2016 and this was my 7th year or so and seen the awesome development around visualization going on and want to thank everybody for there contributions. I seen presentations from oVirt, OpenStack and quite a few great Redhat people, just like the last previous years. I personally been supporting Linux-HA (pacemaker) DRBD/iSCSI/KVM platform for years now and last year I started supporting Ceph RBD clusters with KVM hosts. But I keep hitting some ?pains? and I been wondering why they have not been solved and how I can best request help around this? Tools that don't seem to fully work with Ceph RBD yet: - virsh snapshot-create --quiesce --disk-only $quest - virt-filesystems ${array_volume[@]} - guestmount --ro ${array_volume[@]} -m $mount /mnt/$name-$disk - virt-install also doesn't have Ceph RBD and I currently use virsh edit to add the RBD storage disks. My request is to get these tools working as well with Ceph RBD and maybe not only integrate oVirt or OpenStack alike systems. I use several types of back-up strategies depending on the size on the data storage use. I am not sure how oVirt and alike create secure encrypted incremental off-site back-ups of large data sets but I use(d) combinations of rdiff-backup, duplicity and full guest dumps with dd, rbd export, xz. Are these issues known and important enough, roadmap, bug reports? Should I start working for Redhat as I don't have the resources myself, but I can help with some money towards a crowed fund or bounty. Maybe my software is outdated? virsh 1.2.9 ceph version 0.80.10 libguestfs-tools 1:1.28.1-1 3.16.7-ckt20-1+deb8u2 My current workarounds for full storage exports: virsh domfsfreeze $quest sleep 2 virsh domblklist $quest rbd snap create --snap snapshot $blkdevice virsh domfsthaw $quest rbd export $blkdevice at snapshot - | xz -1 | ssh -p 222 $user@$server "dd of=/$location/$blkdevice$snapshot-$daystamp.dd.disk.gz" rbd snap rm $blkdevice at snapshot Kind regards, Jelle de Jong irc: #tuxcrafter _______________________________________________ libvirt-users mailing list libvirt-users at redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
Richard W.M. Jones
2016-Feb-02 08:48 UTC
[Libguestfs] virsh, virt-filesystems, guestmount, virt-install not working well with ceph rbd yet?
On Mon, Feb 01, 2016 at 10:52:59PM +0100, Jelle de Jong wrote: [...] Upstream libguestfs supports Ceph and has done for many years. It was disabled in RHEL up to RHEL <= 7.1, and then enabled in RHEL 7.2.> - virt-filesystems ${array_volume[@]} > - guestmount --ro ${array_volume[@]} -m $mount /mnt/$name-$diskYou'll need to provide the precise commands that you're running. You need to use a URL with the -a option, as described here: http://libguestfs.org/guestfish.1.html#adding-remote-storage> Maybe my software is outdated? > libguestfs-tools 1:1.28.1-1What precise version? RHEL 7.1 and 7.2 both had 1.28.1-1, but with different release numbers. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
Possibly Parallel Threads
- virsh, virt-filesystems, guestmount, virt-install not working well with ceph rbd yet?
- ceph rbd pool and libvirt manageability (virt-install)
- Re: ceph rbd pool and libvirt manageability (virt-install)
- Re: [ceph-users] Setting RBD cache parameters for libvirt+qemu
- Ceph RBD locking for libvirt-managed LXC (someday) live migrations