John Nielsen
2013-Jun-07 21:41 UTC
[libvirt-users] Setting RBD cache parameters for libvirt+qemu
I am running some qemu-kvm virtual machines via libvirt using Ceph RBD as the back-end storage. Today I was testing an update to libvirt-1.0.6 on one of my hosts and discovered that it includes this change: [libvirt] [PATCH] Forbid use of ':' in RBD pool names ...People are known to be abusing the lack of escaping in current libvirt to pass arbitrary args to QEMU. I am one of those people--I have been storing colon-separated arguments in my disk source 'name' attribute to control the RBD cache settings on a per-volume basis. Under libvirt-1.0.6 such VM's refuse to start: error: Failed to start domain foo error: unsupported configuration: ':' not allowed in RBD source volume name 'mypool/foo:rbd_cache_size=268435456:rbd_cache_max_dirty=134217728:rbd_cache_target_dirty=33554432:rbd_cache_max_dirty_age=5' So my question is how do I control the cache settings now? Thanks, JN
Josh Durgin
2013-Jun-07 23:01 UTC
Re: [libvirt-users] [ceph-users] Setting RBD cache parameters for libvirt+qemu
On 06/07/2013 02:41 PM, John Nielsen wrote:> I am running some qemu-kvm virtual machines via libvirt using Ceph RBD as the back-end storage. Today I was testing an update to libvirt-1.0.6 on one of my hosts and discovered that it includes this change: > [libvirt] [PATCH] Forbid use of ':' in RBD pool names > ...People are known to be abusing the lack of escaping in current libvirt to pass arbitrary args to QEMU. > > I am one of those people--I have been storing colon-separated arguments in my disk source 'name' attribute to control the RBD cache settings on a per-volume basis. Under libvirt-1.0.6 such VM's refuse to start: > error: Failed to start domain foo > error: unsupported configuration: ':' not allowed in RBD source volume name 'mypool/foo:rbd_cache_size=268435456:rbd_cache_max_dirty=134217728:rbd_cache_target_dirty=33554432:rbd_cache_max_dirty_age=5' > > So my question is how do I control the cache settings now?You can set the cache mode with the cache attribute on the <driver> element inside a <disk> [1]. This works for rbd with QEMU 1.2 or later. QEMU will try to read /etc/ceph/ceph.conf by default as well, so you can put more specific settings in a [global] or [client.foo] section there. Note that if you enable writeback caching here, you need to include cache='writeback' in the xml as well, or qemu will not send flushes. Josh [1] http://libvirt.org/formatdomain.html#elementsDisks
John Nielsen
2013-Jun-07 23:18 UTC
Re: [libvirt-users] [ceph-users] Setting RBD cache parameters for libvirt+qemu
On Jun 7, 2013, at 5:01 PM, Josh Durgin <josh.durgin@inktank.com> wrote:> On 06/07/2013 02:41 PM, John Nielsen wrote: >> I am running some qemu-kvm virtual machines via libvirt using Ceph RBD as the back-end storage. Today I was testing an update to libvirt-1.0.6 on one of my hosts and discovered that it includes this change: >> [libvirt] [PATCH] Forbid use of ':' in RBD pool names >> ...People are known to be abusing the lack of escaping in current libvirt to pass arbitrary args to QEMU. >> >> I am one of those people--I have been storing colon-separated arguments in my disk source 'name' attribute to control the RBD cache settings on a per-volume basis. Under libvirt-1.0.6 such VM's refuse to start: >> error: Failed to start domain foo >> error: unsupported configuration: ':' not allowed in RBD source volume name 'mypool/foo:rbd_cache_size=268435456:rbd_cache_max_dirty=134217728:rbd_cache_target_dirty=33554432:rbd_cache_max_dirty_age=5' >> >> So my question is how do I control the cache settings now? > > You can set the cache mode with the cache attribute on the <driver> > element inside a <disk> [1]. This works for rbd with QEMU 1.2 or later. > > QEMU will try to read /etc/ceph/ceph.conf by default as well, so you > can put more specific settings in a [global] or [client.foo] section > there. Note that if you enable writeback caching here, you need to > include cache='writeback' in the xml as well, or qemu will not send > flushes.Thanks for the quick reply. Is it safe to assume that the "foo" in "client.foo" should match the "auth username" attribute from the XML? If I want different cache settings for different VM's or volumes on the same host is there a way to specify that any more? Can I specify multiple profiles in ceph.conf or point different qemu instances to different ceph.conf files? JN
Maybe Matching Threads
- Re: [ceph-users] Setting RBD cache parameters for libvirt+qemu
- Re: [ceph-users] Setting RBD cache parameters for libvirt+qemu
- Re: [ceph-users] Setting RBD cache parameters for libvirt+qemu
- Create RBD Format 2 disk images with qemu-image
- Re: RBD images locking