libvirt users - Jul 2019 - Why librbd disallow VM live migration if the disk cache mode is not none or directsync

I'm curious that why librbd sets this limitation? The rule first
appeared in librbd.git commit d57485f73ab. Theoretically, a
write-through cache is also safe for VM migration, if the cache
implementation guarantees that cache invalidation and disk write are
synchronous operations.

For example, I'm using Ceph RBD images as VM storage backend. The Ceph
librbd supports synchronous write-through cache, by setting
rbd_cache_max_dirty to zero, and setting
rbd_cache_block_writes_upfront to true, thus it would be safe for VM
migration. Is that true? Any suggestion would be appreciated. Thanks.


Ming-Hung Tsai

Sorry, the title and content are wrong. I mean libvirt, not librbd.

Ming-Hung Tsai <mingnus@gmail.com> :
>
> I'm curious that why librbd sets this limitation? The rule first
> appeared in librbd.git commit d57485f73ab. Theoretically, a
> write-through cache is also safe for VM migration, if the cache
> implementation guarantees that cache invalidation and disk write are
> synchronous operations.
>
> For example, I'm using Ceph RBD images as VM storage backend. The Ceph
> librbd supports synchronous write-through cache, by setting
> rbd_cache_max_dirty to zero, and setting
> rbd_cache_block_writes_upfront to true, thus it would be safe for VM
> migration. Is that true? Any suggestion would be appreciated. Thanks.

On 7/29/19 3:51 AM, Ming-Hung Tsai wrote:
> I'm curious that why librbd sets this limitation? The rule first
> appeared in librbd.git commit d57485f73ab. Theoretically, a
> write-through cache is also safe for VM migration, if the cache
> implementation guarantees that cache invalidation and disk write are
> synchronous operations.
> 
> For example, I'm using Ceph RBD images as VM storage backend. The Ceph
> librbd supports synchronous write-through cache, by setting
> rbd_cache_max_dirty to zero, and setting
> rbd_cache_block_writes_upfront to true, thus it would be safe for VM
> migration. Is that true? Any suggestion would be appreciated. Thanks.
The commit you refer to is very old and my hunch is that things looked 
different in 2012. Things might have changed since then and if 
write-through wasn't safe ~7 years ago, it might be safe now (with some 
tuning).

Michal

Michal Privoznik <mprivozn@redhat.com> :
>
> On 7/29/19 3:51 AM, Ming-Hung Tsai wrote:
> > I'm curious that why librbd sets this limitation? The rule first
> > appeared in librbd.git commit d57485f73ab. Theoretically, a
> > write-through cache is also safe for VM migration, if the cache
> > implementation guarantees that cache invalidation and disk write are
> > synchronous operations.
> >
> > For example, I'm using Ceph RBD images as VM storage backend. The
Ceph
> > librbd supports synchronous write-through cache, by setting
> > rbd_cache_max_dirty to zero, and setting
> > rbd_cache_block_writes_upfront to true, thus it would be safe for VM
> > migration. Is that true? Any suggestion would be appreciated. Thanks.
>
> The commit you refer to is very old and my hunch is that things looked
> different in 2012. Things might have changed since then and if
> write-through wasn't safe ~7 years ago, it might be safe now (with some
> tuning).
>
> Michal
Yes, but now the limitation is still there, in
qemuMigrationSrcIsSafe(). Should we relax the limitation?


Ming-Hung Tsai