libvirt users - May 2018 - Re: Create qcow2 v3 volumes via libvirt

On Mon, Apr 30, 2018 at 08:42:56PM +0200, Gionatan Danti
wrote:
> Another question: how reliable are qcow2 ver2/3 files nowadays? Are you
> using them in production environments?
qcow2 is widely used in production at large scale in general. Just not
with internal snapshots - almost everything uses external snapshots,
aka backing file chains.
> At the moment, I am using RAW files and filesystem-level snapshot to manage
> versioning; however, as virt-manager has direct support for managing qcow2
> internal snapshots, it would be easier to deploy qcow2 disks.
> 
> What strikes me is that, if thing have not changed, Red Hat support policy
> was to *not* support internal snapshots. So, are they reliable enough for
> production VMs?
The QEMU community still tends to discourage use of internal snapshots.
There are not even any QMP monitor commands to use them - you are forced
to use the legacy HMP interface to QEMU for mgmt. All of the workaround
providing interesting block storage mgmt is focused on external snapshots
(aka the backing_file option). There are some technical downsides to
internal snapshots IIUC, such as inability to free the space used by the
internal snapshot when it is deleted, loading/saving snapshots blocks
execution of the guest OS, and probably more I've forgotten about.

The only nice thing about internal snapshots is simplicity of mgmt, and
that is a very nice thing indeed, which is why virt-manager has code
to support that - it was much easier to add that code for external
snapshots. Just a shame about all the downsides :-(

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Il 01-05-2018 10:56 Daniel P. Berrangé ha scritto:
> qcow2 is widely used in production at large scale in general. Just not
> with internal snapshots - almost everything uses external snapshots,
> aka backing file chains.
> 
> The QEMU community still tends to discourage use of internal snapshots.
> There are not even any QMP monitor commands to use them - you are 
> forced
> to use the legacy HMP interface to QEMU for mgmt. All of the workaround
> providing interesting block storage mgmt is focused on external 
> snapshots
> (aka the backing_file option). There are some technical downsides to
> internal snapshots IIUC, such as inability to free the space used by 
> the
> internal snapshot when it is deleted, loading/saving snapshots blocks
> execution of the guest OS, and probably more I've forgotten about.
> 
> The only nice thing about internal snapshots is simplicity of mgmt, and
> that is a very nice thing indeed, which is why virt-manager has code
> to support that - it was much easier to add that code for external
> snapshots. Just a shame about all the downsides :-(
So internal snapshots remain something very useful for lab/tests, but 
are not recommended for regular use in production environment, right?

Thanks.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

On 05/01/2018 03:11 PM, Gionatan Danti wrote:
> Il 01-05-2018 10:56 Daniel P. Berrangé ha scritto:
>> qcow2 is widely used in production at large scale in general. Just not
>> with internal snapshots - almost everything uses external snapshots,
>> aka backing file chains.
>>
>> The QEMU community still tends to discourage use of internal snapshots.
>> There are not even any QMP monitor commands to use them - you are
forced
>> to use the legacy HMP interface to QEMU for mgmt. All of the workaround
>> providing interesting block storage mgmt is focused on external
snapshots
>> (aka the backing_file option). There are some technical downsides to
>> internal snapshots IIUC, such as inability to free the space used by
the
>> internal snapshot when it is deleted, loading/saving snapshots blocks
>> execution of the guest OS, and probably more I've forgotten about.
>>
>> The only nice thing about internal snapshots is simplicity of mgmt, and
>> that is a very nice thing indeed, which is why virt-manager has code
>> to support that - it was much easier to add that code for external
>> snapshots. Just a shame about all the downsides :-(
> 
> So internal snapshots remain something very useful for lab/tests, but 
> are not recommended for regular use in production environment, right?
That's fairly accurate.  Also, mixing internal and external snapshots at 
the same time is likely to trigger some known data-loss problems, if you 
are not extremely careful, so for now, just pick one or the other and 
stick to it (I hope to someday enhance qemu to refuse operations that 
would risk data loss, or to perform a slower version of the operation 
with the same result instead of its current fast but wrong 
implementations, when dealing with mixed-internal/external backing chains).

https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg00865.html

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

I have been using internal snapshots on production qcow2 images for a 
couple of years, admittedly as infrequently as possible with one 
exception and that exception has had multiple snapshots taken and 
removed using virt-manager's GUI.

I was unaware of this:
> There are some technical downsides to
> internal snapshots IIUC, such as inability to free the space used by the
> internal snapshot when it is deleted,
This might explain why this VM recently kept going into a paused state 
and I had to extend the volume to get it to stay up.  This VM is used 
for testing our software in SharePoint and we make heavy use of 
snapshots.  Is there nothing I can to do recover that space?

What would be the best practice then for a VM that needs to be able to 
create and remove snapshots on a regular basis?

*Paul O'Rorke*
*Tracker Software Products (Canada) Limited *
www.tracker-software.com <http://www.tracker-software.com/>
Tel: +1 (250) 324 1621
Fax: +1 (250) 324 1623

<http://www.tracker-software.com/>

Support:
http://www.tracker-software.com/support
Download latest Releases
http://www.tracker-software.com/downloads/




On 2018-05-01 01:11 PM, Gionatan Danti wrote:
> Il 01-05-2018 10:56 Daniel P. Berrangé ha scritto:
>> qcow2 is widely used in production at large scale in general. Just not
>> with internal snapshots - almost everything uses external snapshots,
>> aka backing file chains.
>>
>> The QEMU community still tends to discourage use of internal snapshots.
>> There are not even any QMP monitor commands to use them - you are
forced
>> to use the legacy HMP interface to QEMU for mgmt. All of the workaround
>> providing interesting block storage mgmt is focused on external 
>> snapshots
>> (aka the backing_file option). There are some technical downsides to
>> internal snapshots IIUC, such as inability to free the space used by
the
>> internal snapshot when it is deleted, loading/saving snapshots blocks
>> execution of the guest OS, and probably more I've forgotten about.
>>
>> The only nice thing about internal snapshots is simplicity of mgmt, and
>> that is a very nice thing indeed, which is why virt-manager has code
>> to support that - it was much easier to add that code for external
>> snapshots. Just a shame about all the downsides :-(
>
> So internal snapshots remain something very useful for lab/tests, but 
> are not recommended for regular use in production environment, right?
>
> Thanks.
>

--------------A0EDD616F7AC6502641C0A0F
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-signature">I have been using internal
snapshots on
      production qcow2 images for a couple of years, admittedly as
      infrequently as possible with one exception and that exception has
      had multiple snapshots taken and removed using virt-manager's
GUI.<br>
      <br>
      I was unaware of this:<br>
      <blockquote type="cite">There are some technical downsides
to
        <br>
        internal snapshots IIUC, such as inability to free the space
        used by the
        <br>
        internal snapshot when it is deleted,</blockquote>
      <br>
      This might explain why this VM recently kept going into a paused
      state and I had to extend the volume to get it to stay up.  This
      VM is used for testing our software in SharePoint and we make
      heavy use of snapshots.  Is there nothing I can to do recover that
      space?  <br>
      <br>
      What would be the best practice then for a VM that needs to be
      able to create and remove snapshots on a regular basis?<br>
      <br>
      <meta http-equiv="content-type" content="text/html;
charset=utf-8">
      <title></title>
      <p><b>Paul O'Rorke</b><br>
        <b>Tracker Software Products (Canada) Limited </b><br>
        <a
href="http://www.tracker-software.com/">www.tracker-software.com</a><br>
        Tel:
        +1 (250) 324 1621<br>
        Fax: +1 (250) 324 1623<br>
        <br>
        <a href="http://www.tracker-software.com/"> <img
src="https://www.tracker-software.com/fckfiles/image/images/resellers/logo/TrackerSofwareProducts_Logo_330x100.png"
            name="image.png" height="100"
border="0" align="bottom"
            width="330">
        </a>
        <br>
        <br>
        Support:
        <br>
        <a
href="http://www.tracker-software.com/support">http://www.tracker-software.com/support
        </a><br>
        Download latest Releases
        <br>
        <a
href="http://www.tracker-software.com/downloads/">http://www.tracker-software.com/downloads/</a></p>
      <p><br>
        <br>
      </p>
      <p style="margin-bottom: 0cm; line-height:
100%"><br>
      </p>
    </div>
    <div class="moz-cite-prefix">On 2018-05-01 01:11 PM,
Gionatan Danti
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:d166c5e3340dcfb685734e13127c4132@assyoma.it">Il
      01-05-2018 10:56 Daniel P. Berrangé ha scritto:
      <br>
      <blockquote type="cite">qcow2 is widely used in production
at
        large scale in general. Just not
        <br>
        with internal snapshots - almost everything uses external
        snapshots,
        <br>
        aka backing file chains.
        <br>
        <br>
        The QEMU community still tends to discourage use of internal
        snapshots.
        <br>
        There are not even any QMP monitor commands to use them - you
        are forced
        <br>
        to use the legacy HMP interface to QEMU for mgmt. All of the
        workaround
        <br>
        providing interesting block storage mgmt is focused on external
        snapshots
        <br>
        (aka the backing_file option). There are some technical
        downsides to
        <br>
        internal snapshots IIUC, such as inability to free the space
        used by the
        <br>
        internal snapshot when it is deleted, loading/saving snapshots
        blocks
        <br>
        execution of the guest OS, and probably more I've forgotten
        about.
        <br>
        <br>
        The only nice thing about internal snapshots is simplicity of
        mgmt, and
        <br>
        that is a very nice thing indeed, which is why virt-manager has
        code
        <br>
        to support that - it was much easier to add that code for
        external
        <br>
        snapshots. Just a shame about all the downsides :-(
        <br>
      </blockquote>
      <br>
      So internal snapshots remain something very useful for lab/tests,
      but are not recommended for regular use in production environment,
      right?
      <br>
      <br>
      Thanks.
      <br>
      <br>
    </blockquote>
    <br>
  </body>
</html>

--------------A0EDD616F7AC6502641C0A0F--