thr3ads.net - libvirt users - Re: [libvirt-users] Create qcow2 v3 volumes via libvirt [May 2018]

If this information is useful, please help other people find it:
Share via:

Paul O'Rorke

2018-May-01 21:17 UTC

Re: [libvirt-users] Create qcow2 v3 volumes via libvirt

I have been using internal snapshots on production qcow2 images for a 
couple of years, admittedly as infrequently as possible with one 
exception and that exception has had multiple snapshots taken and 
removed using virt-manager's GUI.

I was unaware of this:> There are some technical downsides to
> internal snapshots IIUC, such as inability to free the space used by the
> internal snapshot when it is deleted,
This might explain why this VM recently kept going into a paused state 
and I had to extend the volume to get it to stay up.  This VM is used 
for testing our software in SharePoint and we make heavy use of 
snapshots.  Is there nothing I can to do recover that space?

What would be the best practice then for a VM that needs to be able to 
create and remove snapshots on a regular basis?

*Paul O'Rorke*
*Tracker Software Products (Canada) Limited *
www.tracker-software.com <http://www.tracker-software.com/>
Tel: +1 (250) 324 1621
Fax: +1 (250) 324 1623

<http://www.tracker-software.com/>

Support:
http://www.tracker-software.com/support
Download latest Releases
http://www.tracker-software.com/downloads/




On 2018-05-01 01:11 PM, Gionatan Danti wrote:> Il 01-05-2018 10:56 Daniel P. Berrangé ha scritto:
>> qcow2 is widely used in production at large scale in general. Just not
>> with internal snapshots - almost everything uses external snapshots,
>> aka backing file chains.
>>
>> The QEMU community still tends to discourage use of internal snapshots.
>> There are not even any QMP monitor commands to use them - you are
forced
>> to use the legacy HMP interface to QEMU for mgmt. All of the workaround
>> providing interesting block storage mgmt is focused on external 
>> snapshots
>> (aka the backing_file option). There are some technical downsides to
>> internal snapshots IIUC, such as inability to free the space used by
the
>> internal snapshot when it is deleted, loading/saving snapshots blocks
>> execution of the guest OS, and probably more I've forgotten about.
>>
>> The only nice thing about internal snapshots is simplicity of mgmt, and
>> that is a very nice thing indeed, which is why virt-manager has code
>> to support that - it was much easier to add that code for external
>> snapshots. Just a shame about all the downsides :-(
>
> So internal snapshots remain something very useful for lab/tests, but 
> are not recommended for regular use in production environment, right?
>
> Thanks.
>

--------------A0EDD616F7AC6502641C0A0F
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-signature">I have been using internal
snapshots on
      production qcow2 images for a couple of years, admittedly as
      infrequently as possible with one exception and that exception has
      had multiple snapshots taken and removed using virt-manager's
GUI.<br>
      <br>
      I was unaware of this:<br>
      <blockquote type="cite">There are some technical downsides
to
        <br>
        internal snapshots IIUC, such as inability to free the space
        used by the
        <br>
        internal snapshot when it is deleted,</blockquote>
      <br>
      This might explain why this VM recently kept going into a paused
      state and I had to extend the volume to get it to stay up.  This
      VM is used for testing our software in SharePoint and we make
      heavy use of snapshots.  Is there nothing I can to do recover that
      space?  <br>
      <br>
      What would be the best practice then for a VM that needs to be
      able to create and remove snapshots on a regular basis?<br>
      <br>
      <meta http-equiv="content-type" content="text/html;
charset=utf-8">
      <title></title>
      <p><b>Paul O'Rorke</b><br>
        <b>Tracker Software Products (Canada) Limited </b><br>
        <a
href="http://www.tracker-software.com/">www.tracker-software.com</a><br>
        Tel:
        +1 (250) 324 1621<br>
        Fax: +1 (250) 324 1623<br>
        <br>
        <a href="http://www.tracker-software.com/"> <img
src="https://www.tracker-software.com/fckfiles/image/images/resellers/logo/TrackerSofwareProducts_Logo_330x100.png"
            name="image.png" height="100"
border="0" align="bottom"
            width="330">
        </a>
        <br>
        <br>
        Support:
        <br>
        <a
href="http://www.tracker-software.com/support">http://www.tracker-software.com/support
        </a><br>
        Download latest Releases
        <br>
        <a
href="http://www.tracker-software.com/downloads/">http://www.tracker-software.com/downloads/</a></p>
      <p><br>
        <br>
      </p>
      <p style="margin-bottom: 0cm; line-height:
100%"><br>
      </p>
    </div>
    <div class="moz-cite-prefix">On 2018-05-01 01:11 PM,
Gionatan Danti
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:d166c5e3340dcfb685734e13127c4132@assyoma.it">Il
      01-05-2018 10:56 Daniel P. Berrangé ha scritto:
      <br>
      <blockquote type="cite">qcow2 is widely used in production
at
        large scale in general. Just not
        <br>
        with internal snapshots - almost everything uses external
        snapshots,
        <br>
        aka backing file chains.
        <br>
        <br>
        The QEMU community still tends to discourage use of internal
        snapshots.
        <br>
        There are not even any QMP monitor commands to use them - you
        are forced
        <br>
        to use the legacy HMP interface to QEMU for mgmt. All of the
        workaround
        <br>
        providing interesting block storage mgmt is focused on external
        snapshots
        <br>
        (aka the backing_file option). There are some technical
        downsides to
        <br>
        internal snapshots IIUC, such as inability to free the space
        used by the
        <br>
        internal snapshot when it is deleted, loading/saving snapshots
        blocks
        <br>
        execution of the guest OS, and probably more I've forgotten
        about.
        <br>
        <br>
        The only nice thing about internal snapshots is simplicity of
        mgmt, and
        <br>
        that is a very nice thing indeed, which is why virt-manager has
        code
        <br>
        to support that - it was much easier to add that code for
        external
        <br>
        snapshots. Just a shame about all the downsides :-(
        <br>
      </blockquote>
      <br>
      So internal snapshots remain something very useful for lab/tests,
      but are not recommended for regular use in production environment,
      right?
      <br>
      <br>
      Thanks.
      <br>
      <br>
    </blockquote>
    <br>
  </body>
</html>

--------------A0EDD616F7AC6502641C0A0F--

Eric Blake

2018-May-01 21:31 UTC

head link

Re: [libvirt-users] Create qcow2 v3 volumes via libvirt

On 05/01/2018 04:17 PM, Paul O'Rorke wrote:> I have been using internal snapshots on production qcow2 images for a 
> couple of years, admittedly as infrequently as possible with one 
> exception and that exception has had multiple snapshots taken and 
> removed using virt-manager's GUI.
> 
> I was unaware of this:
>> There are some technical downsides to
>> internal snapshots IIUC, such as inability to free the space used by
the
>> internal snapshot when it is deleted,
This is not an insurmountable difficulty, just one that no one has spent 
time coding up.
> 
> This might explain why this VM recently kept going into a paused state 
> and I had to extend the volume to get it to stay up.  This VM is used 
> for testing our software in SharePoint and we make heavy use of 
> snapshots.  Is there nothing I can to do recover that space?
If you have no internal snapshots, you can do a 'qemu-img convert' to 
copy just the portion of the image that is actively in use; the copy 
will use less disk space than the original because it got rid of the 
now-unused space.  'virt-sparsify' from libguestfs takes this one step 
further, by also removing unused space within the guest filesystem itself.

In fact, even if you do have internal snapshots, there is probably a 
sequence of 'qemu-img convert' invocations that can ultimately convert 
all of your internal snapshots into an external chain of snapshots; but 
I don't have a ready formula off-hand to point to (experiment on an 
image you don't care about, before doing it on your production image).
> What would be the best practice then for a VM that needs to be able to 
> create and remove snapshots on a regular basis?
In a managed environment, external snapshots probably have the most 
support for creating and later merging portions of a chain of snapshots, 
although we could still improve libvirt to make this feel like more of a 
first class citizen.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Paul O'Rorke

2018-May-01 21:53 UTC

head link

Re: [libvirt-users] Create qcow2 v3 volumes via libvirt

Thanks for the suggestions Eric, great stuff and most appreciated.

I will have a play in my sandbox...

*Paul O'Rorke*
*Tracker Software Products (Canada) Limited *
www.tracker-software.com <http://www.tracker-software.com/>
Tel: +1 (250) 324 1621
Fax: +1 (250) 324 1623

<http://www.tracker-software.com/>

Support:
http://www.tracker-software.com/support
Download latest Releases
http://www.tracker-software.com/downloads/




On 2018-05-01 02:31 PM, Eric Blake wrote:> On 05/01/2018 04:17 PM, Paul O'Rorke wrote:
>> I have been using internal snapshots on production qcow2 images for a 
>> couple of years, admittedly as infrequently as possible with one 
>> exception and that exception has had multiple snapshots taken and 
>> removed using virt-manager's GUI.
>>
>> I was unaware of this:
>>> There are some technical downsides to
>>> internal snapshots IIUC, such as inability to free the space used
by
>>> the
>>> internal snapshot when it is deleted,
>
> This is not an insurmountable difficulty, just one that no one has 
> spent time coding up.
>
>>
>> This might explain why this VM recently kept going into a paused 
>> state and I had to extend the volume to get it to stay up.  This VM 
>> is used for testing our software in SharePoint and we make heavy use 
>> of snapshots.  Is there nothing I can to do recover that space?
>
> If you have no internal snapshots, you can do a 'qemu-img convert'
to
> copy just the portion of the image that is actively in use; the copy 
> will use less disk space than the original because it got rid of the 
> now-unused space.  'virt-sparsify' from libguestfs takes this one
step
> further, by also removing unused space within the guest filesystem 
> itself.
>
> In fact, even if you do have internal snapshots, there is probably a 
> sequence of 'qemu-img convert' invocations that can ultimately
convert
> all of your internal snapshots into an external chain of snapshots; 
> but I don't have a ready formula off-hand to point to (experiment on 
> an image you don't care about, before doing it on your production
image).
>
>> What would be the best practice then for a VM that needs to be able 
>> to create and remove snapshots on a regular basis?
>
> In a managed environment, external snapshots probably have the most 
> support for creating and later merging portions of a chain of 
> snapshots, although we could still improve libvirt to make this feel 
> like more of a first class citizen.
>

--------------BC922824B7CD1D01FB0F49CF
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Thanks for the suggestions Eric, great stuff and most
      appreciated.<br>
    </p>
    <p>I will have a play in my sandbox...</p>
    <div class="moz-signature">
      <meta http-equiv="content-type" content="text/html;
charset=utf-8">
      <title></title>
      <p><b>Paul O'Rorke</b><br>
        <b>Tracker Software Products (Canada) Limited </b><br>
        <a
href="http://www.tracker-software.com/">www.tracker-software.com</a><br>
        Tel:
        +1 (250) 324 1621<br>
        Fax: +1 (250) 324 1623<br>
        <br>
        <a href="http://www.tracker-software.com/"> <img
src="https://www.tracker-software.com/fckfiles/image/images/resellers/logo/TrackerSofwareProducts_Logo_330x100.png"
            name="image.png" border="0"
height="100" align="bottom"
            width="330">
        </a>
        <br>
        <br>
        Support:
        <br>
        <a
href="http://www.tracker-software.com/support">http://www.tracker-software.com/support
        </a><br>
        Download latest Releases
        <br>
        <a
href="http://www.tracker-software.com/downloads/">http://www.tracker-software.com/downloads/</a></p>
      <p><br>
        <br>
      </p>
      <p style="margin-bottom: 0cm; line-height:
100%"><br>
      </p>
    </div>
    <div class="moz-cite-prefix">On 2018-05-01 02:31 PM, Eric
Blake
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:aa233c98-b041-d916-3f36-0ec4f6aa7741@redhat.com">On
      05/01/2018 04:17 PM, Paul O'Rorke wrote:
      <br>
      <blockquote type="cite">I have been using internal
snapshots on
        production qcow2 images for a couple of years, admittedly as
        infrequently as possible with one exception and that exception
        has had multiple snapshots taken and removed using
        virt-manager's GUI.
        <br>
        <br>
        I was unaware of this:
        <br>
        <blockquote type="cite">There are some technical
downsides to
          <br>
          internal snapshots IIUC, such as inability to free the space
          used by the
          <br>
          internal snapshot when it is deleted,
          <br>
        </blockquote>
      </blockquote>
      <br>
      This is not an insurmountable difficulty, just one that no one has
      spent time coding up.
      <br>
      <br>
      <blockquote type="cite">
        <br>
        This might explain why this VM recently kept going into a paused
        state and I had to extend the volume to get it to stay up.  This
        VM is used for testing our software in SharePoint and we make
        heavy use of snapshots.  Is there nothing I can to do recover
        that space?
        <br>
      </blockquote>
      <br>
      If you have no internal snapshots, you can do a 'qemu-img convert'
      to copy just the portion of the image that is actively in use; the
      copy will use less disk space than the original because it got rid
      of the now-unused space.  'virt-sparsify' from libguestfs takes
      this one step further, by also removing unused space within the
      guest filesystem itself.
      <br>
      <br>
      In fact, even if you do have internal snapshots, there is probably
      a sequence of 'qemu-img convert' invocations that can ultimately
      convert all of your internal snapshots into an external chain of
      snapshots; but I don't have a ready formula off-hand to point to
      (experiment on an image you don't care about, before doing it on
      your production image).
      <br>
      <br>
      <blockquote type="cite">What would be the best practice
then for a
        VM that needs to be able to create and remove snapshots on a
        regular basis?
        <br>
      </blockquote>
      <br>
      In a managed environment, external snapshots probably have the
      most support for creating and later merging portions of a chain of
      snapshots, although we could still improve libvirt to make this
      feel like more of a first class citizen.
      <br>
      <br>
    </blockquote>
    <br>
  </body>
</html>

--------------BC922824B7CD1D01FB0F49CF--

Paul O'Rorke

2018-May-01 21:54 UTC

head link

Re: [libvirt-users] Create qcow2 v3 volumes via libvirt

Thanks for the suggestions Eric,

great stuff and most appreciated.

I must play in my sandbox...

*Paul O'Rorke*
*Tracker Software Products (Canada) Limited *
www.tracker-software.com <http://www.tracker-software.com/>
Tel: +1 (250) 324 1621
Fax: +1 (250) 324 1623

<http://www.tracker-software.com/>

Support:
http://www.tracker-software.com/support
Download latest Releases
http://www.tracker-software.com/downloads/




On 2018-05-01 02:31 PM, Eric Blake wrote:> On 05/01/2018 04:17 PM, Paul O'Rorke wrote:
>> I have been using internal snapshots on production qcow2 images for a 
>> couple of years, admittedly as infrequently as possible with one 
>> exception and that exception has had multiple snapshots taken and 
>> removed using virt-manager's GUI.
>>
>> I was unaware of this:
>>> There are some technical downsides to
>>> internal snapshots IIUC, such as inability to free the space used
by
>>> the
>>> internal snapshot when it is deleted,
>
> This is not an insurmountable difficulty, just one that no one has 
> spent time coding up.
>
>>
>> This might explain why this VM recently kept going into a paused 
>> state and I had to extend the volume to get it to stay up.  This VM 
>> is used for testing our software in SharePoint and we make heavy use 
>> of snapshots.  Is there nothing I can to do recover that space?
>
> If you have no internal snapshots, you can do a 'qemu-img convert'
to
> copy just the portion of the image that is actively in use; the copy 
> will use less disk space than the original because it got rid of the 
> now-unused space.  'virt-sparsify' from libguestfs takes this one
step
> further, by also removing unused space within the guest filesystem 
> itself.
>
> In fact, even if you do have internal snapshots, there is probably a 
> sequence of 'qemu-img convert' invocations that can ultimately
convert
> all of your internal snapshots into an external chain of snapshots; 
> but I don't have a ready formula off-hand to point to (experiment on 
> an image you don't care about, before doing it on your production
image).
>
>> What would be the best practice then for a VM that needs to be able 
>> to create and remove snapshots on a regular basis?
>
> In a managed environment, external snapshots probably have the most 
> support for creating and later merging portions of a chain of 
> snapshots, although we could still improve libvirt to make this feel 
> like more of a first class citizen.
>

--------------25527470BD155BBE7A6BF54A
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Thanks for the suggestions Eric, <br>
    </p>
    <p>great stuff and most appreciated.<br>
    </p>
    <p>I must play in my sandbox...</p>
    <div class="moz-signature">
      <meta http-equiv="content-type" content="text/html;
charset=utf-8">
      <title></title>
      <p><b>Paul O'Rorke</b><br>
        <b>Tracker Software Products (Canada) Limited </b><br>
        <a
href="http://www.tracker-software.com/">www.tracker-software.com</a><br>
        Tel:
        +1 (250) 324 1621<br>
        Fax: +1 (250) 324 1623<br>
        <br>
        <a href="http://www.tracker-software.com/"> <img
src="https://www.tracker-software.com/fckfiles/image/images/resellers/logo/TrackerSofwareProducts_Logo_330x100.png"
            name="image.png" border="0"
height="100" align="bottom"
            width="330">
        </a>
        <br>
        <br>
        Support:
        <br>
        <a
href="http://www.tracker-software.com/support">http://www.tracker-software.com/support
        </a><br>
        Download latest Releases
        <br>
        <a
href="http://www.tracker-software.com/downloads/">http://www.tracker-software.com/downloads/</a></p>
      <p><br>
        <br>
      </p>
      <p style="margin-bottom: 0cm; line-height:
100%"><br>
      </p>
    </div>
    <div class="moz-cite-prefix">On 2018-05-01 02:31 PM, Eric
Blake
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:aa233c98-b041-d916-3f36-0ec4f6aa7741@redhat.com">On
      05/01/2018 04:17 PM, Paul O'Rorke wrote:
      <br>
      <blockquote type="cite">I have been using internal
snapshots on
        production qcow2 images for a couple of years, admittedly as
        infrequently as possible with one exception and that exception
        has had multiple snapshots taken and removed using
        virt-manager's GUI.
        <br>
        <br>
        I was unaware of this:
        <br>
        <blockquote type="cite">There are some technical
downsides to
          <br>
          internal snapshots IIUC, such as inability to free the space
          used by the
          <br>
          internal snapshot when it is deleted,
          <br>
        </blockquote>
      </blockquote>
      <br>
      This is not an insurmountable difficulty, just one that no one has
      spent time coding up.
      <br>
      <br>
      <blockquote type="cite">
        <br>
        This might explain why this VM recently kept going into a paused
        state and I had to extend the volume to get it to stay up.  This
        VM is used for testing our software in SharePoint and we make
        heavy use of snapshots.  Is there nothing I can to do recover
        that space?
        <br>
      </blockquote>
      <br>
      If you have no internal snapshots, you can do a 'qemu-img convert'
      to copy just the portion of the image that is actively in use; the
      copy will use less disk space than the original because it got rid
      of the now-unused space.  'virt-sparsify' from libguestfs takes
      this one step further, by also removing unused space within the
      guest filesystem itself.
      <br>
      <br>
      In fact, even if you do have internal snapshots, there is probably
      a sequence of 'qemu-img convert' invocations that can ultimately
      convert all of your internal snapshots into an external chain of
      snapshots; but I don't have a ready formula off-hand to point to
      (experiment on an image you don't care about, before doing it on
      your production image).
      <br>
      <br>
      <blockquote type="cite">What would be the best practice
then for a
        VM that needs to be able to create and remove snapshots on a
        regular basis?
        <br>
      </blockquote>
      <br>
      In a managed environment, external snapshots probably have the
      most support for creating and later merging portions of a chain of
      snapshots, although we could still improve libvirt to make this
      feel like more of a first class citizen.
      <br>
      <br>
    </blockquote>
    <br>
  </body>
</html>

--------------25527470BD155BBE7A6BF54A--

Paul O'Rorke

2018-May-22 18:53 UTC

head link

Re: [libvirt-users] Create qcow2 v3 volumes via libvirt

Hi Eric and list,

I had another production VM start pausing itself.  This one had been 
running for more than 4 years on a 60G LVM volume.  It has had the 
occasional snapshot during that time though all have been "removed" 
using the virt-manager gui so I used qemu-img as you suggested.

# qemu-img convert /dev/trk-kvm-02-vg/rt44 -O qcow2 /mnt/scratch/rt44.qcow2

dd'd the qcow2 image back on to the LV after testing it boots OK 
directly from the image and it is in production again.

The VM itself reports ample space available:

$ df -h
Filesystem                       Size  Used Avail Use% Mounted on
udev                             3.9G     0  3.9G   0% /dev
tmpfs                            789M  8.8M  780M   2% /run
/dev/mapper/RT--vg-root           51G   21G   28G  42% /
tmpfs                            3.9G     0  3.9G   0% /dev/shm
tmpfs                            5.0M     0  5.0M   0% /run/lock
tmpfs                            3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/vda1                        472M  155M  293M  35% /boot
192.168.0.16:/volume1/fileLevel  8.1T  2.5T  5.6T  31% /mnt/nfs/fileLevel
tmpfs                            789M     0  789M   0% /run/user/1000

I would prefer to not get caught out again with this machine pausing, 
how can I determine how much space is being used up by 'deleted' 
internal snapshots?  Do you have any suggested reading on this?

If I extend the LVM volume but not the guest file system will snapshots 
be "at the end" of the LV and "outside" the guest file
system?

If I were to expand the guest's ext4 file system I would want to do it 
unmounted and from a live CD but I'm having a heck of a time getting my 
live distro to use the virtio disk drivers.  Any advice there?

sincerely

*Paul O'Rorke*
*Tracker Software Products (Canada) Limited *
www.tracker-software.com <http://www.tracker-software.com/>
Tel: +1 (250) 324 1621
Fax: +1 (250) 324 1623

<http://www.tracker-software.com/>

Support:
http://www.tracker-software.com/support
Download latest Releases
http://www.tracker-software.com/downloads/




On 2018-05-01 02:31 PM, Eric Blake wrote:> On 05/01/2018 04:17 PM, Paul O'Rorke wrote:
>> I have been using internal snapshots on production qcow2 images for a 
>> couple of years, admittedly as infrequently as possible with one 
>> exception and that exception has had multiple snapshots taken and 
>> removed using virt-manager's GUI.
>>
>> I was unaware of this:
>>> There are some technical downsides to
>>> internal snapshots IIUC, such as inability to free the space used
by
>>> the
>>> internal snapshot when it is deleted,
>
> This is not an insurmountable difficulty, just one that no one has 
> spent time coding up.
>
>>
>> This might explain why this VM recently kept going into a paused 
>> state and I had to extend the volume to get it to stay up.  This VM 
>> is used for testing our software in SharePoint and we make heavy use 
>> of snapshots.  Is there nothing I can to do recover that space?
>
> If you have no internal snapshots, you can do a 'qemu-img convert'
to
> copy just the portion of the image that is actively in use; the copy 
> will use less disk space than the original because it got rid of the 
> now-unused space.  'virt-sparsify' from libguestfs takes this one
step
> further, by also removing unused space within the guest filesystem 
> itself.
>
> In fact, even if you do have internal snapshots, there is probably a 
> sequence of 'qemu-img convert' invocations that can ultimately
convert
> all of your internal snapshots into an external chain of snapshots; 
> but I don't have a ready formula off-hand to point to (experiment on 
> an image you don't care about, before doing it on your production
image).
>
>> What would be the best practice then for a VM that needs to be able 
>> to create and remove snapshots on a regular basis?
>
> In a managed environment, external snapshots probably have the most 
> support for creating and later merging portions of a chain of 
> snapshots, although we could still improve libvirt to make this feel 
> like more of a first class citizen.
>

--------------7CE7EC5BD8DDB43601136727
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Hi Eric and list,</p>
    <p>I had another production VM start pausing itself.  This one had
      been running for more than 4 years on a 60G LVM volume.  It has
      had the occasional snapshot during that time though all have been
      "removed" using the virt-manager gui so I used qemu-img as you
      suggested.<br>
    </p>
    <p><tt># qemu-img convert /dev/trk-kvm-02-vg/rt44 -O qcow2
        /mnt/scratch/rt44.qcow2</tt><br>
    </p>
    <p>dd'd the qcow2 image back on to the LV after testing it boots
OK
      directly from the image and it is in production again. <br>
    </p>
    <p>The VM itself reports ample space available:</p>
    <p> <tt>$ df -h</tt><tt><br>
      </tt><tt>Filesystem                       Size  Used Avail
Use%
        Mounted on</tt><tt><br>
      </tt><tt>udev                             3.9G     0  3.9G  
0%
        /dev</tt><tt><br>
      </tt><tt>tmpfs                            789M  8.8M  780M  
2%
        /run</tt><tt><br>
      </tt><tt>/dev/mapper/RT--vg-root           51G   21G   28G 
42% /</tt><tt><br>
      </tt><tt>tmpfs                            3.9G     0  3.9G  
0%
        /dev/shm</tt><tt><br>
      </tt><tt>tmpfs                            5.0M     0  5.0M  
0%
        /run/lock</tt><tt><br>
      </tt><tt>tmpfs                            3.9G     0  3.9G  
0%
        /sys/fs/cgroup</tt><tt><br>
      </tt><tt>/dev/vda1                        472M  155M  293M 
35%
        /boot</tt><tt><br>
      </tt><tt>192.168.0.16:/volume1/fileLevel  8.1T  2.5T  5.6T 
31%
        /mnt/nfs/fileLevel</tt><tt><br>
      </tt><tt>tmpfs                            789M     0  789M  
0%
        /run/user/1000</tt><br>
    </p>
    <p>I would prefer to not get caught out again with this machine
      pausing, how can I determine how much space is being used up by
      'deleted' internal snapshots?  Do you have any suggested reading
      on this?</p>
    <p>If I extend the LVM volume but not the guest file system will
      snapshots be "at the end" of the LV and "outside" the
guest file
      system?</p>
    <p>If I were to expand the guest's ext4 file system I would want
to
      do it unmounted and from a live CD but I'm having a heck of a time
      getting my live distro to use the virtio disk drivers.  Any advice
      there?</p>
    <p>sincerely<br>
    </p>
    <div class="moz-signature">
      <meta http-equiv="content-type" content="text/html;
charset=utf-8">
      <title></title>
      <p><b>Paul O'Rorke</b><br>
        <b>Tracker Software Products (Canada) Limited </b><br>
        <a
href="http://www.tracker-software.com/">www.tracker-software.com</a><br>
        Tel:
        +1 (250) 324 1621<br>
        Fax: +1 (250) 324 1623<br>
        <br>
        <a href="http://www.tracker-software.com/"> <img
src="https://www.tracker-software.com/fckfiles/image/images/resellers/logo/TrackerSofwareProducts_Logo_330x100.png"
            name="image.png" height="60"
border="0" align="bottom"
            width="198">
        </a>
        <br>
        <br>
        Support:
        <br>
        <a
href="http://www.tracker-software.com/support">http://www.tracker-software.com/support
        </a><br>
        Download latest Releases
        <br>
        <a
href="http://www.tracker-software.com/downloads/">http://www.tracker-software.com/downloads/</a></p>
      <p><br>
        <br>
      </p>
      <p style="margin-bottom: 0cm; line-height:
100%"><br>
      </p>
    </div>
    <div class="moz-cite-prefix">On 2018-05-01 02:31 PM, Eric
Blake
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:aa233c98-b041-d916-3f36-0ec4f6aa7741@redhat.com">On
      05/01/2018 04:17 PM, Paul O'Rorke wrote:
      <br>
      <blockquote type="cite">I have been using internal
snapshots on
        production qcow2 images for a couple of years, admittedly as
        infrequently as possible with one exception and that exception
        has had multiple snapshots taken and removed using
        virt-manager's GUI.
        <br>
        <br>
        I was unaware of this:
        <br>
        <blockquote type="cite">There are some technical
downsides to
          <br>
          internal snapshots IIUC, such as inability to free the space
          used by the
          <br>
          internal snapshot when it is deleted,
          <br>
        </blockquote>
      </blockquote>
      <br>
      This is not an insurmountable difficulty, just one that no one has
      spent time coding up.
      <br>
      <br>
      <blockquote type="cite">
        <br>
        This might explain why this VM recently kept going into a paused
        state and I had to extend the volume to get it to stay up.  This
        VM is used for testing our software in SharePoint and we make
        heavy use of snapshots.  Is there nothing I can to do recover
        that space?
        <br>
      </blockquote>
      <br>
      If you have no internal snapshots, you can do a 'qemu-img convert'
      to copy just the portion of the image that is actively in use; the
      copy will use less disk space than the original because it got rid
      of the now-unused space.  'virt-sparsify' from libguestfs takes
      this one step further, by also removing unused space within the
      guest filesystem itself.
      <br>
      <br>
      In fact, even if you do have internal snapshots, there is probably
      a sequence of 'qemu-img convert' invocations that can ultimately
      convert all of your internal snapshots into an external chain of
      snapshots; but I don't have a ready formula off-hand to point to
      (experiment on an image you don't care about, before doing it on
      your production image).
      <br>
      <br>
      <blockquote type="cite">What would be the best practice
then for a
        VM that needs to be able to create and remove snapshots on a
        regular basis?
        <br>
      </blockquote>
      <br>
      In a managed environment, external snapshots probably have the
      most support for creating and later merging portions of a chain of
      snapshots, although we could still improve libvirt to make this
      feel like more of a first class citizen.
      <br>
      <br>
    </blockquote>
    <br>
  </body>
</html>

--------------7CE7EC5BD8DDB43601136727--

Seemingly Similar Threads

Search for more seemingly similar threads

libvirt users - May 2018 - Re: Create qcow2 v3 volumes via libvirt

Re: [libvirt-users] Create qcow2 v3 volumes via libvirt

Re: [libvirt-users] Create qcow2 v3 volumes via libvirt

Re: [libvirt-users] Create qcow2 v3 volumes via libvirt

Re: [libvirt-users] Create qcow2 v3 volumes via libvirt

Re: [libvirt-users] Create qcow2 v3 volumes via libvirt

Seemingly Similar Threads