Fedora xen - May 2008 - [ANNOUNCE] virt-df 2.1.0 - a 'df' tool for virtual guests

I''m pleased to announce the most recent release of virt-df (2.1.0).

Virt-df is ''df'' for virtual guests. Run the program on the
host / dom0
to display disk space used and available on all partitions on all
guests.  You don''t need to run any sort of program/agent within the
guest.

  Home page:       http://et.redhat.com/~rjones/virt-df/
  Source/binaries: http://et.redhat.com/~rjones/virt-df/files/
  Developer repository:
      http://hg.et.redhat.com/virt/applications/virt-df--devel

This version supports most common filesystems and partitioning
schemes, including:
 - Linux ext2/3
 - DOS FAT32
 - Windows NTFS
 - Linux LVM2 (volume groups and logical volumes)
 - Primary and extended disk partitions
 - Linux swap
 - Linux suspend partition

You can run it in your host / dom0 to display guest filesystems:

# virt-df -c qemu:///system -h
Filesystem                             Size       Used  Available Type
rhel51x32kvm:hda1                  96.8 MiB   14.6 MiB   82.2 MiB Linux ext2/3
rhel51x32kvm:VolGroup00/LogVol00    6.4 GiB    3.6 GiB    2.8 GiB Linux ext2/3
rhel51x32kvm:VolGroup00/LogVol01  992.0 MiB                       Linux swap

You can also run it on general disk images, or disk devices:

# virt-df -t /dev/sda
Filesystem                        1K-blocks       Used  Available Type
/dev/sda:hda1                      25599996   11309448   14290552 Windows NTFS
/dev/sda:hda2                        992016      93772     898244 Linux ext2/3
/dev/sda:F9VG/F9Root               23316072    7818164   15497908 Linux ext2/3
/dev/sda:F9VG/F9Swap                1015808                       Linux swap
/dev/sda:RHEL51VG/RHEL51Root       22382184    7796640   14585544 Linux ext2/3
/dev/sda:RHEL51VG/RHEL51Swap        2031616                       Linux swap
/dev/sda:VolGroup/FAT32Test          916736          4     914676 DOS/Windows

You can write the output to a CSV file (use --csv option) in order to
import the data easily into spreadsheets and databases.

Included also is an experimental command line tool called
''diskzip''
which intelligently compresses disk images by leaving out the bits
which aren''t actually used in the filesystems / partitions / volume
groups contained within.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v

On Tue, May 13, 2008 at 05:42:38PM +0100, Richard W.M. Jones
wrote:
> I''m pleased to announce the most recent release of virt-df
(2.1.0).
> 
> Virt-df is ''df'' for virtual guests. Run the program on
the host / dom0
> to display disk space used and available on all partitions on all
> guests.  You don''t need to run any sort of program/agent within
the
> guest.
> 
>   Home page:       http://et.redhat.com/~rjones/virt-df/
>   Source/binaries: http://et.redhat.com/~rjones/virt-df/files/
>   Developer repository:
>       http://hg.et.redhat.com/virt/applications/virt-df--devel
> 
> This version supports most common filesystems and partitioning
> schemes, including:
>  - Linux ext2/3
>  - DOS FAT32
>  - Windows NTFS
>  - Linux LVM2 (volume groups and logical volumes)
>  - Primary and extended disk partitions
>  - Linux swap
>  - Linux suspend partition
I''m assuming this only works for raw file & block devices ? Are you
planning
to support the funky QCow / VMDK formats too ?   The other thing that could 
be annoying is that Fedora 9 support for encrypting all volumes - might need
to prompt for a decryption key for that.
> Included also is an experimental command line tool called
''diskzip''
> which intelligently compresses disk images by leaving out the bits
> which aren''t actually used in the filesystems / partitions /
volume
> groups contained within.
That''s pretty neat. Which file systems does that work for ?   VMWare
have
a funky  guest tool which tries to let you get to a similar point. It works
by basically openning a file inside the guest VM and filling it with zeros
until the entire disk is full. THeir backend can then detect and discard
all the sectors with zeros. Understanding the filesystem metdata is a much
nicer way todo this :-)

Regards,
Daniel
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

On Thu, May 15, 2008 at 02:54:53PM +0100, Daniel P. Berrange
wrote:
> > This version supports most common filesystems and partitioning
> > schemes, including:
> >  - Linux ext2/3
> >  - DOS FAT32
> >  - Windows NTFS
> >  - Linux LVM2 (volume groups and logical volumes)
> >  - Primary and extended disk partitions
> >  - Linux swap
> >  - Linux suspend partition
> 
> I''m assuming this only works for raw file & block devices ?
Are you planning
> to support the funky QCow / VMDK formats too ?
Yes, interesting point.  Since I always use flat files or straight
partitions for my guests I admit I hadn''t given this much thought :-)
However support for these formats is just a matter of decoding enough
of the structure to enable the same mappings to be made in the
''virt-df'' library, same as for LVM2 or indeed MBR partitions
now.

I''ll take a look at it.  IIRC there are several different undocumented
variations on the QCow format?
> The other thing that could be annoying is that Fedora 9 support for
> encrypting all volumes - might need to prompt for a decryption key
> for that.
Yes -- any encrypted volumes aren''t going to work at the moment, and
couldn''t work unless there was a way to access the passphrase.
> > Included also is an experimental command line tool called
''diskzip''
> > which intelligently compresses disk images by leaving out the bits
> > which aren''t actually used in the filesystems / partitions /
volume
> > groups contained within.
> 
> That''s pretty neat. Which file systems does that work for ?  
VMWare have
> a funky  guest tool which tries to let you get to a similar point. It works
> by basically openning a file inside the guest VM and filling it with zeros
> until the entire disk is full. THeir backend can then detect and discard
> all the sectors with zeros. Understanding the filesystem metdata is a much
> nicer way todo this :-)
It works with most of the formats supported by virt-df.  For MBR & LVM
once you''ve parsed the partition tables / LVM metadata, then the
information you need just falls out naturally.  For NTFS & DOS FAT
(you won''t believe it but ...) you need to find the allocation
bitmaps/tables in both cases in order to calculate blocks used/free
for df anyway.  For ext2 it''s a little bit more tricky because one
needs to additionally parse the group block free bitmaps [this bit
doesn''t work at the moment, but is in principle very simple to add].

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-top is ''top'' for virtual machines.  Tiny program with
many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

On Thu, May 15, 2008 at 07:53:55PM +0100, Richard W.M. Jones
wrote:
> On Thu, May 15, 2008 at 02:54:53PM +0100, Daniel P. Berrange wrote:
> > > This version supports most common filesystems and partitioning
> > > schemes, including:
> > >  - Linux ext2/3
> > >  - DOS FAT32
> > >  - Windows NTFS
> > >  - Linux LVM2 (volume groups and logical volumes)
> > >  - Primary and extended disk partitions
> > >  - Linux swap
> > >  - Linux suspend partition
> > 
> > I''m assuming this only works for raw file & block devices
? Are you planning
> > to support the funky QCow / VMDK formats too ?
> 
> Yes, interesting point.  Since I always use flat files or straight
> partitions for my guests I admit I hadn''t given this much thought
:-)
> However support for these formats is just a matter of decoding enough
> of the structure to enable the same mappings to be made in the
> ''virt-df'' library, same as for LVM2 or indeed MBR
partitions now.
> 
> I''ll take a look at it.  IIRC there are several different
undocumented
> variations on the QCow format?
There''s version 1, and version 2.

And then the incompatible version 2 inflicted by Xen :-(
> > The other thing that could be annoying is that Fedora 9 support for
> > encrypting all volumes - might need to prompt for a decryption key
> > for that.
> 
> Yes -- any encrypted volumes aren''t going to work at the moment,
and
> couldn''t work unless there was a way to access the passphrase.
The passphrase is in the user''s brain. THe OS prompts for it at boot
time,
so virt-df would need todo similar if it wanted to support decryption.
Perhaps its just not a important use case. In the ISP model, guest admins
won''t trust the host admin so you won''t have the keys anyway.
If the guest
admin does trust the host admin, would they really be using encryption
in the guest ?

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|