libvirt users - May 2011 - converting save/dump output into physical memory image

A lot of people in the security community, myself included, are
interested in memory forensics these days.  Virtualization is a natural
fit with memory forensics because it allows one to get access to a
guest's memory without having to introduce any extra software into the
guest or otherwise interfere with it.  Incident responders are
particularly interested in getting memory dumps from systems they're
investigating.

Virsh has "save" and "dump" commands for storing the state
of a guest to
a file on disk, but memory of KVM guests doesn't get saved in the
"standard" input format for memory forensics tools, which is a raw
physical memory image.  (This is what you'd get via the classical "dd
/dev/mem" approach or the contemporary equivalent using the crash
driver; and VMware Server and Workstation produce .vmem files, which are
such raw physical memory images, when a guest is paused or snapshotted.)

In order to analyze the memory of Libvirt/KVM guests with my Linux
memory forensics software, Second Look, I've created a tool for
converting Libvirt-QEMU-save files (output of virsh save command) or
QEMU-savevm files (output of virsh dump command) to raw physical memory
images.

I've got a basic working capability, though I'm still tracking down some
problems with a guest allocated 8GB RAM--not all the memory seems to be
present in the save or dump file.  And I haven't tested very extensively
yet, version support is limited to what I myself am currently running, etc.

I'd like to know if this is a capability that others are interested in.
 Is this something that would be of interest to the Libvirt project if I
were to contribute the code, or to the KVM project, or do you think it
best exists as a separate project?

I've also got a proof-of-concept tool for converting hibernate images to
raw physical memory images.  Perhaps a collection of tools for
converting various memory dump formats would be a good project.  Anyone
else interested in this kind of stuff?  As an author of commercial
memory forensics software I've got a vested interest in availability of
good memory acquisition capabilities.  But there are a number of people
working on FOSS Linux memory analysis tools, too...

Andrew

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20110505/8fcd6f48/attachment.sig>

On 05/05/2011 11:56 AM, Andrew Tappert wrote:
> 
> A lot of people in the security community, myself included, are
> interested in memory forensics these days.  Virtualization is a natural
> fit with memory forensics because it allows one to get access to a
> guest's memory without having to introduce any extra software into the
> guest or otherwise interfere with it.  Incident responders are
> particularly interested in getting memory dumps from systems they're
> investigating.
> 
> Virsh has "save" and "dump" commands for storing the
state of a guest to
> a file on disk, but memory of KVM guests doesn't get saved in the
> "standard" input format for memory forensics tools, which is a
raw
> physical memory image.  (This is what you'd get via the classical
"dd
> /dev/mem" approach or the contemporary equivalent using the crash
> driver; and VMware Server and Workstation produce .vmem files, which are
> such raw physical memory images, when a guest is paused or snapshotted.)
Libvirt also has the virDomainMemoryPeek API; right now, it is not
exposed by virsh, but we could add a command-line-interface for it if
that proves useful.  Does that API fit your needs any better than
converting a qemu dump image back into raw memory?

-- 
Eric Blake   eblake at redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 619 bytes
Desc: OpenPGP digital signature
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20110505/ca88133f/attachment.sig>

On Thu, May 05, 2011 at 01:56:10PM -0400, Andrew Tappert
wrote:
> 
> A lot of people in the security community, myself included, are
> interested in memory forensics these days.  Virtualization is a natural
> fit with memory forensics because it allows one to get access to a
> guest's memory without having to introduce any extra software into the
> guest or otherwise interfere with it.  Incident responders are
> particularly interested in getting memory dumps from systems they're
> investigating.
> 
> Virsh has "save" and "dump" commands for storing the
state of a guest to
> a file on disk, but memory of KVM guests doesn't get saved in the
> "standard" input format for memory forensics tools, which is a
raw
> physical memory image.  (This is what you'd get via the classical
"dd
> /dev/mem" approach or the contemporary equivalent using the crash
> driver; and VMware Server and Workstation produce .vmem files, which are
> such raw physical memory images, when a guest is paused or snapshotted.)
> 
> In order to analyze the memory of Libvirt/KVM guests with my Linux
> memory forensics software, Second Look, I've created a tool for
> converting Libvirt-QEMU-save files (output of virsh save command) or
> QEMU-savevm files (output of virsh dump command) to raw physical memory
> images.
> 
> I've got a basic working capability, though I'm still tracking down
some
> problems with a guest allocated 8GB RAM--not all the memory seems to be
> present in the save or dump file.  And I haven't tested very
extensively
> yet, version support is limited to what I myself am currently running, etc.
KVM only allocates memory on demand, when the guest touches the page,
so the data will almost certainly be sparse.

FWIW, the 'crash' tool has been taught how to understand the output
of 'virsh dump' for Linux guests. I agree, it could be nice to have
a raw memory dump though, instead of the KVM save format, at least
as an option.

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Andrew Tappert <andrew at ...> writes:
> 
> 
> A lot of people in the security community, myself included, are
> interested in memory forensics these days.  Virtualization is a natural
> fit with memory forensics because it allows one to get access to a
> guest's memory without having to introduce any extra software into the
> guest or otherwise interfere with it.  Incident responders are
> particularly interested in getting memory dumps from systems they're
> investigating.
> 
I am definitely interested in this concept!
I haven't played much yet with memory forensic tools (like Volatility) as
atm
I'm focusing on other things but I'm planning to switch to it in a very
near
future.

For what I know it's not possible to analyze memory contained in libvirt
snapshots taken through qemu savevm.
This feature is really powerful under my perspective as it atomically takes the
state of the whole machine: disk, memory, processes; everything is saved in the
same moment.

I am already able to get all the useful information from the disk state: 
through the qemu convert I can get a qcow2 disk image containing the state
of the disk
at the moment of the snapshot, through libguestfs I can mount and read this
image to see FS changes.

I am looking for a similar solution for memory, given a snapshot taken 
through the savevm command I want to get a file to read through Volatility.