similar to: [PATCH 0/2] added icat and fls0 APIs for deleted files recovery

On 07/03/16 13:29, Richard W.M. Jones wrote: > On Sun, Mar 06, 2016 at 05:42:24PM +0200, Matteo Cafasso wrote: >> As discussed in the topic: https://www.redhat.com/archives/libguestfs/2016-March/msg00018.html >> >> I'd like to add to libguestfs the disk forensics capabilities offered by The Sleuth Kit. >> http://www.sleuthkit.org/ >> >> The two APIs

On Sun, Mar 06, 2016 at 05:42:24PM +0200, Matteo Cafasso wrote: > As discussed in the topic: https://www.redhat.com/archives/libguestfs/2016-March/msg00018.html > > I'd like to add to libguestfs the disk forensics capabilities offered by The Sleuth Kit. > http://www.sleuthkit.org/ > > The two APIs I'm adding with the patch are a simple example of which type of features

"icat" name comes from the employed command line tool which might be replaced later on with a different implementation. The command name is a bit confusing because it's similar to "cat" but act as "donwload". Therefore I am renaming it with a more clear name. At the same time I cleaned up a bit the code and improved it's readability and code comments. This

The command name is a bit confusing because it's similar to "cat" but act as "donwload". Therefore I am renaming it with a more clear name. At the same go I cleaned up a bit the code following the standards and improved the API documentation. This patch is ready for review. Code available at: https://github.com/noxdafox/libguestfs/tree/download_inode Matteo Cafasso (3):

Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- appliance/packagelist.in | 3 ++ daemon/Makefile.am | 1 + daemon/tsk.c | 122 +++++++++++++++++++++++++++++++++++++++++++++++ generator/actions.ml | 33 +++++++++++++ po/POTFILES | 1 + src/MAX_PROC_NR | 2 +- 6 files changed, 161 insertions(+), 1 deletion(-) create mode 100644

The Sleuth Kit is a filesystem forensic tool for accessing disk volumes and extracting digital evidence from. http://www.sleuthkit.org/ The icat API allows to download a file from a device given its metadata number (inode). It supports multiple filesystem types. The icat command allows to access to otherwise unreachable files such as filesystem data structures and deleted files which content is

On Sun, Mar 06, 2016 at 05:42:25PM +0200, Matteo Cafasso wrote: > +static int > +file_out (const char *cmd) > +{ > + int r; > + FILE *fp; > + char buffer[GUESTFS_MAX_CHUNK_SIZE]; Soon libguestfs will prevent you from using large stack allocations. This is easy to fix. See: https://www.redhat.com/archives/libguestfs/2016-March/msg00052.html > diff --git

On Mon, Mar 07, 2016 at 08:14:41PM +0200, noxdafox wrote: > As the API documentation says, this is the low level API which I > have provided as an example. > > I took inspiration from the guestfs_ls0 API which does a similar job > storing the content of a directory onto a host file. > > If I understood correctly (the dynamic code generation is still > confusing me a bit),

blkcat allows to extract files given their data units (clusters for NTFS, fragments for ExtX). This API allows to recover files which icat cannot. For example icat often fails retrieving deleted files on Ext3/Ext4 filesystems. As this API works at data unit level, some garbage at the beginning or at the end of the files is expected. An example of blkcat can be found here:

Thanks, I have pushed this patch series. Could you consider changing: > + optional = Some "icat"; I think it would be nice to have a single feature, and to call the feature "sleuthkit" or "forensics" or something like that. We don't need to have one feature per API since installation of a single package (sleuthkit) is sufficient to make all the APIs

The blkls API downloads on the host a range of unallocated blocks on the virtual disk image. This allows to recover deleted data on filesystems where icat fails. Example: guestfish --ro -a /home/noxdafox/ubuntu.qcow2 ><fs> run ><fs> mount /dev/sda1 / ><fs> write /test.txt "$foo$bar$" ><fs> rm /test.txt ><fs> umount / ><fs> blkls

The "icat" name comes from the employed command line tool which might be replaced at any time with a different implementation. The command name is a bit confusing because it's similar to "cat" but act as "donwload". download_inode is more clear and descriptive. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- daemon/sleuthkit.c | 2

On 02/03/16 17:53, Richard W.M. Jones wrote: > On Wed, Mar 02, 2016 at 05:47:40PM +0200, noxdafox wrote: >> Greetings, >> >> I am playing around with the idea of using libguestfs as a forensic >> tool to investigate VM disk images. >> >> Some use cases as example: >> * Sandbox for malware analysis. >> * Incident response in cloud environments.

In order to support the new features I am renaming the file with a better name. The file sleuthkit.c will contain the code depending on the sleuthkit package. The original tsk.c file will contain the logic built using libtsk which is the sleuthkit core library. This patch is ready for review. Code available at: https://github.com/noxdafox/libguestfs/tree/sleuthkit_rename Signed-off-by: Matteo

The ffind API allows to retrieve a file name from a device given its inode. The function returns a struct "tsknode" which contains the file name, its inode and it's allocation status. The struct will be employed by other APIs as well (fls, ifind etc..). $ ./run guestfish --ro -a /home/noxdafox/disks/ubuntu.qcow2 ><fs> run ><fs> ffind /dev/sda1 2 tsk_name: /

>From 'available icat' to 'available sleuthkit'. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- daemon/tsk.c | 6 +++--- generator/actions.ml | 2 +- tests/tsk/test-icat.sh | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/daemon/tsk.c b/daemon/tsk.c index 647d1e7..0fe1250 100644 --- a/daemon/tsk.c +++ b/daemon/tsk.c @@ -31,12

Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- daemon/Makefile.am | 1 + daemon/tsk.c | 114 +++++++++++++++++++++++++++++++++++++++++++++++++++ generator/actions.ml | 13 ++++++ po/POTFILES | 1 + src/MAX_PROC_NR | 2 +- 5 files changed, 130 insertions(+), 1 deletion(-) create mode 100644 daemon/tsk.c diff --git a/daemon/Makefile.am

On 02/03/16 18:24, Richard W.M. Jones wrote: > On Wed, Mar 02, 2016 at 05:59:32PM +0200, noxdafox wrote: >> One of the patches I'm talking about would add TSK (The Sleuth Kit) >> as a dependency within the appliance. >> >> This would bring new APIs such as: >> 'fls' more powerful 'ls' command allowing to get list of deleted >> files or

On 07/03/16 21:45, Richard W.M. Jones wrote: > Thanks, I have pushed this patch series. > > Could you consider changing: > >> + optional = Some "icat"; > I think it would be nice to have a single feature, and to call the > feature "sleuthkit" or "forensics" or something like that. We don't > need to have one feature per API since

With this API we complete the set of functions required to extract deleted files/data from most of the available filesystems. The function allows to extract data units (blocks) within a given range from a partition. The tests show an example on how the function can be used to retrieve deleted data. Matteo Cafasso (2): New API: download_blocks Added download_blocks API test