Matteo Cafasso
2016-Jul-17 17:40 UTC
[Libguestfs] [PATCH v2 0/2] Added download_blocks API
v2: - Rebase on top of master Matteo Cafasso (2): New API: download_blocks Added download_blocks API test daemon/sleuthkit.c | 41 ++++++++++++++++++++++++++- generator/actions.ml | 24 ++++++++++++++++ gobject/Makefile.inc | 2 ++ src/MAX_PROC_NR | 2 +- tests/tsk/Makefile.am | 1 + tests/tsk/test-download-blocks.sh | 58 +++++++++++++++++++++++++++++++++++++++ 6 files changed, 126 insertions(+), 2 deletions(-) create mode 100755 tests/tsk/test-download-blocks.sh -- 2.8.1
This function allows to download file system data units (blocks) from
the given partition.
The API can be used to detect data hidden within filesystem bad blocks
or slack space.
Moreover for filesystems such as Ext3 and Ext4, this function is the
only way to retrieve deleted files. An example is given in the function
tests.
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
---
daemon/sleuthkit.c | 41 ++++++++++++++++++++++++++++++++++++++++-
generator/actions.ml | 24 ++++++++++++++++++++++++
gobject/Makefile.inc | 2 ++
src/MAX_PROC_NR | 2 +-
4 files changed, 67 insertions(+), 2 deletions(-)
diff --git a/daemon/sleuthkit.c b/daemon/sleuthkit.c
index 1262b68..eb7f877 100644
--- a/daemon/sleuthkit.c
+++ b/daemon/sleuthkit.c
@@ -32,6 +32,7 @@
static int send_command_output (const char *cmd);
GUESTFSD_EXT_CMD(str_icat, icat);
+GUESTFSD_EXT_CMD(str_blkls, blkls);
int
do_download_inode (const mountable_t *mountable, int64_t inode)
@@ -46,7 +47,8 @@ do_download_inode (const mountable_t *mountable, int64_t
inode)
}
/* Construct the command. */
- ret = asprintf(&cmd, "%s -r %s %" PRIi64, str_icat,
mountable->device, inode);
+ ret = asprintf (&cmd, "%s -r %s %" PRIi64,
+ str_icat, mountable->device, inode);
if (ret < 0) {
reply_with_perror ("asprintf");
return -1;
@@ -55,6 +57,43 @@ do_download_inode (const mountable_t *mountable, int64_t
inode)
return send_command_output (cmd);
}
+/* Takes optional arguments, consult optargs_bitmask. */
+int
+do_download_blocks (const mountable_t *mountable, int64_t start, int64_t stop,
+ int unallocated)
+{
+ int ret;
+ const char *params;
+ CLEANUP_FREE char *cmd = NULL;
+
+ /* Data unit address start must be greater than 0 */
+ if (start < 0) {
+ reply_with_error ("starting address must be greater than zero");
+ return -1;
+ }
+
+ /* Data unit address end must be greater than start */
+ if (stop <= start) {
+ reply_with_error ("stopping address must greater than starting
address");
+ return -1;
+ }
+
+ if (!(optargs_bitmask & GUESTFS_DOWNLOAD_BLOCKS_UNALLOCATED_BITMASK))
+ params = " -e";
+ else
+ params = "";
+
+ /* Construct the command. */
+ ret = asprintf (&cmd, "%s %s %s %" PRIi64 "-%"
PRIi64,
+ str_blkls, mountable->device, params, start, stop);
+ if (ret < -0) {
+ reply_with_perror ("asprintf");
+ return -1;
+ }
+
+ return send_command_output (cmd);
+}
+
/* Run the given command, collect the output and send it to the appliance.
* Return 0 on success, -1 on error.
*/
diff --git a/generator/actions.ml b/generator/actions.ml
index 4423313..10f291f 100644
--- a/generator/actions.ml
+++ b/generator/actions.ml
@@ -13177,6 +13177,30 @@ The optional C<force> boolean controls whether
the context
is reset for customizable files, and also whether the
user, role and range parts of the file context is changed." };
+ { defaults with
+ name = "download_blocks"; added = (1, 33, 44);
+ style = RErr, [Mountable "device"; Int64 "start"; Int64
"stop"; FileOut "filename"], [OBool
"unallocated"];
+ proc_nr = Some 468;
+ optional = Some "sleuthkit";
+ progress = true; cancellable = true;
+ shortdesc = "download the given data units from the disk";
+ longdesc = "\
+Download the data units from F<start> address
+to F<stop> from the disk partition (eg. F</dev/sda1>)
+and save them as F<filename> on the local machine.
+
+The use of this API on sparse disk image formats such as QCOW,
+may result in large zero-filled files downloaded on the host.
+
+The size of a data unit varies across filesystem implementations.
+On NTFS filesystems data units are referred as clusters
+while on ExtX ones they are referred as fragments.
+
+If the optional C<unallocated> flag is true (default is false),
+only the unallocated blocks will be extracted.
+This is useful to detect hidden data or to retrieve deleted files
+which data units have not been overwritten yet." };
+
]
(* Non-API meta-commands available only in guestfish.
diff --git a/gobject/Makefile.inc b/gobject/Makefile.inc
index 797bb33..7698c2a 100644
--- a/gobject/Makefile.inc
+++ b/gobject/Makefile.inc
@@ -68,6 +68,7 @@ guestfs_gobject_headers= \
include/guestfs-gobject/optargs-copy_file_to_file.h \
include/guestfs-gobject/optargs-cpio_out.h \
include/guestfs-gobject/optargs-disk_create.h \
+ include/guestfs-gobject/optargs-download_blocks.h \
include/guestfs-gobject/optargs-e2fsck.h \
include/guestfs-gobject/optargs-fstrim.h \
include/guestfs-gobject/optargs-glob_expand.h \
@@ -157,6 +158,7 @@ guestfs_gobject_sources= \
src/optargs-copy_file_to_file.c \
src/optargs-cpio_out.c \
src/optargs-disk_create.c \
+ src/optargs-download_blocks.c \
src/optargs-e2fsck.c \
src/optargs-fstrim.c \
src/optargs-glob_expand.c \
diff --git a/src/MAX_PROC_NR b/src/MAX_PROC_NR
index 5873851..1023289 100644
--- a/src/MAX_PROC_NR
+++ b/src/MAX_PROC_NR
@@ -1 +1 @@
-467
+468
--
2.8.1
Matteo Cafasso
2016-Jul-17 17:40 UTC
[Libguestfs] [PATCH v2 2/2] Added download_blocks API test
The test shows how the function can be used to retrieve deleted data from a disk partition. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- tests/tsk/Makefile.am | 1 + tests/tsk/test-download-blocks.sh | 58 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+) create mode 100755 tests/tsk/test-download-blocks.sh diff --git a/tests/tsk/Makefile.am b/tests/tsk/Makefile.am index f9b2fef..0b50839 100644 --- a/tests/tsk/Makefile.am +++ b/tests/tsk/Makefile.am @@ -19,6 +19,7 @@ include $(top_srcdir)/subdir-rules.mk TESTS = \ test-download-inode.sh \ + test-download-blocks.sh \ test-filesystem-walk.sh TESTS_ENVIRONMENT = $(top_builddir)/run --test diff --git a/tests/tsk/test-download-blocks.sh b/tests/tsk/test-download-blocks.sh new file mode 100755 index 0000000..234b557 --- /dev/null +++ b/tests/tsk/test-download-blocks.sh @@ -0,0 +1,58 @@ +#!/bin/bash - +# libguestfs +# Copyright (C) 2016 Red Hat Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +# Test the download_blocks command. + +set -e + +if [ -n "$SKIP_TEST_DOWNLOAD_BLOCKS_SH" ]; then + echo "$0: test skipped because environment variable is set." + exit 77 +fi + +rm -f test-download-blocks.bin + +# Skip if TSK is not supported by the appliance. +if ! guestfish add /dev/null : run : available "sleuthkit"; then + echo "$0: skipped because TSK is not available in the appliance" + exit 77 +fi + +if [ ! -s ../../test-data/phony-guests/blank-fs.img ]; then + echo "$0: skipped because blank-fs.img is zero-sized" + exit 77 +fi + +# download Master File Table ($MFT). +guestfish --ro -a ../../test-data/phony-guests/blank-fs.img <<EOF +run +mount /dev/sda1 / +write /test.txt "$foo$bar$" +rm /test.txt +umount / +download-blocks /dev/sda1 0 8192 test-download-blocks.bin unallocated:true +EOF + +# test extracted data contains $foo$bar$ string +grep -q "$foo$bar$" test-download-blocks.bin +if [ $? neq 0 ]; then + echo "$0: removed data not found." + exit 1 +fi + +rm -f test-download-blocks.bin -- 2.8.1
Pino Toscano
2016-Jul-20 14:04 UTC
Re: [Libguestfs] [PATCH v2 0/2] Added download_blocks API
Hi, On Sunday, 17 July 2016 20:40:18 CEST Matteo Cafasso wrote:> v2: > > - Rebase on top of master > > Matteo Cafasso (2): > New API: download_blocks > Added download_blocks API test > > daemon/sleuthkit.c | 41 ++++++++++++++++++++++++++- > generator/actions.ml | 24 ++++++++++++++++ > gobject/Makefile.inc | 2 ++ > src/MAX_PROC_NR | 2 +- > tests/tsk/Makefile.am | 1 + > tests/tsk/test-download-blocks.sh | 58 +++++++++++++++++++++++++++++++++++++++ > 6 files changed, 126 insertions(+), 2 deletions(-) > create mode 100755 tests/tsk/test-download-blocks.shThe series LGTM, I pushed it after removing the extra change in do_download_inode and fixing the version. More a curiosity question than a complain or something else: how are these APIs are supposed to be used? What is the forensics-related workflow using them? Considering they are quite specific, I was thinking about adding a documentation paragraph and/or some example to describe/show them better, what do you think? Thanks, -- Pino Toscano