Libguestfs - Mar 2016 - Re: [PATCH 0/2] added icat and fls0 APIs for deleted files recovery

As discussed in the topic:
https://www.redhat.com/archives/libguestfs/2016-March/msg00018.html

I'd like to add to libguestfs the disk forensics capabilities offered by The
Sleuth Kit.
http://www.sleuthkit.org/

The two APIs I'm adding with the patch are a simple example of which type of
features TSK can enable.

The icat API it's very similar to the previously added ntfscat_i but it
works with multiple filesystem types and has an interesting added value: it can
extract deleted files. The end result heavily depends on the filesystem
internals and on how much it has been used. On FAT, Ext2 and NTFS it can recover
part or the whole file pretty easily as long as it's sectors have not been
overwritten by others. For Ext3 and Ext4 it's a bit more difficult, usually
data carving is more effective in these cases. I haven't tested on other
filesystems.

The fls0 API works similar to find but it returns more information such as the
files/directories metadata addresses (inodes) and whether the files are
allocated (in use) or not (deleted). Later on I will add more fine grained APIs
such as fls0-d for showing only deleted files and fls0-m which reports the
creation/access/deletion timestamps. I might need some help for the related fls
API not relying on an external file.

A concrete example on a Windows 7 disk image.

$ ./run guestfish --ro -a /home/noxdafox/disks/win7clone.qcow2
><fs> run
There's no need to mount the disks as TSK reads the raw data structures of
the filesystems.
><fs> fls0 /dev/sda2 /home/noxdafox/disk-content.txt
r/r 15711-128-1:       
$Recycle.Bin/S-1-5-21-2379395878-2832339042-1309242031-1000/desktop.ini
-/r * 60015-128-1:     
$Recycle.Bin/S-1-5-21-2379395878-2832339042-1309242031-1000/$R07QQZ2.txt
-/r * 60015-128-3:     
$Recycle.Bin/S-1-5-21-2379395878-2832339042-1309242031-1000/$R07QQZ2.txt:Zone.Identifier

We can see the content of the recycle bin where I trashed and deleted a
readme.txt file (renamed as $R07QQZ2.txt).
The asterisk (*) shows the file is not allocated and it has been deleted from
the disk.
We can see its inode number (60015) which means the inode has not been recycled
yet so there's quite a good chance we can recover it.
><fs> icat /dev/sda2 60015 /home/noxdafox/recovered.txt
The file gets fully recovered in my home directory. The correct sha1 proves the
recover was 100% successful.

I decided to add these APIs because they cover a use case I think libguestfs
could immediately benefit from. With TSK tools is possible to do more in depth
disk analysis to quickly discover hiddend partitions (malware rootkits) or
hidded data in unallocated sectors.

Matteo Cafasso (2):
  added icat and fls0 APIs
  added icat and fls0 APIs tests

 Makefile.am              |   1 +
 appliance/packagelist.in |   3 ++
 configure.ac             |   1 +
 daemon/Makefile.am       |   1 +
 daemon/tsk.c             | 122 +++++++++++++++++++++++++++++++++++++++++++++++
 generator/actions.ml     |  33 +++++++++++++
 po/POTFILES              |   1 +
 src/MAX_PROC_NR          |   2 +-
 tests/tsk/Makefile.am    |  27 +++++++++++
 tests/tsk/test-fls0.sh   |  55 +++++++++++++++++++++
 tests/tsk/test-icat.sh   |  53 ++++++++++++++++++++
 11 files changed, 298 insertions(+), 1 deletion(-)
 create mode 100644 daemon/tsk.c
 create mode 100644 tests/tsk/Makefile.am
 create mode 100755 tests/tsk/test-fls0.sh
 create mode 100755 tests/tsk/test-icat.sh

--
2.7.0

Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
---
 appliance/packagelist.in |   3 ++
 daemon/Makefile.am       |   1 +
 daemon/tsk.c             | 122 +++++++++++++++++++++++++++++++++++++++++++++++
 generator/actions.ml     |  33 +++++++++++++
 po/POTFILES              |   1 +
 src/MAX_PROC_NR          |   2 +-
 6 files changed, 161 insertions(+), 1 deletion(-)
 create mode 100644 daemon/tsk.c

diff --git a/appliance/packagelist.in b/appliance/packagelist.in
index aa5e07e..0951417 100644
--- a/appliance/packagelist.in
+++ b/appliance/packagelist.in
@@ -46,6 +46,7 @@ ifelse(REDHAT,1,
   pcre
   reiserfs-utils
   libselinux
+  sleuthkit
   syslinux-extlinux
   systemd              dnl for /sbin/reboot and udevd
   vim-minimal
@@ -89,6 +90,7 @@ dnl iproute has been renamed to iproute2
   ntfs-3g
   openssh-client
   reiserfsprogs
+  sleuthkit
   sysvinit                  dnl for /sbin/reboot
   systemd                   dnl alternative for /sbin/reboot
   ufsutils
@@ -120,6 +122,7 @@ ifelse(ARCHLINUX,1,
   ntfs-3g
   pcre
   reiserfsprogs
+  sleuthkit
   systemd
   vim
   xz
diff --git a/daemon/Makefile.am b/daemon/Makefile.am
index caba83e..4e2051b 100644
--- a/daemon/Makefile.am
+++ b/daemon/Makefile.am
@@ -179,6 +179,7 @@ guestfsd_SOURCES = \
 	syslinux.c \
 	tar.c \
 	truncate.c \
+	tsk.c \
 	umask.c \
 	upload.c \
 	utimens.c \
diff --git a/daemon/tsk.c b/daemon/tsk.c
new file mode 100644
index 0000000..8cc35eb
--- /dev/null
+++ b/daemon/tsk.c
@@ -0,0 +1,122 @@
+/* libguestfs - the guestfsd daemon
+ * Copyright (C) 2009-2016 Red Hat Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
USA.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <inttypes.h>
+#include <string.h>
+#include <unistd.h>
+
+#include "guestfs_protocol.h"
+#include "daemon.h"
+#include "actions.h"
+#include "optgroups.h"
+
+static int file_out (const char *cmd);
+
+GUESTFSD_EXT_CMD(str_tsk_probe, icat);
+
+int
+optgroup_icat_available (void)
+{
+  return prog_exists (str_tsk_probe);
+}
+
+int
+do_icat (const mountable_t *mountable, int64_t inode)
+{
+  CLEANUP_FREE char *cmd = NULL;
+
+  /* Inode must be greater than 0 */
+  if (inode < 0) {
+    reply_with_error ("inode must be >= 0");
+    return -1;
+  }
+
+  /* Construct the command. */
+  if (asprintf (&cmd, "icat -r %s %" PRIi64,
mountable->device, inode) == -1) {
+    reply_with_perror ("asprintf");
+    return -1;
+  }
+
+  return file_out (cmd);
+}
+
+int
+do_fls0 (const mountable_t *mountable)
+{
+  CLEANUP_FREE char *cmd = NULL;
+
+  /* Construct the command. */
+  if (asprintf (&cmd, "fls -pr %s", mountable->device) == -1)
{
+    reply_with_perror ("asprintf");
+    return -1;
+  }
+
+  return file_out (cmd);
+}
+
+static int
+file_out (const char *cmd)
+{
+  int r;
+  FILE *fp;
+  char buffer[GUESTFS_MAX_CHUNK_SIZE];
+
+  if (verbose)
+    fprintf (stderr, "%s\n", cmd);
+
+  fp = popen (cmd, "r");
+  if (fp == NULL) {
+    reply_with_perror ("%s", cmd);
+    return -1;
+  }
+
+  /* Now we must send the reply message, before the file contents.  After
+   * this there is no opportunity in the protocol to send any error
+   * message back.  Instead we can only cancel the transfer.
+   */
+  reply (NULL, NULL);
+
+  while ((r = fread (buffer, 1, sizeof buffer, fp)) > 0) {
+    if (send_file_write (buffer, r) < 0) {
+      pclose (fp);
+      return -1;
+    }
+  }
+
+  if (ferror (fp)) {
+    fprintf (stderr, "fread: %m");
+    send_file_end (1);		/* Cancel. */
+    pclose (fp);
+    return -1;
+  }
+
+  if (pclose (fp) != 0) {
+    fprintf (stderr, "pclose: %m");
+    send_file_end (1);		/* Cancel. */
+    return -1;
+  }
+
+  if (send_file_end (0))	/* Normal end of file. */
+    return -1;
+
+  return 0;
+}
diff --git a/generator/actions.ml b/generator/actions.ml
index 287d7f5..ff6aa3f 100644
--- a/generator/actions.ml
+++ b/generator/actions.ml
@@ -12919,6 +12919,39 @@ within the C<$Extend> folder.
 The filesystem from which to extract the file must be unmounted,
 otherwise the call will fail." };

+  { defaults with
+    name = "icat"; added = (1, 33, 14);
+    style = RErr, [Mountable "device"; Int64 "inode";
FileOut "filename"], [];
+    proc_nr = Some 464;
+    optional = Some "icat";
+    progress = true; cancellable = true;
+    shortdesc = "download a file to the local machine given its
inode";
+    longdesc = "\
+Download a file given its inode from the disk partition (eg.
F</dev/sda1>)
+and save it as F<filename> on the local machine.
+
+This allows to download deleted or inaccessible files." };
+
+  { defaults with
+    name = "fls0"; added = (1, 33, 14);
+    style = RErr, [Mountable "device"; FileOut "filename"],
[];
+    proc_nr = Some 465;
+    optional = Some "icat";
+    progress = true; cancellable = true;
+    shortdesc = "list the content of the disk";
+    longdesc = "\
+This specialized command is used to get a listing of
+the content of the disk partition (eg. F</dev/sda1>).
+The list of filenames is written to the local file F<filenames> (on the
host).
+
+The command can display hidden or deleted files.
+
+The output information contains the file type, the metadata address
+and the full path of the file.
+
+If the file was deleted, an asterisk will be shown between the file type
+and the metadata address." };
+
 ]

 (* Non-API meta-commands available only in guestfish.
diff --git a/po/POTFILES b/po/POTFILES
index 0fb99b0..4fbc551 100644
--- a/po/POTFILES
+++ b/po/POTFILES
@@ -108,6 +108,7 @@ daemon/sync.c
 daemon/syslinux.c
 daemon/tar.c
 daemon/truncate.c
+daemon/tsk.c
 daemon/umask.c
 daemon/upload.c
 daemon/utimens.c
diff --git a/src/MAX_PROC_NR b/src/MAX_PROC_NR
index 26551cf..073c57b 100644
--- a/src/MAX_PROC_NR
+++ b/src/MAX_PROC_NR
@@ -1 +1 @@
-463
+465
--
2.7.0

Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
---
 Makefile.am            |  1 +
 configure.ac           |  1 +
 tests/tsk/Makefile.am  | 27 +++++++++++++++++++++++++
 tests/tsk/test-fls0.sh | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/tsk/test-icat.sh | 53 ++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 137 insertions(+)
 create mode 100644 tests/tsk/Makefile.am
 create mode 100755 tests/tsk/test-fls0.sh
 create mode 100755 tests/tsk/test-icat.sh

diff --git a/Makefile.am b/Makefile.am
index fe92630..42ff072 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -83,6 +83,7 @@ SUBDIRS += tests/journal
 SUBDIRS += tests/relative-paths
 SUBDIRS += tests/gdisk
 SUBDIRS += tests/regressions
+SUBDIRS += tests/tsk
 endif

 # libguestfs-test-tool
diff --git a/configure.ac b/configure.ac
index fd8e534..c031707 100644
--- a/configure.ac
+++ b/configure.ac
@@ -281,6 +281,7 @@ AC_CONFIG_FILES([Makefile
                  tests/selinux/Makefile
                  tests/syslinux/Makefile
                  tests/tmpdirs/Makefile
+                 tests/tsk/Makefile
                  tests/xfs/Makefile
                  tests/xml/Makefile
                  tools/Makefile
diff --git a/tests/tsk/Makefile.am b/tests/tsk/Makefile.am
new file mode 100644
index 0000000..b9a4195
--- /dev/null
+++ b/tests/tsk/Makefile.am
@@ -0,0 +1,27 @@
+# libguestfs
+# Copyright (C) 2012-2016 Red Hat Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+include $(top_srcdir)/subdir-rules.mk
+
+TESTS = \
+	test-icat.sh \
+	test-fls0.sh
+
+TESTS_ENVIRONMENT = $(top_builddir)/run --test
+
+EXTRA_DIST = \
+	$(TESTS)
diff --git a/tests/tsk/test-fls0.sh b/tests/tsk/test-fls0.sh
new file mode 100755
index 0000000..18c7690
--- /dev/null
+++ b/tests/tsk/test-fls0.sh
@@ -0,0 +1,55 @@
+#!/bin/bash -
+# libguestfs
+# Copyright (C) 2016 Red Hat Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# Test the ntfscat-i command.
+
+set -e
+
+if [ -n "$SKIP_TEST_FLS0_SH" ]; then
+    echo "$0: test skipped because environment variable is set."
+    exit 77
+fi
+
+rm -f test-fls0.txt
+
+# Skip if TSK is not supported by the appliance.
+if ! guestfish add /dev/null : run : available "icat"; then
+    echo "$0: skipped because TSK is not supported by the appliance"
+    exit 77
+fi
+
+if [ ! -s ../../test-data/phony-guests/ubuntu.img ]; then
+    echo "$0: skipped because ubuntu.img is zero-sized"
+    exit 77
+fi
+
+# list ubuntu image contents
+guestfish --ro -a ../../test-data/phony-guests/ubuntu.img <<EOF
+run
+fls0 /dev/sda2 test-fls0.txt
+EOF
+
+# test bin directory is in the list
+grep -q "bin" test-fls0.txt
+
+if [ $? neq 0 ]; then
+    echo "$0: /bin not found in files list."
+    exit 1
+fi
+
+rm -f test-fls0.txt
diff --git a/tests/tsk/test-icat.sh b/tests/tsk/test-icat.sh
new file mode 100755
index 0000000..90356c4
--- /dev/null
+++ b/tests/tsk/test-icat.sh
@@ -0,0 +1,53 @@
+#!/bin/bash -
+# libguestfs
+# Copyright (C) 2016 Red Hat Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# Test the ntfscat-i command.
+
+set -e
+
+if [ -n "$SKIP_TEST_ICAT_SH" ]; then
+    echo "$0: test skipped because environment variable is set."
+    exit 77
+fi
+
+rm -f test-mft.bin
+
+# Skip if TSK is not supported by the appliance.
+if ! guestfish add /dev/null : run : available "icat"; then
+    echo "$0: skipped because TSK is not supported by the appliance"
+    exit 77
+fi
+
+if [ ! -s ../../test-data/phony-guests/windows.img ]; then
+    echo "$0: skipped because windows.img is zero-sized"
+    exit 77
+fi
+
+# download Master File Table ($MFT).
+guestfish --ro -a ../../test-data/phony-guests/windows.img <<EOF
+run
+icat /dev/sda2 0 test-mft.bin
+EOF
+
+# test extracted file is the Master File Table
+if [ `head -c 5 test-mft.bin` != "FILE0" ]; then
+    echo "$0: wrong file extracted."
+    exit 1
+fi
+
+rm -f test-mft.bin
--
2.7.0

On Sun, Mar 06, 2016 at 05:42:24PM +0200, Matteo Cafasso
wrote:
> As discussed in the topic:
https://www.redhat.com/archives/libguestfs/2016-March/msg00018.html
> 
> I'd like to add to libguestfs the disk forensics capabilities offered
by The Sleuth Kit.
> http://www.sleuthkit.org/
> 
> The two APIs I'm adding with the patch are a simple example of which
type of features TSK can enable.
A few comments in general terms:

The current splitting of the commits doesn't make much sense to me.
I think it would be better as:

 - commit to add TSK to the appliance

 - commit to add the icat API

 - tests for icat

 - commit to add the fls0 API

 - tests for fls0

although it would be fine to combine the tests with the new API, or
even have all the tests as a single separate commit (as now).

This benefits you because it will allow patches to go upstream
earlier.  For example, a commit to add TSK to the appliance is a
simple and obvious change that I see no problem with.  Also the icat
API is closer to being ready than the fls0 API (see below for
explanation).
> ><fs> fls0 /dev/sda2 /home/noxdafox/disk-content.txt
> 
> r/r 15711-128-1:       
$Recycle.Bin/S-1-5-21-2379395878-2832339042-1309242031-1000/desktop.ini
> -/r * 60015-128-1:     
$Recycle.Bin/S-1-5-21-2379395878-2832339042-1309242031-1000/$R07QQZ2.txt
> -/r * 60015-128-3:     
$Recycle.Bin/S-1-5-21-2379395878-2832339042-1309242031-1000/$R07QQZ2.txt:Zone.Identifier
What is `/home/noxdafox/disk-content.txt'?

The problem with this API is it pushes all the parsing up in the
stack, to libguestfs consumers.

In general we'd like to avoid that and have just one place where all
parsing needs to be done (ie. libguestfs itself), so it'd be nicer to
have an API that returns a list of structs (RStructList) with all the
important fields parsed out.

Does TSK have a machine-readable mode?  If it does, it'll definitely
make things easier if (eg) JSON or XML output is available.  If not,
push upstream to add that to TSK -- it's a simple change for them,
which will make their tools much more usable, a win for everyone.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

On Sun, Mar 06, 2016 at 05:42:25PM +0200, Matteo Cafasso
wrote:
> +static int
> +file_out (const char *cmd)
> +{
> +  int r;
> +  FILE *fp;
> +  char buffer[GUESTFS_MAX_CHUNK_SIZE];
Soon libguestfs will prevent you from using large stack allocations.
This is easy to fix.  See:
https://www.redhat.com/archives/libguestfs/2016-March/msg00052.html
> diff --git a/generator/actions.ml b/generator/actions.ml
> index 287d7f5..ff6aa3f 100644
> --- a/generator/actions.ml
> +++ b/generator/actions.ml
> @@ -12919,6 +12919,39 @@ within the C<$Extend> folder.
>  The filesystem from which to extract the file must be unmounted,
>  otherwise the call will fail." };
> 
> +  { defaults with
> +    name = "icat"; added = (1, 33, 14);
> +    style = RErr, [Mountable "device"; Int64 "inode";
FileOut "filename"], [];
> +    proc_nr = Some 464;
> +    optional = Some "icat";
> +    progress = true; cancellable = true;
> +    shortdesc = "download a file to the local machine given its
inode";
> +    longdesc = "\
> +Download a file given its inode from the disk partition (eg.
F</dev/sda1>)
> +and save it as F<filename> on the local machine.
> +
> +This allows to download deleted or inaccessible files." };
This one looks OK.
> +  { defaults with
> +    name = "fls0"; added = (1, 33, 14);
> +    style = RErr, [Mountable "device"; FileOut
"filename"], [];
> +    proc_nr = Some 465;
> +    optional = Some "icat";
> +    progress = true; cancellable = true;
> +    shortdesc = "list the content of the disk";
> +    longdesc = "\
> +This specialized command is used to get a listing of
> +the content of the disk partition (eg. F</dev/sda1>).
> +The list of filenames is written to the local file F<filenames> (on
the host).
> +
> +The command can display hidden or deleted files.
> +
> +The output information contains the file type, the metadata address
> +and the full path of the file.
> +
> +If the file was deleted, an asterisk will be shown between the file type
> +and the metadata address." };
As discussed, this shouldn't be a FileOut API at all.  It should
return a list of structs (RStructList).

If the output could be very large then we may need to split it into an
internal API, but let's not worry about that right now.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

On 07/03/16 13:29, Richard W.M. Jones wrote:
> On Sun, Mar 06, 2016 at 05:42:24PM +0200, Matteo Cafasso wrote:
>> As discussed in the topic:
https://www.redhat.com/archives/libguestfs/2016-March/msg00018.html
>>
>> I'd like to add to libguestfs the disk forensics capabilities
offered by The Sleuth Kit.
>> http://www.sleuthkit.org/
>>
>> The two APIs I'm adding with the patch are a simple example of
which type of features TSK can enable.
> A few comments in general terms:
>
> The current splitting of the commits doesn't make much sense to me.
> I think it would be better as:
>
>   - commit to add TSK to the appliance
>
>   - commit to add the icat API
>
>   - tests for icat
>
>   - commit to add the fls0 API
>
>   - tests for fls0
>
> although it would be fine to combine the tests with the new API, or
> even have all the tests as a single separate commit (as now).
>
> This benefits you because it will allow patches to go upstream
> earlier.  For example, a commit to add TSK to the appliance is a
> simple and obvious change that I see no problem with.  Also the icat
> API is closer to being ready than the fls0 API (see below for
> explanation).
Indeed I've done quite a poor job in this. I will split it as
suggested.
>
>>> <fs> fls0 /dev/sda2 /home/noxdafox/disk-content.txt
>> r/r 15711-128-1:       
$Recycle.Bin/S-1-5-21-2379395878-2832339042-1309242031-1000/desktop.ini
>> -/r * 60015-128-1:     
$Recycle.Bin/S-1-5-21-2379395878-2832339042-1309242031-1000/$R07QQZ2.txt
>> -/r * 60015-128-3:     
$Recycle.Bin/S-1-5-21-2379395878-2832339042-1309242031-1000/$R07QQZ2.txt:Zone.Identifier
> What is `/home/noxdafox/disk-content.txt'?
It's the local (host side) file where to store the command
output.
>
> The problem with this API is it pushes all the parsing up in the
> stack, to libguestfs consumers.
>
> In general we'd like to avoid that and have just one place where all
> parsing needs to be done (ie. libguestfs itself), so it'd be nicer to
> have an API that returns a list of structs (RStructList) with all the
> important fields parsed out.
As the API documentation says, this is the low level API which I have 
provided as an example.

I took inspiration from the guestfs_ls0 API which does a similar job 
storing the content of a directory onto a host file.

If I understood correctly (the dynamic code generation is still 
confusing me a bit), the way Libguestfs implements commands which could 
have a large output is via first dumping it onto a local file and then 
iterating over it.
This command would list the entire content of a disk including the 
deleted files therefore we need to expect a large output.

What is missing is the higher level implementation which would pretty 
much look like the libguestfs_ls API. I need to better understand how to 
implement it and suggestions are more than appreciated. I tried to trace 
back how the guestfs_find is implemented for example, but I'm still a 
bit disoriented by the automagic code generation.
>
> Does TSK have a machine-readable mode?  If it does, it'll definitely
> make things easier if (eg) JSON or XML output is available.  If not,
> push upstream to add that to TSK -- it's a simple change for them,
> which will make their tools much more usable, a win for everyone.
I personally disagree on this. The TSK `fls` command is a clone of the 
bash `ls` one. Maybe it's more similar to `ls -al` as it returns 
additional information. IMHO asking to upstream to add JSON or XML 
output format would sound pretty much as asking the same to bash for the 
`ls` utility.

The end result is to still return a list of structs or a list of 
strings. But parsing the `fls` output shouldn't be that hard. It's 
documentation is here:
http://wiki.sleuthkit.org/index.php?title=Fls
>
> Rich.
>