Libguestfs - Jan 2017 - ntfs-3g data deduplication support in guestmount for creating file based back-ups on visualization platform

Hello everybody,

I am using libguestfs-tools version 1.28.1 in a bunch of bash scripts to 
make back-ups of the files on a visualisation platform.

I been trying to get read access to ntfs volumes with data deduplication 
working.

The ntfs-3g guys developed some support and I have this working when 
using the commands our back-up servers.

However when using the guestmount tool the read suport foor dedub is gone.

See this email for all the commands I use (working mount -t ntfs and 
non-working guestmount commands).

https://sourceforge.net/p/ntfs-3g/mailman/message/35595392/

How can I see what version of ntfs-3g is used in guestmount and if it is 
using the /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-80000013.so 
plugin that is on my back-up server?

Thank you in advance!

Kind regards,

Jelle de Jong

On Tue, Jan 10, 2017 at 05:06:13PM +0100, Jelle de Jong
wrote:
> Hello everybody,
> 
> I am using libguestfs-tools version 1.28.1 in a bunch of bash
> scripts to make back-ups of the files on a visualisation platform.
> 
> I been trying to get read access to ntfs volumes with data
> deduplication working.
Do you have more information about data deduplication?  It's the first
I've heard of it for NTFS (or indeed any ordinary Unix filesystem).
In fact I thought it was a feature of the block layer rather than the
filesystem.  But anyway ...
> The ntfs-3g guys developed some support and I have this working when
> using the commands our back-up servers.
> 
> However when using the guestmount tool the read suport foor dedub is gone.
This is maybe not especially surprising.  A lot of layers have to work
correctly in order for a feature to be passed through guestmount.
You've got (at least) kernel+FUSE on the host side (for guestmount),
the libguestfs API, the appliance kernel, and ntfs-3g in the
appliance.  Getting discard support working involved a considerable
effort.
> See this email for all the commands I use (working mount -t ntfs and
> non-working guestmount commands).
> 
> https://sourceforge.net/p/ntfs-3g/mailman/message/35595392/
> 
> How can I see what version of ntfs-3g is used in guestmount and if
> it is using the
> /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-80000013.so plugin
> that is on my back-up server?
Probably helpful to refer to the diagram here:

  http://libguestfs.org/guestfs-internals.1.html

ntfs-3g in your case is running in the "appliance" box in that
diagram.

The plugin file *should* be copied into the appliance, assuming that
it's part of the Debian ntfs-3g package and the Debian ntfs-3g package
isn't doing anything weird.

You can easily check using virt-rescue:

  $ virt-rescue --ro -a windows.img
  ><rescue> ls -l /usr/lib/x86_64-linux-gnu/ntfs-3g/
  ><rescue> mount /dev/sda1 /sysroot

(and then use whatever ntfs commands are needed to check that the
plugin is being used)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

Thank you Richard!

I will do some more work and testing on this tomorrow, and report back 
with some more information you asked.

In the meanwhile, could you help me with information how I can upgrade 
the ntfs-3g version (same as host) in the "appliance" and add the 
plug-in library?

On 16/01/17 15:29, Richard W.M. Jones wrote:
> The plugin file *should* be copied into the appliance, assuming that
> it's part of the Debian ntfs-3g package and the Debian ntfs-3g package
> isn't doing anything weird.
The plug-in is added manually and is not part of the ntfs-3g (yet), it 
is manually downloaded and copied.

Thank you in advance!

Kind regards,

Jelle de Jong