Libguestfs - Nov 2016 - Transform and "Lenses xx and xx could be used to load this file"

Hi David,

We have a general problem using Augeas in libguestfs and I wonder
if you have any advice.

The problem is that we often -- as with yesterday's patch -- add new
config file paths to lenses.  Or we want to parse config files which
are not necessarily in the exact paths that Augeas is expecting.

Now it's easy enough to parse such files using an arbitrary lens by
calling aug_transform, eg:

  $ ls
  passwd
  $ augtool -r `pwd`
  augtool> transform passwd incl /passwd
  augtool> load
  augtool> ls /files/passwd/
  abrt/                 lightdm/              rjones/
  adm/                  lirc/                 root/
  akmods/               lp/                   rpc/
  # etc

As you can see I applied the 'passwd' lens to the /passwd file
because it's not in the expected location (/etc/passwd).

However if the Augeas lens happens to contain that path already,
things go wrong:

  augtool> transform passwd incl /etc/passwd
  augtool> load
  # /etc/passwd is not parsed by the passwd lens
  augtool> get /augeas/files/etc/passwd/error/message
  /augeas/files/etc/passwd/error/message = Lenses @Passwd and passwd.lns could
be used to load this file

This is specifically a problem when libguestfs adds a call to
aug_transform to cater for a file that is not covered by an existing
lens, but then Augeas adds that file, and then we get the error above
until libguestfs removes the call to aug_transform.  We cannot control
when Augeas and libguestfs are released in different Linux distros.

There are workarounds for this involving fishing out the existing
transform and working out if it covers the file or not, but that's
pretty tedious.

It seems to be that Augeas could be cleverer here and could just
ignore the case where two identical transforms apply to the same file
and DWIM.

What do you think?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

Hi Rich,

On Thu, Nov 3, 2016 at 7:27 AM, Richard W.M. Jones <rjones@redhat.com>
wrote:
> However if the Augeas lens happens to contain that path already,
> things go wrong:
>
>   augtool> transform passwd incl /etc/passwd
>   augtool> load
>   # /etc/passwd is not parsed by the passwd lens
>   augtool> get /augeas/files/etc/passwd/error/message
>   /augeas/files/etc/passwd/error/message = Lenses @Passwd and passwd.lns
> could be used to load this file
>
> This is specifically a problem when libguestfs adds a call to
> aug_transform to cater for a file that is not covered by an existing
> lens, but then Augeas adds that file, and then we get the error above
> until libguestfs removes the call to aug_transform.  We cannot control
> when Augeas and libguestfs are released in different Linux distros.
>
That's a good point that I hadn't thought about before.

> There are workarounds for this involving fishing out the existing
> transform and working out if it covers the file or not, but that's
> pretty tedious.
>
> It seems to be that Augeas could be cleverer here and could just
> ignore the case where two identical transforms apply to the same file
> and DWIM.
>
I just put up a PR[1] that makes it ok for multiple transforms to handle
the same file as long as they use the same lens. I'll wait with making the
release that I was going to make today until I hear back from you - could
you make sure you run this patch through whatever Augeas-relevant tests you
have in libguestfs ? I want to make sure I don't have some unintentional
screwup in there that might break loading or saving where it used to work.
(and hearing back some time next week is  early enough; there's nothing
requiring a release be made right now)

David

[1] https://github.com/hercules-team/augeas/pull/425