Puppet users - Sep 2011 - commands in Puppet providers being escaped differently to a shell

Hi list,

I''m making a few modifications to the puppet-virt type from GitHub
(https://github.com/carlasouza/puppet-virt.git) and I''ve run across an
interesting problem with what looks like how a "command" in a Puppet
provider is escaping or interpreting it''s arguments. Explaining this
is a bit long winded so bear with me...

We want to pass in multiple kernel boot arguments when doing a KVM
virt-install, a simple example is below. --extra-args takes a single
argument and can''t be specified twice so if you want to specify
multiple you have to group them together as one string like so:

    virt-install --name foo --extra-args "dhcp console=ttyS0 ks=http://
ks/foo.ks"

This gets translated by virt-install (Python) into individual
arguments after "-append" to the qemu-kvm binary like so:

    /usr/bin/qemu-kvm <heaps of other arguments> -append method=http://
gs2repo.gs2.tradefair/mrepo/latest/fedora15-x86_64/RPMS.os/ noipv6
console=ttyS0,115200 dhcp ks=http://gs2repo.gs2.tradefair/mrepo/
kickstarts/fedora15.ks

So the transition of arguments is "Shell -> Python -> Binary"
and
everything works grand. Now, trying to replicate the same thing in
Puppet has proved difficult. Nearest I can tell with my limited
Puppet / Ruby knowledge, the argument list is being handled
differently and so what gets put on the qemu-kvm binary''s command line
is one single string of arguments that aren''t expanded out properly
and so qemu-kvm doesn''t function as we intend.

Here''s how I''m trying to do it in Puppet. A simplified virt
resource
in a manifest, note the double sets of quotes:

    virt { "foo":
      ensure => running,
      boot_options  => ''"noipv6 console=ttyS0,115200 dhcp
ks=http://
gs2repo.gs2.tradefair/mrepo/kickstarts/fedora15.ks"'',
    }

This gets translated into the provider (heavily simplified below for
brevity), arguments are pulled from resource parameters and translated
into arrays and appended as arguments to the Puppet/Ruby virt-install
command:

    Puppet::Type.type(:virt).provide(:libvirt) do
        commands :virtinstall => "/usr/bin/virt-install"
        bootargs = ["--extra-args", resource[:boot_options] ]
        def install
            virtinstall generalargs + networkargs + graphicargs +
bootargs

Applying this manifest and looking at the debug output, it seems to be
escaped correctly:

    debug: Puppet::Type::Virt::ProviderLibvirt: Executing ''/usr/bin/
virt-install --name foo  --extra-args "dhcp console=ttyS0 ks=http://ks/
foo.ks"''
    notice: /Stage[main]//Virt[foo]/ensure: created

However if you then check to see what qemu-kvm command is now running,
it''s still got the shell quotes in the argument list:

    /usr/bin/qemu-kvm <heaps of other arguments> -append method=http://
gs2repo.gs2.tradefair/mrepo/latest/fedora15-x86_64/RPMS.os/ "noipv6
console=ttyS0,115200 dhcp ks=http://gs2repo.gs2.tradefair/mrepo/
kickstarts/fedora15.ks"

This causes some strange side affects, like the machine kickstarting
but no console, or if you reorder things, you can get a console but
the kickstart isn''t found. Single quotes vs double quotes
isn''t much
better, qemu-kvm just behaves different but still incorrect. I''ve
tried passing the arguments as an array from Puppet/Ruby, but this
won''t work as virt-install must take a single multi-word string as the
one and only argument to --extra-args or it complains. You could argue
that virt-install could be more helpful and accept multiple --extra-
args, but it''s the only tool at hand.

So, somewhere between Puppet/Ruby and Python the arguments are being
passed to virt-install strangely, maybe the way Puppet exec''s
it''s
commands? I''m not sure.

For the moment I''ve managed to work around this using a dirty, dirty
hack:

    commands :virtinstall => "/bin/sh"
    virtinstallargs = generalargs + networkargs + graphicargs +
bootargs
    virtinstallstring = "/usr/bin/virt-install " +
virtinstallargs.join('' '')
    virtinstall [ "-c", virtinstallstring ]

I''d like to be able to do it without using a shell to get the desired
escaping behaviour.

It''s a bit of a strange one, but if anyone has any suggestions I could
try I''ll give them a go.

Thanks,

-Luke

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sep 8, 3:27 am, Luke Bigum <Luke.Bi...@lmax.com>
wrote:
> Hi list,
>
> I''m making a few modifications to the puppet-virt type from GitHub
> (https://github.com/carlasouza/puppet-virt.git) and I''ve run
across an
> interesting problem with what looks like how a "command" in a
Puppet
> provider is escaping or interpreting it''s arguments. Explaining
this
> is a bit long winded so bear with me...
>
> We want to pass in multiple kernel boot arguments when doing a KVM
> virt-install, a simple example is below. --extra-args takes a single
> argument and can''t be specified twice so if you want to specify
> multiple you have to group them together as one string like so:
>
>     virt-install --name foo --extra-args "dhcp console=ttyS0
ks=http://
> ks/foo.ks"
>
> This gets translated by virt-install (Python) into individual
> arguments after "-append" to the qemu-kvm binary like so:
>
>     /usr/bin/qemu-kvm <heaps of other arguments> -append
method=http://
> gs2repo.gs2.tradefair/mrepo/latest/fedora15-x86_64/RPMS.os/ noipv6
> console=ttyS0,115200 dhcp ks=http://gs2repo.gs2.tradefair/mrepo/
> kickstarts/fedora15.ks
>
> So the transition of arguments is "Shell -> Python ->
Binary" and
> everything works grand. Now, trying to replicate the same thing in
> Puppet has proved difficult. Nearest I can tell with my limited
> Puppet / Ruby knowledge, the argument list is being handled
> differently and so what gets put on the qemu-kvm binary''s command
line
> is one single string of arguments that aren''t expanded out
properly
> and so qemu-kvm doesn''t function as we intend.
>
> Here''s how I''m trying to do it in Puppet. A simplified
virt resource
> in a manifest, note the double sets of quotes:
>
>     virt { "foo":
>       ensure => running,
>       boot_options  => ''"noipv6 console=ttyS0,115200 dhcp
ks=http://
> gs2repo.gs2.tradefair/mrepo/kickstarts/fedora15.ks"'',
>     }
>
> This gets translated into the provider (heavily simplified below for
> brevity), arguments are pulled from resource parameters and translated
> into arrays and appended as arguments to the Puppet/Ruby virt-install
> command:
>
>     Puppet::Type.type(:virt).provide(:libvirt) do
>         commands :virtinstall => "/usr/bin/virt-install"
>         bootargs = ["--extra-args", resource[:boot_options] ]
>         def install
>             virtinstall generalargs + networkargs + graphicargs +
> bootargs
>
> Applying this manifest and looking at the debug output, it seems to be
> escaped correctly:
>
>     debug: Puppet::Type::Virt::ProviderLibvirt: Executing
''/usr/bin/
> virt-install --name foo  --extra-args "dhcp console=ttyS0
ks=http://ks/
> foo.ks"''
>     notice: /Stage[main]//Virt[foo]/ensure: created
>
> However if you then check to see what qemu-kvm command is now running,
> it''s still got the shell quotes in the argument list:
>
>     /usr/bin/qemu-kvm <heaps of other arguments> -append
method=http://
> gs2repo.gs2.tradefair/mrepo/latest/fedora15-x86_64/RPMS.os/ "noipv6
> console=ttyS0,115200 dhcp ks=http://gs2repo.gs2.tradefair/mrepo/
> kickstarts/fedora15.ks"
>
> This causes some strange side affects, like the machine kickstarting
> but no console, or if you reorder things, you can get a console but
> the kickstart isn''t found. Single quotes vs double quotes
isn''t much
> better, qemu-kvm just behaves different but still incorrect. I''ve
> tried passing the arguments as an array from Puppet/Ruby, but this
> won''t work as virt-install must take a single multi-word string as
the
> one and only argument to --extra-args or it complains. You could argue
> that virt-install could be more helpful and accept multiple --extra-
> args, but it''s the only tool at hand.
>
> So, somewhere between Puppet/Ruby and Python the arguments are being
> passed to virt-install strangely, maybe the way Puppet exec''s
it''s
> commands? I''m not sure.
>
> For the moment I''ve managed to work around this using a dirty,
dirty
> hack:
>
>     commands :virtinstall => "/bin/sh"
>     virtinstallargs = generalargs + networkargs + graphicargs +
> bootargs
>     virtinstallstring = "/usr/bin/virt-install " +
> virtinstallargs.join('' '')
>     virtinstall [ "-c", virtinstallstring ]
>
> I''d like to be able to do it without using a shell to get the
desired
> escaping behaviour.
>
> It''s a bit of a strange one, but if anyone has any suggestions I
could
> try I''ll give them a go.
It looks like the LibVirt provider is launching virt-install directly
instead of via the shell (i.e. it is forking and then exec''ing /usr/
bin/virt-install, instead of, for instance, using the system() command
or a %x string).  If it is indeed doing that then it should not be
quoting any arguments, even if though they contain whitespace or shell
metacharacters, because the arguments are not processed by the shell.

Your hack solves the problem by forcing the provider to run virt-
install via the shell, which isn''t that bad.  Better, however, would
be to make the provider do the right thing with the arguments in the
first place.  Since you provide only a simplified approximation to the
provider code, I can''t tell what would be needed to accomplish that.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Ahhhh! That makes sense, I should be thinking like C/C++ execv, one
array element per argument, no matter what it''s string content.
Dropping the quotes in the parameter sting and using the original Ruby
command exec works fine. I should have tried that before - I was
confusing myself by trying too hard to emulate what the shell version
of virt-install looked like.

Thanks for the nudge in the right direction, John :-)

-Luke

On Sep 8, 3:08 pm, jcbollinger <John.Bollin...@stJude.org>
wrote:
>
> It looks like the LibVirt provider is launching virt-install directly
> instead of via the shell (i.e. it is forking and then exec''ing
/usr/
> bin/virt-install, instead of, for instance, using the system() command
> or a %x string).  If it is indeed doing that then it should not be
> quoting any arguments, even if though they contain whitespace or shell
> metacharacters, because the arguments are not processed by the shell.
>
> Your hack solves the problem by forcing the provider to run virt-
> install via the shell, which isn''t that bad.  Better, however,
would
> be to make the provider do the right thing with the arguments in the
> first place.  Since you provide only a simplified approximation to the
> provider code, I can''t tell what would be needed to accomplish
that.
>
> John
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.