Puppet users - Jul 2013 - username/password combo for custom provider (couchbase)

Hi,

I''m trying to program custom type for managing Couchbase buckets.

Problem is that every RW operation to Couchbase cluster (creating,
deleting, resizing bucket) requires admin privileges (username/password).


Do you have any suggestion how should I pass the username/password to
provider? If I make it a param, then every resource of type
''couchbucket'' will need to have ''username
=>'' and ''password =>'' params,
which will (probably) be the same across the board.

Now, to circumvent that repetition, I''m thinking of maybe setting
something like:

Couchbucket {
  username => $::couchbase::admin_user,
  password => $::couchbase::admin_password,
}

in the ''::couchbase'' class. Offcourse there will be similar
settings for
hostname and port of the couchnode, and probably something like purge =>
true.


But still, if someone wants to use the type without including the base
class, this is obviously not a viable option, because he will have to
address the username/password in every couchbucket resource definition.

So, what I am interested in is your opinion about how to proceed with
this one.



-- 
Jakov Sosic
www.srce.unizg.hr

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Crosspost to puppet-dev to get better feedback.

On Sat, Jul 13, 2013 at 7:54 AM, Jakov Sosic <jsosic@srce.hr> wrote:
> I''m trying to program custom type for managing Couchbase buckets.
>
> Problem is that every RW operation to Couchbase cluster (creating,
> deleting, resizing bucket) requires admin privileges (username/password).
>
>
> Do you have any suggestion how should I pass the username/password to
> provider? If I make it a param, then every resource of type
> ''couchbucket'' will need to have ''username
=>'' and ''password =>'' params,
> which will (probably) be the same across the board.
>
There''s two options, one to store the username/password on server like
mysql resource (~/my.cnf), or in catalog. I had discussions where some
users prefer the former, but I''m in the later camp.

Now, to circumvent that repetition, I''m thinking of maybe
setting
> something like:
>
> Couchbucket {
>   username => $::couchbase::admin_user,
>   password => $::couchbase::admin_password,
> }
>
> in the ''::couchbase'' class. Offcourse there will be
similar settings for
> hostname and port of the couchnode, and probably something like purge =>
> true.
>
This works, just slightly cumbersome.

But still, if someone wants to use the type without including the
base
> class, this is obviously not a viable option, because he will have to
> address the username/password in every couchbucket resource definition.
>
> So, what I am interested in is your opinion about how to proceed with
> this one.
>
In vmware modules, we use a transport resource to specify connectivity.

https://github.com/vmware/vmware-vcsa/blob/master/manifests/init.pp#L44-L51

Right now, transport is a resource and metaparameter, but I think
there''s
some value perhaps to turn it back to a regular parameter so we can specify
defaults.

Puppet::Type.newtype(:couchbucket) do
...
  newparam(:transport) do
    defaultto ''couch''
  end
end

This should allow a one time specification of username password for all
resources:

transport { ''couch''
  username => ...
  password => ...,
}

couchbucket { ''a'':
  #transport => ''couch'' implied
}

You can find examples in the vmware module how we search the catalog to
find the transport data to initialize and reuse connection.

I''ve intended to convert transport to a stand alone module and support
the
following usage:

1. simple store for username/password (for your exact usage).
2. transport ssh.
3. transport rest.
4. transport soap.

Maybe that would be of interest? I have some other ideas about how to take
advantage of the catalog as data, instead of just resource, but that''s
probably going to deviate from this topic.

Thanks,

Nan

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 07/13/2013 06:43 PM, Nan Liu wrote:
> There''s two options, one to store the username/password on server
like
> mysql resource (~/my.cnf), or in catalog. I had discussions where some
> users prefer the former, but I''m in the later camp.
First approach is OK, if the client program supports dot-conf file with
supplied password. But ''couchbase-cli'' doesn''t. So
I''m stuck with the
former anyway.

> This works, just slightly cumbersome. 
Actually I''ve just found that I have trouble getting it to work :(

This code is generating errors:

  Couchbucket {
    admin_user     => ''Administrator'',
    admin_password => ''secret'',
  }

  couchbucket { ''testbucket'':
    ensure         => present,
  }


Error: Could not set ''present'' on ensure: can''t
convert nil into String
at 75:/etc/puppet/modules/system/manifests/autoload/nodes/server.pp

Error: Could not set ''present'' on ensure: can''t
convert nil into String
at 75:/etc/puppet/modules/system/manifests/autoload/nodes/server.pp
...

And this code works as expected:

  couchbucket { ''testbucket'':
    ensure         => present,
    admin_user     => ''Administrator'',
    admin_password => ''secret'',
  }


So I''m buffled why is that...

> In vmware modules, we use a transport resource to specify connectivity.
> 
> https://github.com/vmware/vmware-vcsa/blob/master/manifests/init.pp#L44-L51
> 
> Right now, transport is a resource and metaparameter, but I think
> there''s some value perhaps to turn it back to a regular parameter
so we
> can specify defaults.
> 
> Puppet::Type.newtype(:couchbucket) do
> ...
>   newparam(:transport) do
>     defaultto ''couch''
>   end
> end
> 
> This should allow a one time specification of username password for all
> resources:
> 
> transport { ''couch''
>   username => ...
>   password => ...,
> }
> 
> couchbucket { ''a'':
>   #transport => ''couch'' implied
> }
This seems nice.

> You can find examples in the vmware module how we search the catalog to
> find the transport data to initialize and reuse connection.
> 
> I''ve intended to convert transport to a stand alone module and
support
> the following usage:
> 
> 1. simple store for username/password (for your exact usage).
> 2. transport ssh.
> 3. transport rest.
> 4. transport soap.
> 
> Maybe that would be of interest? I have some other ideas about how to
> take advantage of the catalog as data, instead of just resource, but
> that''s probably going to deviate from this topic. 
This would be great! It would certainly avoid collisions... Eg. two
different modules (couchbase and vmware for example) both bringing
''transport'' resource with them...

It would be a good idea to merge transport into stdlib.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 07/13/2013 06:43 PM, Nan Liu wrote:
> Puppet::Type.newtype(:couchbucket) do
> ...
>   newparam(:transport) do
>     defaultto ''couch''
>   end
> end
> 
> This should allow a one time specification of username password for all
> resources:
> 
> transport { ''couch''
>   username => ...
>   password => ...,
> }
> 
> couchbucket { ''a'':
>   #transport => ''couch'' implied
> }
> 
OK, I will basically do something like this, but I will implement my own
transport-alike resource until you guys standardize this :)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 07/15/2013 01:54 PM, Jakov Sosic wrote:
> OK, I will basically do something like this, but I will implement my own
> transport-alike resource until you guys standardize this :)
OK, I''m having a big problem ... How can I get the values from the
other
resource?

For example:

couchconnection { ''default'':
  username => ''admin'',
  password => ''admin'',
}

couchbucket { ''test'':
  ramsize    => ''64'',
  connection => Couchconnection[''default''],
}


So, how can I get the value of username in my couchbucket ''create
method''?

I would like to have something like:

  def create
    username = @resource[:connection][:username]
  end


but that obviously doesn''t work :-/


I''ve tried something along these lines, inspired by vmware-vconsole
module:


@couchconnection ||Puppet::Puppetlabs::Couchconnection.retrieve(:resource_ref
=>
resource[:connection], :catalog => resource.catalog)


but zero succes...

And ideas?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 07/15/2013 07:55 PM, Jakov Sosic wrote:
> For example:
> 
> couchconnection { ''default'':
>   username => ''admin'',
>   password => ''admin'',
> }
> 
> couchbucket { ''test'':
>   ramsize    => ''64'',
>   connection => Couchconnection[''default''],
> }
Damn, even my autorequire isn''t working properly...

couchbucket { ''test_cli'':
  ensure     => present,
  ramsize    => ''64M'',
  connection => Couchconnection[''something''],
}



And this is part of couchbucket type:

  newproperty(:connection) do
    desc ''Connection parameters for administrating Couchbase.''
    defaultto ''couch''
  end

  autorequire(:couchconnection) do
    self[:connection]
  end



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 07/15/2013 08:37 PM, Jakov Sosic wrote:
> And this is part of couchbucket type:
> 
>   newproperty(:connection) do
>     desc ''Connection parameters for administrating
Couchbase.''
>     defaultto ''couch''
>   end
> 
>   autorequire(:couchconnection) do
>     self[:connection]
>   end
Hmm, autorequire doesn''t work in my cobbler(repo|system|profile|distro)
providers either, and I''m kinda certain it did work before... Is there
a
bug related to autorequire, or is my code somehow broken?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On Mon, Jul 15, 2013 at 4:54 AM, Jakov Sosic <jsosic@srce.hr>
wrote:
>
> OK, I will basically do something like this, but I will implement my own
> transport-alike resource until you guys standardize this :)
>
Disclaimer, I don''t work at Puppet Labs anymore, and this will be a
personal project as time permits.

This would be great! It would certainly avoid collisions... Eg.
two
> different modules (couchbase and vmware for example) both bringing
> ''transport'' resource with them...
>
Yep, that''s the goal. I''ve seen interest managing database
with puppet
device, and a common module could provide a standard pattern for these type
of usages.

> It would be a good idea to merge transport into stdlib.
>
It''s probably too experimental at this point for me to send PR against
stdlib.
> OK, I will basically do something like this, but I will implement my own
> > transport-alike resource until you guys standardize this :)
>
> OK, I''m having a big problem ... How can I get the values from the
other
> resource?
>
> For example:
>
> couchconnection { ''default'':
>   username => ''admin'',
>   password => ''admin'',
> }
>
> couchbucket { ''test'':
>   ramsize    => ''64'',
>   connection => Couchconnection[''default''],
> }
>
>
> So, how can I get the value of username in my couchbucket ''create
method''?
>
> I would like to have something like:
>
>   def create
>     username = @resource[:connection][:username]
>   end
>
>
> but that obviously doesn''t work :-/
>
The one mistake in my original email was it was searching by resource by
name instead of reference, so perhaps it''s as simple as fixing:

defaultto ''Transport[couch]'' # or maybe defaultto
"Transport[''couch'']"

If that''s not the issue, then I''m not sure. I''ll try
to put together the
transport module when I get some spare time.

Thanks,

Nan

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 07/16/2013 02:56 AM, Nan Liu wrote:
> If that''s not the issue, then I''m not sure. I''ll
try to put together
> the transport module when I get some spare time.
Reference is not a problem, but fetching parameters from that resource
is... As I can see in vmware modules, there is a
PuppetX::PuppetLabs::Transport module with retrieve method, and I''m not
sure if I''m supposed to write something along that lines for my
''transport-alike'' resource also?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 07/16/2013 02:56 AM, Nan Liu wrote:
> Disclaimer, I don''t work at Puppet Labs anymore, and this will be
a
> personal project as time permits. 
Also, best of luck in the future! And thank you for your contribution so
far. You''ve been really helpful (at least to me) in the process of
learning custom types and how are they supposed to work...

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On Tue, Jul 16, 2013 at 2:09 AM, Jakov Sosic <jsosic@srce.hr> wrote:
> On 07/16/2013 02:56 AM, Nan Liu wrote:
>
> > If that''s not the issue, then I''m not sure.
I''ll try to put together
> > the transport module when I get some spare time.
>
> Reference is not a problem, but fetching parameters from that resource
> is... As I can see in vmware modules, there is a
> PuppetX::PuppetLabs::Transport module with retrieve method, and
I''m not
> sure if I''m supposed to write something along that lines for my
> ''transport-alike'' resource also?

This was long overdue, but finally had a opportunity to extend native
puppet resource to manage remote resource via transport. The specific
implementation uses ssh:

  transport { ''esx'':
    username => $username,
    password => $password,
    server => $server,
  }

  service { ''SSH'':
    ensure => ''running'',
    provider => ''ssh'',
    transport => Transport[''esx''],
  }

This was written to run against an ESXi server to update some sshd config:
https://github.com/vmware/vmware-vmware_lib

Thanks,

Nan

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CACqVBqCPKx%2Bnckx0Kb1fx1RvPry%2BpvYDcKBhNf9BpHk8RGDAkg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.