Puppet users - Apr 2010 - Puppet & EC2: Attach an EBS volume at boot?

Is it possible, using puppet, to configure an EC2 instance so that
when it boots it attaches & mounts an EBS volume? If so, what steps
would one take to achieve this?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> Is it possible, using puppet, to configure an EC2 instance so that
> when it boots it attaches & mounts an EBS volume? If so, what steps
> would one take to achieve this?
The mounting should normally be done by your os - ie. put the entry
in /etc/fstab.

However - puppet can manage the /etc/fstab file with the
''mount''
resource if you wanted:

http://docs.puppetlabs.com/references/stable/type.html#mount

But - I''m guessing your problem is that you don''t want to
re-create
your EBS/AMI image every time you change /etc/fstab right?

If you didn''t want to save the entry in fstab and do the whole
''snapshot'' image thing (to persist your /etc/fstab entry in
your EBS/
AMI image) you could have puppet always start at bootup and let puppet
do the mounting.

Puppet will mount the file-system at any time quite happily in this
regard. Just need to make sure that your ensure line is
''mounted''. For
example:

mount {"/mnt/point":
  device => "/dev/sdc1",
  ensure => "mounted",
  ...
}

Just make sure you getting your ordering right - as most people would
be used to mounts appearing quite early in any boot sequence ... if
you manage your service starts in puppet as well, you can let puppet
do the ordering for you.

ken.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Thanks for the reply. I think the mounting part should be straight
forward.

My main problem is defining in puppet the name of the EBS volume to
attach, and having the puppet client on the EC2 instance actually
*attach* the volume after it has "spun-up".

On Apr 21, 1:07 pm, Ken <k...@bob.sh> wrote:
> > Is it possible, using puppet, to configure an EC2 instance so that
> > when it boots it attaches & mounts an EBS volume? If so, what
steps
> > would one take to achieve this?
>
> The mounting should normally be done by your os - ie. put the entry
> in /etc/fstab.
>
> However - puppet can manage the /etc/fstab file with the
''mount''
> resource if you wanted:
>
> http://docs.puppetlabs.com/references/stable/type.html#mount
>
> But - I''m guessing your problem is that you don''t want to
re-create
> your EBS/AMI image every time you change /etc/fstab right?
>
> If you didn''t want to save the entry in fstab and do the whole
> ''snapshot'' image thing (to persist your /etc/fstab entry
in your EBS/
> AMI image) you could have puppet always start at bootup and let puppet
> do the mounting.
>
> Puppet will mount the file-system at any time quite happily in this
> regard. Just need to make sure that your ensure line is
''mounted''. For
> example:
>
> mount {"/mnt/point":
>   device => "/dev/sdc1",
>   ensure => "mounted",
>   ...
>
> }
>
> Just make sure you getting your ordering right - as most people would
> be used to mounts appearing quite early in any boot sequence ... if
> you manage your service starts in puppet as well, you can let puppet
> do the ordering for you.
>
> ken.
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group
athttp://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Having the ebs vol id as a parameter in the node manifest works for us

$ebsvol = "xxx"

Then use the Mount type with that variable in your manifest file.

We''ve observed an issue when mounting EBS volumes in EC2 on our CentOS
5 AMI
though - puppetd seems to hang when executing the mount command.  Occurs if
I use the puppet Mount type, or just call mount via exec/bash script.

Seems related to the amount of data on the EBS vols (40GB+)

Running in debug provides no extra info - running 0.25.4

Thanks,

Matt

On 21 April 2010 13:44, Phillip B Oldham <phillip.oldham@gmail.com> wrote:
> Thanks for the reply. I think the mounting part should be straight
> forward.
>
> My main problem is defining in puppet the name of the EBS volume to
> attach, and having the puppet client on the EC2 instance actually
> *attach* the volume after it has "spun-up".
>
> On Apr 21, 1:07 pm, Ken <k...@bob.sh> wrote:
> > > Is it possible, using puppet, to configure an EC2 instance so
that
> > > when it boots it attaches & mounts an EBS volume? If so, what
steps
> > > would one take to achieve this?
> >
> > The mounting should normally be done by your os - ie. put the entry
> > in /etc/fstab.
> >
> > However - puppet can manage the /etc/fstab file with the
''mount''
> > resource if you wanted:
> >
> > http://docs.puppetlabs.com/references/stable/type.html#mount
> >
> > But - I''m guessing your problem is that you don''t
want to re-create
> > your EBS/AMI image every time you change /etc/fstab right?
> >
> > If you didn''t want to save the entry in fstab and do the
whole
> > ''snapshot'' image thing (to persist your /etc/fstab
entry in your EBS/
> > AMI image) you could have puppet always start at bootup and let puppet
> > do the mounting.
> >
> > Puppet will mount the file-system at any time quite happily in this
> > regard. Just need to make sure that your ensure line is
''mounted''. For
> > example:
> >
> > mount {"/mnt/point":
> >   device => "/dev/sdc1",
> >   ensure => "mounted",
> >   ...
> >
> > }
> >
> > Just make sure you getting your ordering right - as most people would
> > be used to mounts appearing quite early in any boot sequence ... if
> > you manage your service starts in puppet as well, you can let puppet
> > do the ordering for you.
> >
> > ken.
> >
> > --
> > You received this message because you are subscribed to the Google
Groups
> "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
>
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> .
> > For more options, visit this group athttp://
> groups.google.com/group/puppet-users?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
>
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> My main problem is defining in puppet the name of the EBS volume to
> attach, and having the puppet client on the EC2 instance actually
> *attach* the volume after it has "spun-up".
Is this because you want to be able to convert the EBS id to a /dev/
sdX device? So the EBS volume id is what you reference in your
configuration ...?

ken.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I didn''t seen this myself Matt. I was using 200g EBS volumes. Can you
see the mount command in action while its blocking and try to
replicate manually it with all command line options etc.

I found the provided and community Centos images where a bit poor -
and rolled my own from scratch because I wanted to use EBS volumes. I
also wanted it to be ''clean'' without someone else''s
customisations as
well :-).

ken.

On Apr 21, 3:42 pm, Matt <mattmora...@gmail.com>
wrote:
> Having the ebs vol id as a parameter in the node manifest works for us
>
> $ebsvol = "xxx"
>
> Then use the Mount type with that variable in your manifest file.
>
> We''ve observed an issue when mounting EBS volumes in EC2 on our
CentOS 5 AMI
> though - puppetd seems to hang when executing the mount command.  Occurs if
> I use the puppet Mount type, or just call mount via exec/bash script.
>
> Seems related to the amount of data on the EBS vols (40GB+)
>
> Running in debug provides no extra info - running 0.25.4
>
> Thanks,
>
> Matt
>
> On 21 April 2010 13:44, Phillip B Oldham <phillip.old...@gmail.com>
wrote:
>
>
>
>
>
> > Thanks for the reply. I think the mounting part should be straight
> > forward.
>
> > My main problem is defining in puppet the name of the EBS volume to
> > attach, and having the puppet client on the EC2 instance actually
> > *attach* the volume after it has "spun-up".
>
> > On Apr 21, 1:07 pm, Ken <k...@bob.sh> wrote:
> > > > Is it possible, using puppet, to configure an EC2 instance
so that
> > > > when it boots it attaches & mounts an EBS volume? If so,
what steps
> > > > would one take to achieve this?
>
> > > The mounting should normally be done by your os - ie. put the
entry
> > > in /etc/fstab.
>
> > > However - puppet can manage the /etc/fstab file with the
''mount''
> > > resource if you wanted:
>
> > >http://docs.puppetlabs.com/references/stable/type.html#mount
>
> > > But - I''m guessing your problem is that you
don''t want to re-create
> > > your EBS/AMI image every time you change /etc/fstab right?
>
> > > If you didn''t want to save the entry in fstab and do the
whole
> > > ''snapshot'' image thing (to persist your
/etc/fstab entry in your EBS/
> > > AMI image) you could have puppet always start at bootup and let
puppet
> > > do the mounting.
>
> > > Puppet will mount the file-system at any time quite happily in
this
> > > regard. Just need to make sure that your ensure line is
''mounted''. For
> > > example:
>
> > > mount {"/mnt/point":
> > >   device => "/dev/sdc1",
> > >   ensure => "mounted",
> > >   ...
>
> > > }
>
> > > Just make sure you getting your ordering right - as most people
would
> > > be used to mounts appearing quite early in any boot sequence ...
if
> > > you manage your service starts in puppet as well, you can let
puppet
> > > do the ordering for you.
>
> > > ken.
>
> > > --
> > > You received this message because you are subscribed to the
Google Groups
> > "Puppet Users" group.
> > > To post to this group, send email to
puppet-users@googlegroups.com.
> > > To unsubscribe from this group, send email to
> >
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@google
groups.com>
> > .
> > > For more options, visit this group athttp://
> > groups.google.com/group/puppet-users?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google
Groups
> > "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> >
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@google
groups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/puppet-users?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group
athttp://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi Ken,

Were those 200GB populated with 50GB+ of data?

Yes it can be mounted with the same options on the command line fine.  Also
if the EBS vol is already mounted it still hangs when it gets to it (remount
thingy).  iostat shows high I/O on the volume when this happens, so I
thought it was due to puppet trying to back it up - but file ( backup =>
false } or changing it to a mkdir -p on the directory it gets mounted on
made no difference.

I thought it could be related to EBS volumes created from snapshots (i.e.
the EBS vol isn''t populated fully yet) but it doesn''t appear
to.

I agree re the community CentOS images, we also rolled our own AMI to keep
in sync with our bare metal repo.

It doesn''t hang on most of our EBS mounts, but I know we have a few in
live
where we have to kill puppet and finish the catalogue manually.  I
couldn''t
think of any other way to try and narrow it down to provide any more info.

Matt

On 21 April 2010 18:09, Ken <ken@bob.sh> wrote:
> I didn''t seen this myself Matt. I was using 200g EBS volumes. Can
you
> see the mount command in action while its blocking and try to
> replicate manually it with all command line options etc.
>
> I found the provided and community Centos images where a bit poor -
> and rolled my own from scratch because I wanted to use EBS volumes. I
> also wanted it to be ''clean'' without someone
else''s customisations as
> well :-).
>
> ken.
>
> On Apr 21, 3:42 pm, Matt <mattmora...@gmail.com> wrote:
> > Having the ebs vol id as a parameter in the node manifest works for us
> >
> > $ebsvol = "xxx"
> >
> > Then use the Mount type with that variable in your manifest file.
> >
> > We''ve observed an issue when mounting EBS volumes in EC2 on
our CentOS 5
> AMI
> > though - puppetd seems to hang when executing the mount command. 
Occurs
> if
> > I use the puppet Mount type, or just call mount via exec/bash script.
> >
> > Seems related to the amount of data on the EBS vols (40GB+)
> >
> > Running in debug provides no extra info - running 0.25.4
> >
> > Thanks,
> >
> > Matt
> >
> > On 21 April 2010 13:44, Phillip B Oldham
<phillip.old...@gmail.com>
> wrote:
> >
> >
> >
> >
> >
> > > Thanks for the reply. I think the mounting part should be
straight
> > > forward.
> >
> > > My main problem is defining in puppet the name of the EBS volume
to
> > > attach, and having the puppet client on the EC2 instance actually
> > > *attach* the volume after it has "spun-up".
> >
> > > On Apr 21, 1:07 pm, Ken <k...@bob.sh> wrote:
> > > > > Is it possible, using puppet, to configure an EC2
instance so that
> > > > > when it boots it attaches & mounts an EBS volume?
If so, what steps
> > > > > would one take to achieve this?
> >
> > > > The mounting should normally be done by your os - ie. put
the entry
> > > > in /etc/fstab.
> >
> > > > However - puppet can manage the /etc/fstab file with the
''mount''
> > > > resource if you wanted:
> >
> > > >http://docs.puppetlabs.com/references/stable/type.html#mount
> >
> > > > But - I''m guessing your problem is that you
don''t want to re-create
> > > > your EBS/AMI image every time you change /etc/fstab right?
> >
> > > > If you didn''t want to save the entry in fstab and
do the whole
> > > > ''snapshot'' image thing (to persist your
/etc/fstab entry in your EBS/
> > > > AMI image) you could have puppet always start at bootup and
let
> puppet
> > > > do the mounting.
> >
> > > > Puppet will mount the file-system at any time quite happily
in this
> > > > regard. Just need to make sure that your ensure line is
''mounted''.
> For
> > > > example:
> >
> > > > mount {"/mnt/point":
> > > >   device => "/dev/sdc1",
> > > >   ensure => "mounted",
> > > >   ...
> >
> > > > }
> >
> > > > Just make sure you getting your ordering right - as most
people would
> > > > be used to mounts appearing quite early in any boot sequence
... if
> > > > you manage your service starts in puppet as well, you can
let puppet
> > > > do the ordering for you.
> >
> > > > ken.
> >
> > > > --
> > > > You received this message because you are subscribed to the
Google
> Groups
> > > "Puppet Users" group.
> > > > To post to this group, send email to
puppet-users@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > >
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> <puppet-users%2Bunsubscribe@google groups.com>
> > > .
> > > > For more options, visit this group athttp://
> > > groups.google.com/group/puppet-users?hl=en.
> >
> > > --
> > > You received this message because you are subscribed to the
Google
> Groups
> > > "Puppet Users" group.
> > > To post to this group, send email to
puppet-users@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > >
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> <puppet-users%2Bunsubscribe@google groups.com>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/puppet-users?hl=en.
> >
> > --
> > You received this message because you are subscribed to the Google
Groups
> "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
>
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> .
> > For more options, visit this group athttp://
> groups.google.com/group/puppet-users?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
>
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Apr 21, 6:06 pm, Ken <k...@bob.sh> wrote:
> > My main problem is defining in puppet the name of the EBS volume to
> > attach, and having the puppet client on the EC2 instance actually
> > *attach* the volume after it has "spun-up".
>
> Is this because you want to be able to convert the EBS id to a /dev/
> sdX device? So the EBS volume id is what you reference in your
> configuration ...?
Not exactly. I''m working with an OpenSolaris instance. What
I''m hoping
puppet can do is, when the instance starts up, issue the ec2 commands
to attach a specified EBS volume to itself then keep checking whether
the drive is available before issuing the ZFS commands to "import" the
newly-attached drive. Would this be possible? If so, which sections of
the documentation should I be focussing on?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 22 April 2010 08:12, Phillip B Oldham <phillip.oldham@gmail.com> wrote:
> On Apr 21, 6:06 pm, Ken <k...@bob.sh> wrote:
> > > My main problem is defining in puppet the name of the EBS volume
to
> > > attach, and having the puppet client on the EC2 instance actually
> > > *attach* the volume after it has "spun-up".
> >
> > Is this because you want to be able to convert the EBS id to a /dev/
> > sdX device? So the EBS volume id is what you reference in your
> > configuration ...?
>
> Not exactly. I''m working with an OpenSolaris instance. What
I''m hoping
> puppet can do is, when the instance starts up, issue the ec2 commands
> to attach a specified EBS volume to itself then keep checking whether
> the drive is available before issuing the ZFS commands to
"import" the
> newly-attached drive. Would this be possible? If so, which sections of
> the documentation should I be focussing on?
>
> I don''t know much about solaris, but you could use the Exec type
to issue
the EC2 commands to attach the EBS volume to say /dev/sdx, then use the
Mount type or another Exec to mount/import the ZFS system.

If you look at using EC2 facters
http://projects.puppetlabs.com/projects/puppet/wiki/Amazon_Ec2_Patterns then
your instance will even know it''s own instance id etc.

I personally prefer to set-up the instances externally with some other tool,
saves having AWS credentials on the EC2 instance.

Matt

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Apr 22, 9:26 am, Matt <mattmora...@gmail.com>
wrote:
> I personally prefer to set-up the instances externally with some other
tool,
> saves having AWS credentials on the EC2 instance.
So there''s no way for the puppet config on the master to have the AWS
credentials and pass them down to the instance/client?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> I personally prefer to set-up the instances externally with some other
tool,
> saves having AWS credentials on the EC2 instance.
I agree. Its a concern that each box will have so much control over
not only themselves but other instances that the key has access to. If
that 1 box is compromised then the would-be hacker can take control of
all the other boxes as well.

ken.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> Were those 200GB populated with 50GB+ of data?
Yep.
> It doesn''t hang on most of our EBS mounts, but I know we have a
few in live
> where we have to kill puppet and finish the catalogue manually.  I
couldn''t
> think of any other way to try and narrow it down to provide any more info.
I can''t see anything in the source for ''mount'' that
would be extra.
Its really only a wrapper around managing your /etc/fstab and running
the mount command. So I can only assume its something else doing it I
guess?

At that point I''d drop to strace on the stuck puppetd and see what its
doing ... if its a large IO it should be obvious as you will see it
read/write on a particular file descriptor.

ken.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Apr 22, 12:04 pm, Ken <k...@bob.sh> wrote:
> > I personally prefer to set-up the instances externally with some other
tool,
> > saves having AWS credentials on the EC2 instance.
>
> I agree. Its a concern that each box will have so much control over
> not only themselves but other instances that the key has access to. If
> that 1 box is compromised then the would-be hacker can take control of
> all the other boxes as well.
Can the puppet master issue the ec2 commands from the main server
then? If not, any suggestions for other tools with which I can
automate this part of the process?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 22 April 2010 12:19, Phillip B Oldham <phillip.oldham@gmail.com> wrote:
> On Apr 22, 12:04 pm, Ken <k...@bob.sh> wrote:
> > > I personally prefer to set-up the instances externally with some
other
> tool,
> > > saves having AWS credentials on the EC2 instance.
> >
> > I agree. Its a concern that each box will have so much control over
> > not only themselves but other instances that the key has access to. If
> > that 1 box is compromised then the would-be hacker can take control of
> > all the other boxes as well.
>
> Can the puppet master issue the ec2 commands from the main server
> then? If not, any suggestions for other tools with which I can
> automate this part of the process?
>
>
I''d recommend using Ruby with the right aws library.  You can then
build the
node manifest files in your ruby code and deploy to a directory on your
puppet master.

Matt

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
>
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 22 April 2010 12:09, Ken <ken@bob.sh> wrote:
> > Were those 200GB populated with 50GB+ of data?
>
> Yep.
>
> > It doesn''t hang on most of our EBS mounts, but I know we have
a few in
> live
> > where we have to kill puppet and finish the catalogue manually.  I
> couldn''t
> > think of any other way to try and narrow it down to provide any more
> info.
>
> I can''t see anything in the source for ''mount''
that would be extra.
> Its really only a wrapper around managing your /etc/fstab and running
> the mount command. So I can only assume its something else doing it I
> guess?
>
> At that point I''d drop to strace on the stuck puppetd and see what
its
> doing ... if its a large IO it should be obvious as you will see it
> read/write on a particular file descriptor.
>
> I can reproduce it easily on one of our production nodes, so i''ll
look at
running strace on it.

Matt

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hrm. You mean can the client initiate a command on the puppetmaster?
Yes - but only really at compile time ... so its probably not the
right way to do it.

Do you have a ''virtualisation manager app'' that spawns your
amazon
instances or are you using the web GUI amazon provides?

I think architecturally your ''virtualisation manager app'' is
the right
place to do disk attaches - whatever form that takes. That can make
the necessary calls at instance boot time to attach the right drives
as well.

And if you still want the client to trigger the attach ... if your
''virtualisation manager app'' supports an API - the new server
could
call back to that API. The idea being that whatever virt manager you
use would have more fine grained access control then the amazon web
service does.

ken.

On Apr 22, 12:19 pm, Phillip B Oldham <phillip.old...@gmail.com>
wrote:
> On Apr 22, 12:04 pm, Ken <k...@bob.sh> wrote:
>
> > > I personally prefer to set-up the instances externally with some
other tool,
> > > saves having AWS credentials on the EC2 instance.
>
> > I agree. Its a concern that each box will have so much control over
> > not only themselves but other instances that the key has access to. If
> > that 1 box is compromised then the would-be hacker can take control of
> > all the other boxes as well.
>
> Can the puppet master issue the ec2 commands from the main server
> then? If not, any suggestions for other tools with which I can
> automate this part of the process?
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group
athttp://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

So, am I correct in thinking then that the following workflow *can''t*
be done with puppet?

1) The EC2 instance starts up, and the puppet client on the instance
connects to the puppet master.
2) The puppet client receives the EC2 authentication credentials (env
vars, certs, etc) from the master.
3) The master tells the client which EBS volums to attach at which
mount points.
4) The client uses the authentication credentials to execute the EC2
commands to attach the volumes.
5) The client then removes the EC2 creds (unset env vars, rm certs,
etc) for security.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/23/2010 10:07 AM, Phillip B Oldham wrote:
> So, am I correct in thinking then that the following workflow
*can''t*
> be done with puppet?
> 
> 1) The EC2 instance starts up, and the puppet client on the instance
> connects to the puppet master.
> 2) The puppet client receives the EC2 authentication credentials (env
> vars, certs, etc) from the master.
> 3) The master tells the client which EBS volums to attach at which
> mount points.
> 4) The client uses the authentication credentials to execute the EC2
> commands to attach the volumes.
> 5) The client then removes the EC2 creds (unset env vars, rm certs,
> etc) for security.
I didn''t closely follow the thread, but I see no reason why not:

1)
1.1) you upload an image with the infos to connect your instances to
your master
1.2) master gets an autosigned entry for every instance you fire up
2)
2.1) your modules contain a fact to decide whether your volume is setup
correctly or not
2.2) if it is not setup correctly it includes a class to setup the volume
2.3) the master provides within that class the credentials (ie. via
extlookup)
3)
3.1) the master provides that info within the previously outlined class
4)
4.1) the client applies the included class which does that.
5)
5.1) additionally you would include a class which cleans up everything
_after_ the setup, this means that:
5.2) if the fact tells you to apply the "settingup"-class this cleanup
class would require the "settingup"-class
5.3) for security reasons if the fact says that everything is setup you
include only the "cleanup" class, so that you are sure that you
won''t
have any leftovers around.

it''s quite straight forward. What are the tricks?

* write a custom fact to signal whether the volume is correctly setup or not
* write 2 classes, where one setup things and the other one does the cleanup
* depend the cleanup stuff on the setup class _if_ the fact is set to setup

sounds feasible, not?

cheers pete

PS: I assume there are also other ways, would be interesting to hear how
other people solve that problem.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvRb5UACgkQbwltcAfKi3/lVQCdFOcORgnSmqr2B9Z1oR+bIdJf
BDkAn0jYyceWLQuh3FLzbqWmdzARflfg
=Vy/N
-----END PGP SIGNATURE-----

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.