Robert Atkins
2011-Nov-01 10:15 UTC
[Puppet Users] What''s the canonical way to enforce permissions/ownership on a directory subtree?
I''ve just tried this (we assume /opt/jetty-6.1.26 already exists): file { "/opt/jetty-6.1.26": owner => "jetty", group => "users", recurse => true, } ... but it''s taking an *age*. What''s the Right Way? Cheers, Robert. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
madAndroid
2011-Nov-01 13:34 UTC
[Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
how big is the directory structure? we''ve had incredibly painful experiences trying to manage directory perms/ownerships on large directory trees... so much so that we only set the perms on a few of the top level directories and left the rest it''s something to do with needing to do an md5 and stat on every file in the tree that slows it down how important is it that the permissions are forced? we decided eventually that the file attr wouldn''t necessarily change unless someone had access to the directory via ssh.. and only the sys admins do to the server in question anyway hopefully there''s a better way of doing it ... calling all gurus? cheers, Andrew On Nov 1, 11:15 am, Robert Atkins <snikta.tre...@gmail.com> wrote:> I''ve just tried this (we assume /opt/jetty-6.1.26 already exists): > > file { "/opt/jetty-6.1.26": > owner => "jetty", > group => "users", > recurse => true, > > } > > ... but it''s taking an *age*. What''s the Right Way? > > Cheers, Robert.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Luke Bigum
2011-Nov-01 13:42 UTC
[Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
It also has to do with Puppet''s implementation of File resources: it creates in memory Ruby objects for every file and directory it finds recursively, so combine that with the md5 summing and you''ll blow out your CPU and memory usage very quickly. I''ve done something like this in the past: $path="/opt/jetty-6.1.26" exec { "enforce ${path} permissions": command => "/usr/bin/find ${path} ! -uid jetty -exec chown jetty {} \;", onlyif => "test $(/usr/bin/find ${path} -uid jetty | wc -l) -gt 0", } I wouldn''t call it elegant, but much faster. On Nov 1, 1:34 pm, madAndroid <andrewsta...@gmail.com> wrote:> how big is the directory structure? > > we''ve had incredibly painful experiences trying to manage directory > perms/ownerships on large directory trees... > so much so that we only set the perms on a few of the top level > directories and left the rest > > it''s something to do with needing to do an md5 and stat on every file > in the tree that slows it down > > how important is it that the permissions are forced? > we decided eventually that the file attr wouldn''t necessarily change > unless someone had access to the directory via ssh.. > and only the sys admins do to the server in question anyway > > hopefully there''s a better way of doing it ... calling all gurus? > > cheers, > Andrew > > On Nov 1, 11:15 am, Robert Atkins <snikta.tre...@gmail.com> wrote: > > > > > > > > > I''ve just tried this (we assume /opt/jetty-6.1.26 already exists): > > > file { "/opt/jetty-6.1.26": > > owner => "jetty", > > group => "users", > > recurse => true, > > > } > > > ... but it''s taking an *age*. What''s the Right Way? > > > Cheers, Robert.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Robert Atkins
2011-Nov-02 07:07 UTC
[Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
On Nov 1, 9:42 pm, Luke Bigum <Luke.Bi...@lmax.com> wrote:> It also has to do with Puppet''s implementation of File resources: it > creates in memory Ruby objects for every file and directory it finds[...] From some other reading I gathered this was what it''s doing. The jetty dir is up to half a dozen levels deep with almost 4500 files. I''ll use the solution you mentioned below unless there''s a better way of doing it. Cheers, Robert. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Robert Atkins
2011-Nov-02 08:30 UTC
[Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
Thanks for your solution Luke, this is much faster. For future reference here''s the modification I made (there were a couple of typos and/or I''ve got a different dialect of find): exec { "enforce ${jetty_install_dir} permissions": command => "/usr/bin/find ${jetty_install_dir} ! -user jetty -o ! - group users -exec chown jetty:users {} \\;", onlyif => "/usr/bin/test $(/usr/bin/find ${jetty_install_dir} ! - user jetty -o ! -group users | wc -l) -gt 0", } Cheers, Robert. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Christopher Wood
2011-Nov-02 11:33 UTC
Re: [Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
It''s worth mentioning that find -exec this way forks a separate copy of find for each file. You''ll notice how much slower this is on a really large set of files (possibly larger than yours). If you have a recursion-capable chown it''s quicker to use chown -R. On Wed, Nov 02, 2011 at 01:30:04AM -0700, Robert Atkins wrote:> Thanks for your solution Luke, this is much faster. For future > reference here''s the modification I made (there were a couple of typos > and/or I''ve got a different dialect of find): > > exec { "enforce ${jetty_install_dir} permissions": > command => "/usr/bin/find ${jetty_install_dir} ! -user jetty -o ! - > group users -exec chown jetty:users {} \\;", > onlyif => "/usr/bin/test $(/usr/bin/find ${jetty_install_dir} ! - > user jetty -o ! -group users | wc -l) -gt 0", > } > > Cheers, Robert. > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Robert Atkins
2011-Nov-04 05:39 UTC
[Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
On Nov 2, 7:33 pm, Christopher Wood <christopher_w...@pobox.com> wrote:> It''s worth mentioning that find -exec this way forks a separate copy of find for each file. You''ll notice how much slower this is on a really large set of files (possibly larger than yours). If you have a recursion-capable chown it''s quicker to use chown -R.Noted, this is what I''m using now: exec { "enforce ${jetty_install_dir} permissions": command => "/bin/chown jetty:users ${jetty_install_dir}", onlyif => "/usr/bin/test $(/usr/bin/find ${jetty_install_dir} ! - user jetty -o ! -group users | wc -l) -gt 0", subscribe => [File["/opt/jetty/bin/jetty.sh"], File["/opt/jetty/ resources/log4j.xml"]], refreshonly => true, } I can see it execute ("notice: /Stage[main]//Exec[enforce /opt/ jetty-6.1.26 permissions]: Triggered ''refresh'' from 1 events") but the new files are still left with the wrong ownership. When I run the "onlyif" command manually and echo $? I get 0, which says it should run the chown. Can anyone see what I''m missing? Cheers, Robert. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Christopher Wood
2011-Nov-04 13:42 UTC
Re: [Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
On Thu, Nov 03, 2011 at 10:39:29PM -0700, Robert Atkins wrote:> On Nov 2, 7:33 pm, Christopher Wood <christopher_w...@pobox.com> > wrote: > > It''s worth mentioning that find -exec this way forks a separate copy of find for each file. You''ll notice how much slower this is on a really large set of files (possibly larger than yours). If you have a recursion-capable chown it''s quicker to use chown -R. > > Noted, this is what I''m using now: > > exec { "enforce ${jetty_install_dir} permissions": > command => "/bin/chown jetty:users ${jetty_install_dir}",I think you still want chown -R here: command => "/bin/chown -R jetty:users ${jetty_install_dir}",> onlyif => "/usr/bin/test $(/usr/bin/find ${jetty_install_dir} ! - > user jetty -o ! -group users | wc -l) -gt 0", > subscribe => [File["/opt/jetty/bin/jetty.sh"], File["/opt/jetty/ > resources/log4j.xml"]], > refreshonly => true, > } > > I can see it execute ("notice: /Stage[main]//Exec[enforce /opt/ > jetty-6.1.26 permissions]: Triggered ''refresh'' from 1 events") but the > new files are still left with the wrong ownership. When I run the > "onlyif" command manually and echo $? I get 0, which says it should > run the chown. Can anyone see what I''m missing? > > Cheers, Robert. > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Stefan Schulte
2011-Nov-04 21:56 UTC
Re: [Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
On Thu, Nov 03, 2011 at 10:39:29PM -0700, Robert Atkins wrote:> On Nov 2, 7:33 pm, Christopher Wood <christopher_w...@pobox.com> > wrote: > > It''s worth mentioning that find -exec this way forks a separate copy of find for each file. You''ll notice how much slower this is on a really large set of files (possibly larger than yours). If you have a recursion-capable chown it''s quicker to use chown -R. > > Noted, this is what I''m using now: > > exec { "enforce ${jetty_install_dir} permissions": > command => "/bin/chown jetty:users ${jetty_install_dir}", > onlyif => "/usr/bin/test $(/usr/bin/find ${jetty_install_dir} ! - > user jetty -o ! -group users | wc -l) -gt 0", > subscribe => [File["/opt/jetty/bin/jetty.sh"], File["/opt/jetty/ > resources/log4j.xml"]], > refreshonly => true, > } > > I can see it execute ("notice: /Stage[main]//Exec[enforce /opt/ > jetty-6.1.26 permissions]: Triggered ''refresh'' from 1 events") but the > new files are still left with the wrong ownership. When I run the > "onlyif" command manually and echo $? I get 0, which says it should > run the chown. Can anyone see what I''m missing? > > Cheers, Robert. >I don''t see the need to specify refreshonly => true because you already have a propert onlyif statement. Refreshonly means that the command is ONLY run when the exec resource receives a refresh event. A refresh event is triggered when the subscribed resource has changed or a changed resource has set the notify metaparamter. So in your case the refreshonly renders your onlyif useless; the chown command is only run, when /opt/jetty/bin/jetty.sh or /opt/jetty/resources/log4j.xml changes. -Stefan
Robert Atkins
2011-Nov-08 07:15 UTC
[Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
On Nov 4, 9:42 pm, Christopher Wood <christopher_w...@pobox.com> wrote:> I think you still want chown -R here: > > command => "/bin/chown -R jetty:users ${jetty_install_dir}",Of course I do, I fat-fingered it. That''s why it wasn''t working. Thanks for that (also, thanks to Stefan for the advice about refreshonly=>true.) Cheers, Robert. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Felix Frank
2011-Nov-30 14:48 UTC
Re: [Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
Hi, On 11/01/2011 02:42 PM, Luke Bigum wrote:> It also has to do with Puppet''s implementation of File resources: it > creates in memory Ruby objects for every file and directory it finds > recursively, so combine that with the md5 summing and you''ll blow out > your CPU and memory usage very quickly. I''ve done something like this > in the past:seeing as this isn''t mentioned in this thread yet: When recursing through directory trees, you most likely want to specify checksum => "none" in your file resource. This didn''t help in puppet 0.25, but since 2.6 I''ve used it to great benefit. Still, large-ish trees with lots of (small) files will still take a very long time due to the other effect noted above. Cheers, Felix -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Josh Cooper
2011-Nov-30 16:40 UTC
Re: [Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
On Wed, Nov 30, 2011 at 6:48 AM, Felix Frank < felix.frank@alumni.tu-berlin.de> wrote:> Hi, > > On 11/01/2011 02:42 PM, Luke Bigum wrote: > > It also has to do with Puppet''s implementation of File resources: it > > creates in memory Ruby objects for every file and directory it finds > > recursively, so combine that with the md5 summing and you''ll blow out > > your CPU and memory usage very quickly. I''ve done something like this > > in the past: > > seeing as this isn''t mentioned in this thread yet: > > When recursing through directory trees, you most likely want to specify > > checksum => "none" > > in your file resource. This didn''t help in puppet 0.25, but since 2.6 > I''ve used it to great benefit. > > Still, large-ish trees with lots of (small) files will still take a very > long time due to the other effect noted above. >Also if you are running 2.7.0 to 2.7.6, we recently fixed a performance issue, which you''d see when recursing large numbers of files: https://projects.puppetlabs.com/issues/9671 It''s fixed in 2.7.7 Josh -- Josh Cooper Developer, Puppet Labs -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Michael Stahnke
2011-Nov-30 18:43 UTC
Re: [Puppet Users] Re: What''s the canonical way to enforce permissions/ownership on a directory subtree?
On Wed, Nov 30, 2011 at 8:40 AM, Josh Cooper <josh@puppetlabs.com> wrote:> On Wed, Nov 30, 2011 at 6:48 AM, Felix Frank > <felix.frank@alumni.tu-berlin.de> wrote: >> >> Hi, >> >> On 11/01/2011 02:42 PM, Luke Bigum wrote: >> > It also has to do with Puppet''s implementation of File resources: it >> > creates in memory Ruby objects for every file and directory it finds >> > recursively, so combine that with the md5 summing and you''ll blow out >> > your CPU and memory usage very quickly. I''ve done something like this >> > in the past: >> >> seeing as this isn''t mentioned in this thread yet: >> >> When recursing through directory trees, you most likely want to specify >> >> checksum => "none" >> >> in your file resource. This didn''t help in puppet 0.25, but since 2.6 >> I''ve used it to great benefit. >> >> Still, large-ish trees with lots of (small) files will still take a very >> long time due to the other effect noted above. > > Also if you are running 2.7.0 to 2.7.6, we recently fixed a performance > issue, which you''d see when recursing large numbers of > files: https://projects.puppetlabs.com/issues/9671 > It''s fixed in 2.7.7Or 2.7.8rc series (coming later this week), since 2.7.7 got nixed.> Josh > -- > Josh Cooper > Developer, Puppet Labs > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.