thr3ads.net - Puppet users - [Puppet Users] Chmoding files in directory with around 33000 files is slow [Dec 2011]

If this information is useful, please help other people find it:
Share via:

Idar Borlaug

2011-Dec-02 15:41 UTC

[Puppet Users] Chmoding files in directory with around 33000 files is slow

Hi

Just wanted to know if anyhow has a great workaround for modifing user
and group in a folder with 33000 files.
It takes 5 minutes to chmod one file.

file { "/some/dir" :
     ensure => directory,
     recurse => true,
     group => ''jboss'',
     owner => ''jboss''
}

I am using puppet 2.6.4 on the server, might this be fixed in a later version?

-- 
Idar

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Jo Rhett

2011-Dec-02 19:40 UTC

head link

Re: [Puppet Users] Chmoding files in directory with around 33000 files is slow

That''s a directory reading problem that every unix-based program will
have.  You need a different filesystem if you really want 33k files in one
directory.

On Dec 2, 2011, at 7:41 AM, Idar Borlaug wrote:> Just wanted to know if anyhow has a great workaround for modifing user
> and group in a folder with 33000 files.
> It takes 5 minutes to chmod one file.
> 
> file { "/some/dir" :
>     ensure => directory,
>     recurse => true,
>     group => ''jboss'',
>     owner => ''jboss''
> }
> 
> I am using puppet 2.6.4 on the server, might this be fixed in a later
version?
> 
> -- 
> Idar
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
> 
-- 
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source and other
randomness

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Idar Borlaug

2011-Dec-02 21:06 UTC

head link

Re: [Puppet Users] Chmoding files in directory with around 33000 files is slow

On 2 December 2011 20:40, Jo Rhett <jrhett@netconsonance.com>
wrote:> That''s a directory reading problem that every unix-based program
will have.
>  You need a different filesystem if you really want 33k files in one
> directory.
Reading the directory isn''t slow at all. Theres only 33000 files in
it. ls goes in 2 sec. chmod unix command executes in under 2 secs.

-- 
Idar

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Nigel Kersten

2011-Dec-02 21:57 UTC

head link

Re: [Puppet Users] Chmoding files in directory with around 33000 files is slow

On Fri, Dec 2, 2011 at 7:41 AM, Idar Borlaug <idar.borlaug@gmail.com>
wrote:
> Hi
>
> Just wanted to know if anyhow has a great workaround for modifing user
> and group in a folder with 33000 files.
> It takes 5 minutes to chmod one file.
>
> file { "/some/dir" :
>     ensure => directory,
>     recurse => true,
>     group => ''jboss'',
>     owner => ''jboss''
> }
>
> I am using puppet 2.6.4 on the server, might this be fixed in a later
> version?
>
So you''re not concerned with the content at all?

Puppet should be smarter about this, but does it help to add:

checksum => none,

to the resource?


>
> --
> Idar
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>

-- 
Nigel Kersten
Product Manager, Puppet Labs

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Michael Stahnke

2011-Dec-02 22:03 UTC

head link

Re: [Puppet Users] Chmoding files in directory with around 33000 files is slow

On Fri, Dec 2, 2011 at 1:57 PM, Nigel Kersten <nigel@puppetlabs.com>
wrote:>
>
> On Fri, Dec 2, 2011 at 7:41 AM, Idar Borlaug <idar.borlaug@gmail.com>
wrote:
>>
>> Hi
>>
>> Just wanted to know if anyhow has a great workaround for modifing user
>> and group in a folder with 33000 files.
>> It takes 5 minutes to chmod one file.
>>
>> file { "/some/dir" :
>>     ensure => directory,
>>     recurse => true,
>>     group => ''jboss'',
>>     owner => ''jboss''
>> }
>>
>> I am using puppet 2.6.4 on the server, might this be fixed in a later
>> version?
>
>
> So you''re not concerned with the content at all?
>
> Puppet should be smarter about this, but does it help to add:
>
> checksum => none,
>
> to the resource?
>
>
>>
>>
>> --
>> Idar
>>
>> --
>> You received this message because you are subscribed to the Google
Groups
>> "Puppet Users" group.
>> To post to this group, send email to puppet-users@googlegroups.com.
>> To unsubscribe from this group, send email to
>> puppet-users+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/puppet-users?hl=en.
>>
>
>
>
> --
> Nigel Kersten
> Product Manager, Puppet Labs
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
There is also a patch in the 2.7.8rc series that addresses some of
these speed issues.

http://projects.puppetlabs.com/issues/9671

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Tony G.

2011-Dec-03 14:53 UTC

head link

Re: [Puppet Users] Chmoding files in directory with around 33000 files is slow

On Dec 2, 2011 4:03 PM, "Michael Stahnke"
<stahnma@puppetlabs.com> wrote:>
> On Fri, Dec 2, 2011 at 1:57 PM, Nigel Kersten <nigel@puppetlabs.com>
wrote:> >
> >
> > On Fri, Dec 2, 2011 at 7:41 AM, Idar Borlaug
<idar.borlaug@gmail.com>
wrote:> >>
> >> Hi
> >>
> >> Just wanted to know if anyhow has a great workaround for modifing
user
> >> and group in a folder with 33000 files.
> >> It takes 5 minutes to chmod one file.
> >>
> >> file { "/some/dir" :
> >>     ensure => directory,
> >>     recurse => true,
> >>     group => ''jboss'',
> >>     owner => ''jboss''
> >> }
> >>
> >> I am using puppet 2.6.4 on the server, might this be fixed in a
later
> >> version?
> >
> >
> > So you''re not concerned with the content at all?
> >
> > Puppet should be smarter about this, but does it help to add:
> >
> > checksum => none,
> >
> > to the resource?
> >
> >
> >>
> >>
> >> --
> >> Idar
> >>
> >> --
> >> You received this message because you are subscribed to the Google
Groups> >> "Puppet Users" group.
> >> To post to this group, send email to
puppet-users@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> puppet-users+unsubscribe@googlegroups.com.
> >> For more options, visit this group at
> >> http://groups.google.com/group/puppet-users?hl=en.
> >>
> >
> >
> >
> > --
> > Nigel Kersten
> > Product Manager, Puppet Labs
> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google
Groups> > "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> > puppet-users+unsubscribe@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/puppet-users?hl=en.
>
> There is also a patch in the 2.7.8rc series that addresses some of
> these speed issues.
>
> http://projects.puppetlabs.com/issues/9671
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.>
Another option is to create a cronjob that takes care of the chown through
puppet, that way you reduce a lot the time, my 2¢.

--
Tony
http://www.tonyskapunk.net

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Evgeny

2011-Dec-06 07:18 UTC

head link

[Puppet Users] Re: Chmoding files in directory with around 33000 files is slow

now I''m doing such stuff so

exec { "chown -R jboss:jboss /some/dir":
     logoutput => "on_failure",
}

On Dec 2, 5:41 pm, Idar Borlaug <idar.borl...@gmail.com>
wrote:> Hi
>
> Just wanted to know if anyhow has a great workaround for modifing user
> and group in a folder with 33000 files.
> It takes 5 minutes to chmod one file.
>
> file { "/some/dir" :
>      ensure => directory,
>      recurse => true,
>      group => ''jboss'',
>      owner => ''jboss''
>
> }
>
> I am using puppet 2.6.4 on the server, might this be fixed in a later
version?
>
> --
> Idar
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Puppet users - Dec 2011 - Chmoding files in directory with around 33000 files is slow

[Puppet Users] Chmoding files in directory with around 33000 files is slow

Re: [Puppet Users] Chmoding files in directory with around 33000 files is slow

Re: [Puppet Users] Chmoding files in directory with around 33000 files is slow

Re: [Puppet Users] Chmoding files in directory with around 33000 files is slow

Re: [Puppet Users] Chmoding files in directory with around 33000 files is slow

Re: [Puppet Users] Chmoding files in directory with around 33000 files is slow

[Puppet Users] Re: Chmoding files in directory with around 33000 files is slow