Bernd Adamowicz
2012-Sep-14 11:14 UTC
[Puppet Users] AW: Issue with large directory content
First, thanks for all the replies! And second, I found the solution. Let me
explain:
I tried David''s suggestion first (strace) and it really turned out that
Puppet is accessing/opening every single file below
''/repository/sonatype-work/''. Some excerpts from strace:
32124
lstat("/repository/sonatype-work/nexus/indexer/apache-snapshots-ctx/segments.gen",
{st_mode=S_IFREG|0644, st_size=20, ...}) = 0
32124
lstat("/repository/sonatype-work/nexus/indexer/apache-snapshots-ctx/_0_1.del",
{st_mode=S_IFREG|0750, st_size=9, ...} ) = 0
32124
open("/repository/sonatype-work/nexus/storage/ibiblio-maven2/org/seleniumhq/selenium/selenium-support/2.25.0/selenium-support-2.25.0.jar",
O_RDONLY) = 4
32124
open("/repository/sonatype-work/nexus/storage/ibiblio-maven2/joda-time/joda-time-hibernate/1.2/joda-time-hibernate-1.2.pom",
O_RDONLY) = 4
The other questions from all of you and finally Den''s question, which
was: "Are you trying to set any permissions inside that directory elsewhere
in the manifest?" made me rethink everything and pointed me to the right
place. A few lines down from where I thought the error was I used to have this:
file { [
"${codebase_ng::repository_mount}/${sonatype_work_dir}/nexus",
"${codebase_ng::repository_mount}/${sonatype_work_dir}/nexus/conf"]:
require =>
File["${codebase_ng::repository_mount}/${sonatype_work_dir}"],
ensure => directory,
owner => $nexus_user_id,
group => $nexus_group_id,
mode => 0750,
source => "puppet:///modules/codebase_ng/nexus/conf",
recurse => true,
purge => false,
}
This file resource was just about having configuration files in place below
''/repository/sonatype-work/nexus/conf''. But obviously the
first entry in the file array which resolves to
''/repository/sonatype-work/nexus'' was the trigger for Puppet
to start recursive scanning everything.
So, the misconfiguration was on my side (nice pitfall), but I wonder if this is
expected behaviour. Shouldn''t recursion only be done on the last entry
of the file array? I''m not sure. Maybe the PuppetLabs guys might think
about it.
However, thanks again to all who helped and pointed me to the solution!
Bernd
On 09/14/2012 09:07 AM, David Schmitt wrote:> Re: AW: [Puppet Users] AW: Issue with large directory content
>
> Use strace to take a look at what''s really happening. That should
make
> it much easier to pinpoint the culprit:
>
>
> strace -e file -f puppet agent --test
>
>
>
> Best Regards, David
>
> On 13.09.2012 14:44, Bernd Adamowicz wrote:
> > This keeps being weird. Simply thought to wait until Puppet
> > finishes, but had to quit after one and a half hour. Also tried
> > ''ensure => present'' instead of ''ensure
=> directory'' with no
> > success. No log output at all. Still investigating. But any ideas
> > still highly
> appreciated!
> >
> > Bernd
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
jcbollinger
2012-Sep-14 13:09 UTC
Re: [Puppet Users] AW: Issue with large directory content
On Friday, September 14, 2012 6:14:37 AM UTC-5, badamowicz wrote:> > The other questions from all of you and finally Den''s question, which > was: "Are you trying to set any permissions inside that directory > elsewhere in the manifest?" made me rethink everything and pointed me to > the right place. A few lines down from where I thought the error was I used > to have this: > > file { [ > "${codebase_ng::repository_mount}/${sonatype_work_dir}/nexus", > "${codebase_ng::repository_mount}/${sonatype_work_dir}/nexus/conf"]: > require => > File["${codebase_ng::repository_mount}/${sonatype_work_dir}"], > ensure => directory, > owner => $nexus_user_id, > group => $nexus_group_id, > mode => 0750, > source => "puppet:///modules/codebase_ng/nexus/conf", > recurse => true, > purge => false, > } > > This file resource was just about having configuration files in place > below ''/repository/sonatype-work/nexus/conf''. But obviously the first entry > in the file array which resolves to ''/repository/sonatype-work/nexus'' was > the trigger for Puppet to start recursive scanning everything. > > So, the misconfiguration was on my side (nice pitfall), but I wonder if > this is expected behaviour. Shouldn''t recursion only be done on the last > entry of the file array? >No. Why would you suppose that? When you write a resource declaration of the form someresource { [ ''title1'', ''title2'']: param1 => value1, param2 => value2 } it is shorthand for separate resource declarations, one for each title given, all having the specified parameters: someresource { ''title1'': param1 => value1, param2 => value2 } someresource { ''title2'': param1 => value1, param2 => value2 } The use of an array to specify multiple titles is perhaps a bit quirky in itself (though useful!), but I don''t see why anyone would suppose that resources specified that way would be assigned different parameters from each other. There is nothing specific to the File resource type here, but even if there were, why would you expect Puppet to suppose that you only wanted recursion on one of the specified resources? I don''t see it. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/m7jCxW9yv2AJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Bernd Adamowicz
2012-Sep-17 09:11 UTC
Re: [Puppet Users] AW: Issue with large directory content
On 09/14/2012 03:09 PM, jcbollinger wrote:> > > On Friday, September 14, 2012 6:14:37 AM UTC-5, badamowicz wrote: > > The other questions from all of you and finally Den''s question, which > was: "Are you trying to set any permissions inside that directory > elsewhere in the manifest?" made me rethink everything and pointed > me to the right place. A few lines down from where I thought the > error was I used to have this: > > file { [ > "${codebase_ng::repository_mount}/${sonatype_work_dir}/nexus", > "${codebase_ng::repository_mount}/${sonatype_work_dir}/nexus/conf"]: > require => > File["${codebase_ng::repository_mount}/${sonatype_work_dir}"], > ensure => directory, > owner => $nexus_user_id, > group => $nexus_group_id, > mode => 0750, > source => "puppet:///modules/codebase_ng/nexus/conf", > recurse => true, > purge => false, > } > > This file resource was just about having configuration files in > place below ''/repository/sonatype-work/nexus/conf''. But obviously > the first entry in the file array which resolves to > ''/repository/sonatype-work/nexus'' was the trigger for Puppet to > start recursive scanning everything. > > So, the misconfiguration was on my side (nice pitfall), but I wonder > if this is expected behaviour. Shouldn''t recursion only be done on > the last entry of the file array? > > > > No. Why would you suppose that? When you write a resource declaration > of the form > > someresource { [ ''title1'', ''title2'']: > param1 => value1, > param2 => value2 > } > > it is shorthand for separate resource declarations, one for each title > given, all having the specified parameters: > > someresource { ''title1'': > param1 => value1, > param2 => value2 > } > > someresource { ''title2'': > param1 => value1, > param2 => value2 > } > > The use of an array to specify multiple titles is perhaps a bit quirky > in itself (though useful!), but I don''t see why anyone would suppose > that resources specified that way would be assigned different parameters > from each other. > > There is nothing specific to the File resource type here, but even if > there were, why would you expect Puppet to suppose that you only wanted > recursion on one of the specified resources? I don''t see it. > > > John >Yes, you''re right. My idea was to have the ''source=>'' parameter only applied to the last entry in the array. This was simply an error in reasoning. Thanks Bernd -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.