Alexander Holte-Davidsen
2012-Jul-31 11:56 UTC
[Puppet Users] Long processing time using Augeas
Hi!
I using Augeas to handle dump and passno for certain mount points in
/etc/fstab. Each mount point is defined as it''s own augeas block:
augeas { ''homeLV'':
context =>
''/files/etc/fstab'',
changes => [
''set *[file
"/home"]/dump 0'',
''set *[file
"/home"]/passno 0'',
],
onlyif => ''match
*[file "/home"] size > 0'',
}
augeas { ''optLV'':
context =>
''/files/etc/fstab'',
changes => [
''set *[file
"/opt"]/dump 0'',
''set *[file
"/opt"]/passno 0'',
],
onlyif => ''match
*[file "/opt"] size > 0'',
}
This code is taking a very long to to process, about 10 seconds for six
mount points. When running a debug run, I can see that /etc/fstab is opened
for each augeas code block, also when no change is needed.
debug: Augeas[varLV](provider=augeas): Opening augeas with root /home, lens
path , flags 0
debug: Augeas[varLV](provider=augeas): Augeas version 0.10.0 is installed
debug: Augeas[varLV](provider=augeas): Will attempt to save and only run if
files changed
debug: Augeas[varLV](provider=augeas): sending command ''set''
with params
["/files/etc/fstab/*[file = \"/home\"]/dump", "0"]
debug: Augeas[varLV](provider=augeas): sending command ''set''
with params
["/files/etc/fstab/*[file = \"/home\"]/passno",
"0"]
debug: Augeas[varLV](provider=augeas): Skipping because no files were
changed
debug: Augeas[varLV](provider=augeas): Closed the augeas connection
debug: Augeas[srvLV](provider=augeas): Opening augeas with root /opt, lens
path , flags 0
debug: Augeas[srvLV](provider=augeas): Augeas version 0.10.0 is installed
debug: Augeas[srvLV](provider=augeas): Will attempt to save and only run if
files changed
debug: Augeas[srvLV](provider=augeas): sending command ''set''
with params
["/files/etc/fstab/*[file = \"/opt\"]/dump", "0"]
debug: Augeas[srvLV](provider=augeas): sending command ''set''
with params
["/files/etc/fstab/*[file = \"/opt\"]/passno",
"0"]
debug: Augeas[srvLV](provider=augeas): Skipping because no files were
changed
debug: Augeas[srvLV](provider=augeas): Closed the augeas connection
Are there any obvious ways to speed this up? For example better checking
before the code is executed? Putting all the mount point changes in one
augeas block would probably speed things up a bit, but then I would not be
able to use the onlyif-statement per mount point?
Regards,
Alexander
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Alexander Holte-Davidsen
2012-Jul-31 12:15 UTC
[Puppet Users] Re: Long processing time using Augeas
After some more debugging, it looks like it takes about one second to process each augeas block, so putting all the statements in one block speeds things up a lot. Are there any other ways to do this? I want to be able to check if the mount point exists before I try change... Regards, Alexander On Tue, Jul 31, 2012 at 1:56 PM, Alexander Holte-Davidsen < alexander.davidsen@gmail.com> wrote:> Hi! > > I using Augeas to handle dump and passno for certain mount points in > /etc/fstab. Each mount point is defined as it''s own augeas block: > > augeas { ''homeLV'': > context => > ''/files/etc/fstab'', > changes => [ > ''set *[file > "/home"]/dump 0'', > ''set *[file > "/home"]/passno 0'', > ], > onlyif => ''match *[file > "/home"] size > 0'', > } > augeas { ''optLV'': > context => > ''/files/etc/fstab'', > changes => [ > ''set *[file > "/opt"]/dump 0'', > ''set *[file > "/opt"]/passno 0'', > ], > onlyif => ''match *[file > "/opt"] size > 0'', > } > > This code is taking a very long to to process, about 10 seconds for six > mount points. When running a debug run, I can see that /etc/fstab is opened > for each augeas code block, also when no change is needed. > debug: Augeas[varLV](provider=augeas): Opening augeas with root /home, > lens path , flags 0 > debug: Augeas[varLV](provider=augeas): Augeas version 0.10.0 is installed > debug: Augeas[varLV](provider=augeas): Will attempt to save and only run > if files changed > debug: Augeas[varLV](provider=augeas): sending command ''set'' with params > ["/files/etc/fstab/*[file = \"/home\"]/dump", "0"] > debug: Augeas[varLV](provider=augeas): sending command ''set'' with params > ["/files/etc/fstab/*[file = \"/home\"]/passno", "0"] > debug: Augeas[varLV](provider=augeas): Skipping because no files were > changed > debug: Augeas[varLV](provider=augeas): Closed the augeas connection > debug: Augeas[srvLV](provider=augeas): Opening augeas with root /opt, lens > path , flags 0 > debug: Augeas[srvLV](provider=augeas): Augeas version 0.10.0 is installed > debug: Augeas[srvLV](provider=augeas): Will attempt to save and only run > if files changed > debug: Augeas[srvLV](provider=augeas): sending command ''set'' with params > ["/files/etc/fstab/*[file = \"/opt\"]/dump", "0"] > debug: Augeas[srvLV](provider=augeas): sending command ''set'' with params > ["/files/etc/fstab/*[file = \"/opt\"]/passno", "0"] > debug: Augeas[srvLV](provider=augeas): Skipping because no files were > changed > debug: Augeas[srvLV](provider=augeas): Closed the augeas connection > > > Are there any obvious ways to speed this up? For example better checking > before the code is executed? Putting all the mount point changes in one > augeas block would probably speed things up a bit, but then I would not be > able to use the onlyif-statement per mount point? > > Regards, > Alexander >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Try something like the following instead (added ''incl'' and
''lens'' and
removed context as its default based on incl is what you had) :
augeas { ''homeLV'':
incl => ''/etc/fstab'',
lens => ''Fstab.lns'',
changes => [
''set *[file = "/home"]/dump 0'',
''set *[file = "/home"]/passno 0'',
],
onlyif => ''match *[file = "/home"] size >
0'',
}
I think this helps augeas not autoload everything it can possibly edit and
narrows it in on just one file.
Regards,
Jake
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/CiA2fUipMl8J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Alexander Holte-Davidsen
2012-Aug-01 07:58 UTC
Re: [Puppet Users] Re: Long processing time using Augeas
This did indeed make a huge difference. From about 10 seconds when loading everything, down to 0.22 seconds(!). Thanks for your help, Jake! / Alexander On Tue, Jul 31, 2012 at 8:16 PM, Jake - USPS <jacob.m.mccann@usps.gov>wrote:> Try something like the following instead (added ''incl'' and ''lens'' and > removed context as its default based on incl is what you had) : > > augeas { ''homeLV'': > incl => ''/etc/fstab'', > lens => ''Fstab.lns'', > changes => [ > ''set *[file = "/home"]/dump 0'', > ''set *[file = "/home"]/passno 0'', > ], > onlyif => ''match *[file = "/home"] size > 0'', > } > > I think this helps augeas not autoload everything it can possibly edit and > narrows it in on just one file. > > Regards, > Jake > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/puppet-users/-/CiA2fUipMl8J. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Dominic Cleal
2012-Aug-01 10:17 UTC
Re: [Puppet Users] Re: Long processing time using Augeas
On 31/07/12 19:16, Jake - USPS wrote:> Try something like the following instead (added ''incl'' and ''lens'' and > removed context as its default based on incl is what you had) : > > augeas { ''homeLV'': > incl => ''/etc/fstab'', > lens => ''Fstab.lns'', > changes => [ > ''set *[file = "/home"]/dump 0'', > ''set *[file = "/home"]/passno 0'', > ], > onlyif => ''match *[file = "/home"] size > 0'', > } > > I think this helps augeas not autoload everything it can possibly edit > and narrows it in on just one file.Exactly: Augeas loads all files with all lenses it knows about by default as it doesn''t know ahead of time which commands/paths you''ll use. In Puppet 3.0, there''s an optimisation in the provider (issue #7285) that will use the "context" parameter (optional, but usually specified) to only load lenses that are applicable to that context. Hopefully you''ll see the same drastic reduction in runtime out of the box in the near future. -- Dominic Cleal Red Hat Consulting m: +44 (0)7817 878113 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.