sanjiv singh
2012-Apr-11 09:05 UTC
[Puppet Users] Asynchronous file transfer through puppet ????
Hi All,
I have configured *puppet with passenger with LDAP as
configuration store*.. Using puppet for deploying application different
machines. Application deployment consist of pushing numerous templates,
flat files(JARs,ZIPs,WARs etc) and various system/application level
configuration changes.
I have written puppet manifest accordingly.Here is sample for
pushing two ZIPs.
#--------------------------------------------------------
file { "/home/user/application1.zip":
group => ''app'',
source => "puppet://puppet/${environment}/application1.zip",
}
file { "/home/user/application2.zip":
group => ''app'',
source => "puppet://puppet/${environment}/application2.zip",
}
#-------------------------------------------------------
what am i questioning here is that puppet daemon execution is single
threaded.It processes each resource one-by-one( based on dependencies) .In
this case FILE resource. It transfers file Zips one-by-one irrespective of
dependency.
Is there any way, we can configure puppet/passenger so that it can
transfer files asynchronously. If its does not have dependency,puppet
daemon spawning thread for asynchronous file transfer ?
In my environment, I need to transfer number of files. More files means
more delay in puppet daemon execution.
I am trying to achieve some performance gain over application
deployment.
Regards,
*Sanjiv Singh *
Software Engineer (iLabs)
Impetus Infotech (India) Pvt. Ltd.
**
**
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/nThNbBIMVU8J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
jcbollinger
2012-Apr-11 13:25 UTC
[Puppet Users] Re: Asynchronous file transfer through puppet ????
On Apr 11, 4:05 am, sanjiv singh <sanjiv.is...@gmail.com> wrote:> Hi All, > I have configured *puppet with passenger with LDAP as > configuration store*.. Using puppet for deploying application different > machines. Application deployment consist of pushing numerous templates, > flat files(JARs,ZIPs,WARs etc) and various system/application level > configuration changes. > > I have written puppet manifest accordingly.Here is sample for > pushing two ZIPs. > #-------------------------------------------------------- > file { "/home/user/application1.zip": > group => ''app'', > source => "puppet://puppet/${environment}/application1.zip", > } > file { "/home/user/application2.zip": > group => ''app'', > source => "puppet://puppet/${environment}/application2.zip", > } > #------------------------------------------------------- > what am i questioning here is that puppet daemon execution is single > threaded.It processes each resource one-by-one( based on dependencies) .In > this case FILE resource. It transfers file Zips one-by-one irrespective of > dependency. > > Is there any way, we can configure puppet/passenger so that it can > transfer files asynchronously. If its does not have dependency,puppet > daemon spawning thread for asynchronous file transfer ? > > In my environment, I need to transfer number of files. More files means > more delay in puppet daemon execution. > > I am trying to achieve some performance gain over application > deployment.For application deployment, it is usually much better to build packages suitable for the target''s native packaging system (RPMs, for example), put them in a local repository, and use Package resources to install them. This is likely to perform much better than will using File resources to transfer many files. In any event, unless you''re talking about large numbers of small files, you are probably saturating your bandwidth already. Parallel file transfer will not help you in that case, and it might even introduce enough additional I/O overhead to slow you down overall. In any case, it would likely reduce the maximum number of clients each master could support. If you don''t want to build packages, then a better bet for speeding up your runs is to tweak the ''checksum'' property and maybe the ''backup'' property of your File resources. For the checksum, ''md5lite'' should be somewhat faster than the default (''md5''), and ''mtime'' should be substantially faster. Setting the ''backup'' parameter to ''false'' should also speed things a bit. Note, however, that changing these settings from their defaults gains speed by reducing integrity protection. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.