Puppet users - Sep 2009 - Puppetd downloading sourced file on *every* run

Hi all,

I''m testing fileserving performance under 0.25 and I noticed that  
processing large files seems to take much longer than it should.

Here''s what I''ve got:

class filetest {
	file { "/var/tmp/test.tar":
		source => "puppet:///filetest/test.tar",
		backup => false
	}
}

My test file is about 30MB. As expected, the initial run took a while  
(over 30 seconds), but once the file was in place on the client  
puppetd was still taking more than five seconds to process it. That  
didn''t seem right, so I trussed puppetd and found that puppetd is  
downloading the entire file from the puppetmaster every single time it  
runs.

I was under the impression that puppetd would only download a file if  
the local checksum didn''t match the puppetmaster''s checksum
for that
file. Am I mistaken?

Thanks,

Josh Anderson

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hmm can you send stats of the how big the file and how long compared  
from 0.24.8?

I know we focused a lot of memory usage on the puppetmaster, and the  
amount of calls to the puppetmaster, but I don''t know if we performed  
any performance testing.

-L

--
Larry Ludwig
Reductive Labs


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Sep 10, 2009, at 6:51 PM, Joshua Anderson wrote:
>
>
> I was under the impression that puppetd would only download a file  
> if the local checksum didn''t match the puppetmaster''s
checksum for
> that file. Am I mistaken?
>
You are correct.

-L

--
Larry Ludwig
Reductive Labs


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Nevermind, I fooled myself.

Initially, I had set checksum => mtime on my test file (to try to  
avoid the disk I/O of checksumming a 30MB file). I didn''t realize that
using the source parameter forced an MD5 checksum, and that was why I  
saw the file content being read in truss.

I did find a different problem which I''ll be filing a bug for. The  
checksum_file() function in util/checksums.rb is getting called twice  
per file: once from type/file/checksum.rb and once from type/file/ 
content.rb.

-Josh

On Sep 10, 2009, at 6:06 PM, Larry Ludwig wrote:
>
> Hmm can you send stats of the how big the file and how long compared
> from 0.24.8?
>
> I know we focused a lot of memory usage on the puppetmaster, and the
> amount of calls to the puppetmaster, but I don''t know if we
performed
> any performance testing.
>
> -L
>
> --
> Larry Ludwig
> Reductive Labs
>
>
> >

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---