Nigel Kersten
2011-Feb-17 20:46 UTC
[Puppet Users] Do people walk the filebucket tree searching by path?
https://projects.puppetlabs.com/issues/6353 Our old behavior was that when files were backed up to a filebucket, we also wrote out the path information to the ''paths'' file in the checksum directory. Do people actually use this functionality? Our sanctioned interface "puppet filebucket" only ever restored files by checksum, not by path, but from the wiki and some tickets it looks like we do have users who construct find/exec commands to search by path. This leaves us in a somewhat frustrating position. We''ve broken functionality that at least some people use, but was essentially poking into a private implementation of the filebucket. The whole point of the filebucket at least from a design perspective was to store and retrieve files by checksum, not by path. However that doesn''t appear to be how everyone uses it. How much do you all care? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Joe McDonagh
2011-Feb-17 20:48 UTC
Re: [Puppet Users] Do people walk the filebucket tree searching by path?
I care a lot and had thought that the path would eventually be the main key for retrieving files, with the checksum being sort of like a revision, with some extra metadata when you interfaces with the filebucket... On 02/17/2011 03:46 PM, Nigel Kersten wrote:> https://projects.puppetlabs.com/issues/6353 > > Our old behavior was that when files were backed up to a filebucket, > we also wrote out the path information to the ''paths'' file in the > checksum directory. > > Do people actually use this functionality? Our sanctioned interface > "puppet filebucket" only ever restored files by checksum, not by path, > but from the wiki and some tickets it looks like we do have users who > construct find/exec commands to search by path. > > This leaves us in a somewhat frustrating position. We''ve broken > functionality that at least some people use, but was essentially > poking into a private implementation of the filebucket. > > The whole point of the filebucket at least from a design perspective > was to store and retrieve files by checksum, not by path. However that > doesn''t appear to be how everyone uses it. > > > How much do you all care? >-- Joe McDonagh AIM: YoosingYoonickz IRC: joe-mac on freenode "When the going gets weird, the weird turn pro." -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten
2011-Feb-17 20:55 UTC
Re: [Puppet Users] Do people walk the filebucket tree searching by path?
On Thu, Feb 17, 2011 at 12:48 PM, Joe McDonagh <joseph.e.mcdonagh@gmail.com> wrote:> I care a lot and had thought that the path would eventually be the main key > for retrieving files, with the checksum being sort of like a revision, with > some extra metadata when you interfaces with the filebucket...ok. So it''s unacceptable for you to refer to logs or reports to get the checksum for a given replacement and then restore the file that way? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Joe McDonagh
2011-Feb-17 21:05 UTC
Re: [Puppet Users] Do people walk the filebucket tree searching by path?
On 02/17/2011 03:55 PM, Nigel Kersten wrote:> On Thu, Feb 17, 2011 at 12:48 PM, Joe McDonagh > <joseph.e.mcdonagh@gmail.com> wrote: >> I care a lot and had thought that the path would eventually be the main key >> for retrieving files, with the checksum being sort of like a revision, with >> some extra metadata when you interfaces with the filebucket... > ok. So it''s unacceptable for you to refer to logs or reports to get > the checksum for a given replacement and then restore the file that > way?I''m not sure ''unacceptable'' is the word, I''m not going to stop using puppet because PL didn''t make the filebucket interface to my spec. Just sayin, that''s how I envisioned it back when I started using Puppet... and now with all the fancy doo dads in puppet I''d figure the filebucket would have lots of ways to access the data inside. -- Joe McDonagh AIM: YoosingYoonickz IRC: joe-mac on freenode "When the going gets weird, the weird turn pro." -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Martin Langhoff
2011-Feb-17 21:06 UTC
Re: [Puppet Users] Do people walk the filebucket tree searching by path?
On Thu, Feb 17, 2011 at 3:55 PM, Nigel Kersten <nigel@puppetlabs.com> wrote:> ok. So it''s unacceptable for you to refer to logs or reports to get > the checksum for a given replacement and then restore the file thatIt''s really damn fiddly :-) As a git guts hacker, I appreciate that puppet stores things in a content addressable filesystem. But when I need to use the info in git or in puppet, I refer to it by path :-) .. Actually git has some rich syntax to say "the previous version", like git diff HEAD^ # the prev commit git diff HEAD^^ # two commits back git diff HEAD^^ # three commits back ;-) that kind of glue is of enormous value. m -- martin.langhoff@gmail.com martin@laptop.org -- Software Architect - OLPC - ask interesting questions - don''t get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Trevor Vaughan
2011-Feb-18 05:20 UTC
Re: [Puppet Users] Do people walk the filebucket tree searching by path?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I always expected to be able to access the values by path. Having to recall a hash from a log is pretty much unrealistic without tools to help you map the hash to the file at a later date. I.e. Puppet supplied log parsers/data map store tools, etc... That''s my .02 anyway. Trevor On 02/17/2011 04:06 PM, Martin Langhoff wrote:> On Thu, Feb 17, 2011 at 3:55 PM, Nigel Kersten <nigel@puppetlabs.com> wrote: >> ok. So it''s unacceptable for you to refer to logs or reports to get >> the checksum for a given replacement and then restore the file that > > It''s really damn fiddly :-) > > As a git guts hacker, I appreciate that puppet stores things in a > content addressable filesystem. But when I need to use the info in git > or in puppet, I refer to it by path :-) .. > > Actually git has some rich syntax to say "the previous version", like > > git diff HEAD^ # the prev commit > git diff HEAD^^ # two commits back > git diff HEAD^^ # three commits back ;-) > > that kind of glue is of enormous value. > > > > m- -- Trevor Vaughan Vice President, Onyx Point, Inc. email: tvaughan@onyxpoint.com phone: 410-541-ONYX (6699) pgp: 0x6C701E94 - -- This account not approved for unencrypted sensitive information -- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBAgAGBQJNXgGLAAoJECNCGV1OLcypvXMH/3N06b/Qzpw0AarfQg8k4QOO rqng7+0LmDjJzgmj9bt6fJVy7o4iBtC+24CpjevU4I5jrfLZiQwODdmR/u9CpGUV vKZuRit9MqqAnvkZB6nTfrQgv95S3x4j9ZFtnfdMQH3JKNOXd0xLKH0hx2NwNmF2 UaNt/nb1DrIaEjNwOV/PAKtFClMcWXYgeb3Jtn5KyZUWC616OqRJaqoXPtLoMoTD ukOARqQZHKzH9gvLlph/JMqYhshrp3VChAf2homao5KzlFQgXbDnp6uH+/gQf5sC LMOSum9I9hk+H4PtopuHeFs8AcDDrcuurCLlthL3TPACoudug18NtorxlA6Gs4Q=zv67 -----END PGP SIGNATURE----- -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Alan Barrett
2011-Feb-18 10:18 UTC
Re: [Puppet Users] Do people walk the filebucket tree searching by path?
On Thu, 17 Feb 2011, Nigel Kersten wrote:> https://projects.puppetlabs.com/issues/6353 > > Our old behavior was that when files were backed up to a filebucket, > we also wrote out the path information to the ''paths'' file in the > checksum directory. > > Do people actually use this functionality? Our sanctioned interface > "puppet filebucket" only ever restored files by checksum, not by path, > but from the wiki and some tickets it looks like we do have users who > construct find/exec commands to search by path.Yes, I have often searched the filebucket for a path of interest. In theory, the checksum I need is available from a report, but in practice the report may not be readily available. Questions like "find all the saved versions of this file" are much more easily answered by searching the actual content of the filebucket than by searching all old logs back to the beginning of time. --apb (Alan Barrett) -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Felix Frank
2011-Feb-18 10:32 UTC
Re: [Puppet Users] Do people walk the filebucket tree searching by path?
Hi, we''ve actually built tools around this behaviour. Scheduled puppet resources can put files in a central bucket, and the data is collected by mixing mtime and paths information of the bucketed files. Granted, that''s probably not what the bucket was conceived for, but it sure is convenient (and I''d hate to see it go). Cheers, Felix On 02/17/2011 09:46 PM, Nigel Kersten wrote:> https://projects.puppetlabs.com/issues/6353 > > Our old behavior was that when files were backed up to a filebucket, > we also wrote out the path information to the ''paths'' file in the > checksum directory. > > Do people actually use this functionality? Our sanctioned interface > "puppet filebucket" only ever restored files by checksum, not by path, > but from the wiki and some tickets it looks like we do have users who > construct find/exec commands to search by path. > > This leaves us in a somewhat frustrating position. We''ve broken > functionality that at least some people use, but was essentially > poking into a private implementation of the filebucket. > > The whole point of the filebucket at least from a design perspective > was to store and retrieve files by checksum, not by path. However that > doesn''t appear to be how everyone uses it. > > > How much do you all care? >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
jcbollinger
2011-Feb-18 14:09 UTC
[Puppet Users] Re: Do people walk the filebucket tree searching by path?
On Feb 17, 2:55 pm, Nigel Kersten <ni...@puppetlabs.com> wrote:> On Thu, Feb 17, 2011 at 12:48 PM, Joe McDonagh > > <joseph.e.mcdon...@gmail.com> wrote: > > I care a lot and had thought that the path would eventually be the main key > > for retrieving files, with the checksum being sort of like a revision, with > > some extra metadata when you interfaces with the filebucket... > > ok. So it''s unacceptable for you to refer to logs or reports to get > the checksum for a given replacement and then restore the file that > way?I don''t currently rely on filebucket for much, but that''s partly because recovering files by path is such a hassle. I''m not tied to details of Puppet''s implementation, but no, referring to logs or reports to find a checksum by which to retrieve a file of interest is not attractive. Checksum is undoubtedly a convenient key for Puppet''s purposes, but people invariably want to recover files by path and timestamp/version. I''ve never understood why Puppet didn''t provide an easier way to do that. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten
2011-Feb-21 23:13 UTC
Re: [Puppet Users] Re: Do people walk the filebucket tree searching by path?
Thank you everyone for your feedback on this thread, it''s much appreciated. In light of the fact we were obviously removing useful functionality, regardless of whether it was a sanctioned API or not, we''re putting back the paths functionality. See the following thread on puppet-dev for the details: http://groups.google.com/group/puppet-dev/browse_thread/thread/f988609b7e271d4c -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.