Puppet users - Feb 2011 - Do people walk the filebucket tree searching by path?

https://projects.puppetlabs.com/issues/6353

Our old behavior was that when files were backed up to a filebucket,
we also wrote out the path information to the ''paths'' file in
the
checksum directory.

Do people actually use this functionality? Our sanctioned interface
"puppet filebucket" only ever restored files by checksum, not by path,
but from the wiki and some tickets it looks like we do have users who
construct find/exec commands to search by path.

This leaves us in a somewhat frustrating position. We''ve broken
functionality that at least some people use, but was essentially
poking into a private implementation of the filebucket.

The whole point of the filebucket at least from a design perspective
was to store and retrieve files by checksum, not by path. However that
doesn''t appear to be how everyone uses it.


How much do you all care?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I care a lot and had thought that the path would eventually be the main 
key for retrieving files, with the checksum being sort of like a 
revision, with some extra metadata when you interfaces with the 
filebucket...

On 02/17/2011 03:46 PM, Nigel Kersten wrote:
> https://projects.puppetlabs.com/issues/6353
>
> Our old behavior was that when files were backed up to a filebucket,
> we also wrote out the path information to the ''paths''
file in the
> checksum directory.
>
> Do people actually use this functionality? Our sanctioned interface
> "puppet filebucket" only ever restored files by checksum, not by
path,
> but from the wiki and some tickets it looks like we do have users who
> construct find/exec commands to search by path.
>
> This leaves us in a somewhat frustrating position. We''ve broken
> functionality that at least some people use, but was essentially
> poking into a private implementation of the filebucket.
>
> The whole point of the filebucket at least from a design perspective
> was to store and retrieve files by checksum, not by path. However that
> doesn''t appear to be how everyone uses it.
>
>
> How much do you all care?
>

-- 
Joe McDonagh
AIM: YoosingYoonickz
IRC: joe-mac on freenode
"When the going gets weird, the weird turn pro."

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, Feb 17, 2011 at 12:48 PM, Joe McDonagh
<joseph.e.mcdonagh@gmail.com> wrote:
> I care a lot and had thought that the path would eventually be the main key
> for retrieving files, with the checksum being sort of like a revision, with
> some extra metadata when you interfaces with the filebucket...
ok. So it''s unacceptable for you to refer to logs or reports to get
the checksum for a given replacement and then restore the file that
way?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 02/17/2011 03:55 PM, Nigel Kersten wrote:
> On Thu, Feb 17, 2011 at 12:48 PM, Joe McDonagh
> <joseph.e.mcdonagh@gmail.com>  wrote:
>> I care a lot and had thought that the path would eventually be the main
key
>> for retrieving files, with the checksum being sort of like a revision,
with
>> some extra metadata when you interfaces with the filebucket...
> ok. So it''s unacceptable for you to refer to logs or reports to
get
> the checksum for a given replacement and then restore the file that
> way?
I''m not sure ''unacceptable'' is the word, I''m
not going to stop using
puppet because PL didn''t make the filebucket interface to my spec. Just
sayin, that''s how I envisioned it back when I started using Puppet... 
and now with all the fancy doo dads in puppet I''d figure the filebucket
would have lots of ways to access the data inside.

-- 
Joe McDonagh
AIM: YoosingYoonickz
IRC: joe-mac on freenode
"When the going gets weird, the weird turn pro."

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, Feb 17, 2011 at 3:55 PM, Nigel Kersten <nigel@puppetlabs.com>
wrote:
> ok. So it''s unacceptable for you to refer to logs or reports to
get
> the checksum for a given replacement and then restore the file that
It''s really damn fiddly :-)

As a git guts hacker, I appreciate that puppet stores things in a
content addressable filesystem. But when I need to use the info in git
or in puppet, I refer to it by path :-) ..

Actually git has some rich syntax to say "the previous version", like

   git diff HEAD^ # the prev commit
   git diff HEAD^^ # two commits back
   git diff HEAD^^ # three commits back ;-)

that kind of glue is of enormous value.



m
-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- Software Architect - OLPC
 - ask interesting questions
 - don''t get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I always expected to be able to access the values by path.

Having to recall a hash from a log is pretty much unrealistic without tools to
help you map the hash to the file at a later date.

I.e. Puppet supplied log parsers/data map store tools, etc...

That''s my .02 anyway.

Trevor

On 02/17/2011 04:06 PM, Martin Langhoff wrote:
> On Thu, Feb 17, 2011 at 3:55 PM, Nigel Kersten <nigel@puppetlabs.com>
wrote:
>> ok. So it''s unacceptable for you to refer to logs or reports
to get
>> the checksum for a given replacement and then restore the file that
> 
> It''s really damn fiddly :-)
> 
> As a git guts hacker, I appreciate that puppet stores things in a
> content addressable filesystem. But when I need to use the info in git
> or in puppet, I refer to it by path :-) ..
> 
> Actually git has some rich syntax to say "the previous version",
like
> 
>    git diff HEAD^ # the prev commit
>    git diff HEAD^^ # two commits back
>    git diff HEAD^^ # three commits back ;-)
> 
> that kind of glue is of enormous value.
> 
> 
> 
> m
- -- 
Trevor Vaughan
 Vice President, Onyx Point, Inc.
 email: tvaughan@onyxpoint.com
 phone: 410-541-ONYX (6699)
 pgp: 0x6C701E94

- -- This account not approved for unencrypted sensitive information --
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBAgAGBQJNXgGLAAoJECNCGV1OLcypvXMH/3N06b/Qzpw0AarfQg8k4QOO
rqng7+0LmDjJzgmj9bt6fJVy7o4iBtC+24CpjevU4I5jrfLZiQwODdmR/u9CpGUV
vKZuRit9MqqAnvkZB6nTfrQgv95S3x4j9ZFtnfdMQH3JKNOXd0xLKH0hx2NwNmF2
UaNt/nb1DrIaEjNwOV/PAKtFClMcWXYgeb3Jtn5KyZUWC616OqRJaqoXPtLoMoTD
ukOARqQZHKzH9gvLlph/JMqYhshrp3VChAf2homao5KzlFQgXbDnp6uH+/gQf5sC
LMOSum9I9hk+H4PtopuHeFs8AcDDrcuurCLlthL3TPACoudug18NtorxlA6Gs4Q=zv67
-----END PGP SIGNATURE-----

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, 17 Feb 2011, Nigel Kersten wrote:
> https://projects.puppetlabs.com/issues/6353
> 
> Our old behavior was that when files were backed up to a filebucket,
> we also wrote out the path information to the ''paths''
file in the
> checksum directory.
> 
> Do people actually use this functionality? Our sanctioned interface
> "puppet filebucket" only ever restored files by checksum, not by
path,
> but from the wiki and some tickets it looks like we do have users who
> construct find/exec commands to search by path.
Yes, I have often searched the filebucket for a path of interest.  In
theory, the checksum I need is available from a report, but in practice
the report may not be readily available.  Questions like "find all the
saved versions of this file" are much more easily answered by searching
the actual content of the filebucket than by searching all old logs back
to the beginning of time.

--apb (Alan Barrett)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi,

we''ve actually built tools around this behaviour. Scheduled puppet
resources can put files in a central bucket, and the data is collected
by mixing mtime and paths information of the bucketed files.

Granted, that''s probably not what the bucket was conceived for, but it
sure is convenient (and I''d hate to see it go).

Cheers,
Felix

On 02/17/2011 09:46 PM, Nigel Kersten wrote:
> https://projects.puppetlabs.com/issues/6353
> 
> Our old behavior was that when files were backed up to a filebucket,
> we also wrote out the path information to the ''paths''
file in the
> checksum directory.
> 
> Do people actually use this functionality? Our sanctioned interface
> "puppet filebucket" only ever restored files by checksum, not by
path,
> but from the wiki and some tickets it looks like we do have users who
> construct find/exec commands to search by path.
> 
> This leaves us in a somewhat frustrating position. We''ve broken
> functionality that at least some people use, but was essentially
> poking into a private implementation of the filebucket.
> 
> The whole point of the filebucket at least from a design perspective
> was to store and retrieve files by checksum, not by path. However that
> doesn''t appear to be how everyone uses it.
> 
> 
> How much do you all care?
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Feb 17, 2:55 pm, Nigel Kersten <ni...@puppetlabs.com>
wrote:
> On Thu, Feb 17, 2011 at 12:48 PM, Joe McDonagh
>
> <joseph.e.mcdon...@gmail.com> wrote:
> > I care a lot and had thought that the path would eventually be the
main key
> > for retrieving files, with the checksum being sort of like a revision,
with
> > some extra metadata when you interfaces with the filebucket...
>
> ok. So it''s unacceptable for you to refer to logs or reports to
get
> the checksum for a given replacement and then restore the file that
> way?
I don''t currently rely on filebucket for much, but that''s
partly
because recovering files by path is such a hassle.  I''m not tied to
details of Puppet''s implementation, but no, referring to logs or
reports to find a checksum by which to retrieve a file of interest is
not attractive.  Checksum is undoubtedly a convenient key for Puppet''s
purposes, but people invariably want to recover files by path and
timestamp/version.  I''ve never understood why Puppet didn''t
provide an
easier way to do that.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Thank you everyone for your feedback on this thread, it''s much
appreciated.

In light of the fact we were obviously removing useful functionality,
regardless of whether it was a sanctioned API or not, we''re putting
back the paths functionality.

See the following thread on puppet-dev for the details:

http://groups.google.com/group/puppet-dev/browse_thread/thread/f988609b7e271d4c

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.