The following case is about to go to PSARC. Comments are welcome. eric (i''ll be going out of town tonight, so apologies if i can''t respond immediatley to feedback) -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: zpool_history.txt URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20060503/3d72223e/attachment.txt>
On May 3, 2006, at 8:52, eric kustarz wrote:> In the future, we are looking to add additional data to log and > retrieve, such > as hostname (for shared storage) or username (for delegated > administration).Why not include these in the initial implementation? They both strike me as important and valuable. --Ed -- Ed Gould Sun Microsystems File System Architect Sun Cluster ed.gould at sun.com 17 Network Circle +1.650.786.4937 MS UMPK17-201 x84937 Menlo Park, CA 94025
Ed Gould wrote:> On May 3, 2006, at 8:52, eric kustarz wrote: > >> In the future, we are looking to add additional data to log and >> retrieve, such >> as hostname (for shared storage) or username (for delegated >> administration). > > > Why not include these in the initial implementation? They both strike > me as important and valuable.The main goal is to record what commands have been done to the pool and file systems (so we know if we have a problem code-wise or admin-wise). Adding username and hostname are definitely useful but not necessary to have a functional history. Adding internal events (such as when a resilver started) or adding command history to ztest is also useful, but not necessary to have a functional history for zfs. The username didn''t seem as important now as we don''t have delegated administration yet. hostname is definitely intriguing and i''ll see how much extra work entails in getting that included. So i agree that including these two pieces of information is definitely important and valuable, but the major value will be in the actual logging of the command history even without these two pieces. eric> > --Ed > -- > Ed Gould Sun Microsystems > File System Architect Sun Cluster > ed.gould at sun.com 17 Network Circle > +1.650.786.4937 MS UMPK17-201 > x84937 Menlo Park, CA 94025 >
On Wed, May 03, 2006 at 10:14:51AM -0700, eric kustarz wrote:> Ed Gould wrote: > >On May 3, 2006, at 8:52, eric kustarz wrote: > > > >>In the future, we are looking to add additional data to log and > >>retrieve, such > >>as hostname (for shared storage) or username (for delegated > >>administration). > > > > > >Why not include these in the initial implementation? They both strike > >me as important and valuable. > > The main goal is to record what commands have been done to the pool and > file systems (so we know if we have a problem code-wise or admin-wise). > Adding username and hostname are definitely useful but not necessary to > have a functional history. Adding internal events (such as when a > resilver started) or adding command history to ztest is also useful, but > not necessary to have a functional history for zfs. > > The username didn''t seem as important now as we don''t have delegated > administration yet.That sounds like a job for auditing.> hostname is definitely intriguing and i''ll see how much extra work > entails in getting that included.Particularly in HA-type environments.> So i agree that including these two pieces of information is definitely > important and valuable, but the major value will be in the actual > logging of the command history even without these two pieces.Yes, but, how hard is it really to add those two items? If the answer is "not very" then just do it :) Nico --
On Wed, 2006-05-03 at 11:52, eric kustarz wrote:> The following case is about to go to PSARC. Comments are welcome.and just so folks outside sun can see what a fast-track review looks like, I''ll send my comments here rather than wait for it to show up at PSARC.. (I don''t thing absolutely everything in here needs to be in the initial version integrated. This is mostly food for thought more than anything else). 1) I don''t see a change to the on-disk format document. During commitment review of ZFS we asked that the on-disk format document be logged as part of case materials (but let it be delivered later as it was a work in progress; we just wanted the document archived for posterity). 2) structured or un-structured log information? there are at least two fields now, and potentially more later (if you add the hostname, username, etc.). If it''s unstructured, why? if it''s structured, how do I get at individual fields of each record? 3) more specific version of #2: is there any ambiguity in either the on-disk format or in the "zpool history" output if the command arguments contain whitespace? (IMHO, it''s more critical to get this right on-disk than in the command, which is part of why I asked #1. I think you want to log the argv[] array rather than a single string for the command...) 4) In part, you''re reinventing a simple variant of the solaris audit subsystem which is specialized for zfs. I understand perfectly why you''re reinventing this wheel rather than trying to use audit directly. But have you looked at audit to see if you''re forgetting to log something important? given that there are RBAC profiles for ZFS, we already have a crude form of delegated administration present in the system. 5) I assume that new zfs and zpool subcommands will need to specify whether or not they create a zpool history entry; is there a general rule here going forward? Working backwards from your list of "logged" vs "not logged", how about: - A command which could cause a permanent change to the configuration of the pool MUST be logged. - Events which are operationally interesting (such as the start of a scrub) MAY be logged. 6) could something be logged at boot time when the pool is brought up and likewise during a clean shutdown? seems like those might be of value in looking into trouble around an unclean shutdown. Nits:> if a subcommand was successfully written to disk, but the command > history log I/O failed, then a counter of failed log write I/Os is > incremented.I''m assuming you mean "if a subcommand successfully executes".> zpool attach > zpool replace > zpool scrubis that logged at the start of the operation, the completion of the operation, or both? I''d think I''d want to see "scrub complete" or "resilver complete" in the log so I could know (a) how to tune a scrub interval (b) know how long I was operating with reduced redundancy..> zpool destroy"neat trick". I''m assuming that you have to do a subsequent "zpool import -f -D" to actually see this... - Bill
On Wed, May 03, 2006 at 01:32:27PM -0400, Bill Sommerfeld wrote:> 5) I assume that new zfs and zpool subcommands will need to specify > whether or not they create a zpool history entry; is there a general > rule here going forward? Working backwards from your list of "logged" > vs "not logged", how about: > - A command which could cause a permanent change to the configuration > of the pool MUST be logged. > - Events which are operationally interesting (such as the start of a > scrub) MAY be logged.And since the latter may be common, maybe old items should be pruned. Nico --
On Wed, 2006-05-03 at 12:40 -0500, Nicolas Williams wrote:> On Wed, May 03, 2006 at 01:32:27PM -0400, Bill Sommerfeld wrote: > > 5) I assume that new zfs and zpool subcommands will need to specify > > whether or not they create a zpool history entry; is there a general > > rule here going forward? Working backwards from your list of "logged" > > vs "not logged", how about: > > - A command which could cause a permanent change to the configuration > > of the pool MUST be logged. > > - Events which are operationally interesting (such as the start of a > > scrub) MAY be logged. > > And since the latter may be common, maybe old items should be pruned.I''d vote against pruning. I do a lot of field studies and pruned logs negatively affect our ability to calculate rates and duration of events. -- richard
On Wed, 2006-05-03 at 13:40, Nicolas Williams wrote:> On Wed, May 03, 2006 at 01:32:27PM -0400, Bill Sommerfeld wrote: > > 5) I assume that new zfs and zpool subcommands will need to specify > > whether or not they create a zpool history entry; is there a general > > rule here going forward? Working backwards from your list of "logged" > > vs "not logged", how about: > > - A command which could cause a permanent change to the configuration > > of the pool MUST be logged. > > - Events which are operationally interesting (such as the start of a > > scrub) MAY be logged. > > And since the latter may be common, maybe old items should be pruned.That''s in the proposal already: In the kernel, the log is implemented as a ring buffer. The maximum size is 32MB (enough space for a single indirect block with 128KB blocks). For smaller pools, the maximum size is capped at 1% of the pool size (size determined at pool creation time), but the size is always at least 128KB. The administrator doesn''t have direct control of how big the ring buffer is.
Bill Sommerfeld wrote:>On Wed, 2006-05-03 at 11:52, eric kustarz wrote: > > >>The following case is about to go to PSARC. Comments are welcome. >> >> > >and just so folks outside sun can see what a fast-track review looks >like, I''ll send my comments here rather than wait for it to show up at >PSARC.. >(I don''t thing absolutely everything in here needs to be in the initial >version integrated. This is mostly food for thought more than anything >else). > > 1) I don''t see a change to the on-disk format document. During >commitment review of ZFS we asked that the on-disk format document be >logged as part of case materials (but let it be delivered later as it >was a work in progress; we just wanted the document archived for >posterity). > >Yep, i''ll make sure to update the on-disk document, and get it logged with PSARC.> 2) structured or un-structured log information? there are at least two >fields now, and potentially more later (if you add the hostname, >username, etc.). If it''s unstructured, why? if it''s structured, how do >I get at individual fields of each record? > >Its really just a simple plain text file of strings of the following entries: <string type><string value><delimiter> The delimiter is the ''\0'' character. So right now we record time (string type ''t'') and command (string type ''c''). That will look like: <''\0''><''t''><time encoded as string><''\0''><''c''><"zfs pool create hewitt c1t1d0s3"><''\0''> The extra NULL character preceding the time string is done so i can always find the beginning of a "record" (where a record contains several entries). This is important since its a ring buffer and data is overwritten (though the ''zpool create'' never is). So to find individual fields (entries) you need to parse through the strings. In the future we will have something like: <''\0''><''t''><time string><''\0''><''c''><"zfs pool create hewitt c1t1d0s3"><''\0''><''h''><"super_server"><''\0''><''u''><"ek110237"><''\0''>> 3) more specific version of #2: is there any ambiguity in either the >on-disk format or in the "zpool history" output if the command arguments >contain whitespace? > >(IMHO, it''s more critical to get this right on-disk than in the command, >which is part of why I asked #1. I think you want to log the argv[] >array rather than a single string for the command...) > >Hmm, i''m just taking the argv array and taking each of its entries to form one string and pass that down. So that''s going to potentially change the whitespace. Why does that matter?> 4) In part, you''re reinventing a simple variant of the solaris audit >subsystem which is specialized for zfs. I understand perfectly why >you''re reinventing this wheel rather than trying to use audit directly. >But have you looked at audit to see if you''re forgetting to log >something important? >given that there are RBAC profiles for ZFS, we already have a crude form >of delegated administration present in the system. > >I haven''t looked at audit, do you have a pointer? The nice thing is that if we do forget something or invent something later that we would like to log, we can add it.> 5) I assume that new zfs and zpool subcommands will need to specify >whether or not they create a zpool history entry; is there a general >rule here going forward? Working backwards from your list of "logged" >vs "not logged", how about: > - A command which could cause a permanent change to the configuration >of the pool MUST be logged. > - Events which are operationally interesting (such as the start of a >scrub) MAY be logged. > >The rule is if it modifies state, then we log it. But you''re right, we may change that when we add internal events (a scrub might or might not actually modify state) - but i would like events either logged or not (no MAYBEs).>6) could something be logged at boot time when the pool is brought up >and likewise during a clean shutdown? seems like those might be of >value in looking into trouble around an unclean shutdown. > >Sure thing, we''ll look into that when we see what internal events we want to log.>Nits: > > > >>if a subcommand was successfully written to disk, but the command >>history log I/O failed, then a counter of failed log write I/Os is >>incremented. >> >> > >I''m assuming you mean "if a subcommand successfully executes". > >So its really both - the subcommand successfully executes when its actually written to disk and txg group is synced. This section is all about handling write failures, hence the way i wrote it. Basically, what happens if the subcommand write I/O succeeds, but the log command write I/O doesn''t, what do we do...> > >> zpool attach >> zpool replace >> zpool scrub >> >> > >is that logged at the start of the operation, the completion of the >operation, or both? I''d think I''d want to see "scrub complete" or >"resilver complete" in the log so I could know (a) how to tune a scrub >interval (b) know how long I was operating with reduced redundancy.. > >Still thinking about this one. I''m leaning towards logging just when the scrub starts in this first phase (where the first phase is all about knowing what the admin did). When we add internal events, then we can log when the scrub finishes.> > >>zpool destroy >> >> > >"neat trick". I''m assuming that you have to do a subsequent "zpool >import -f -D" to actually see this... > >heh, yep. eric
Bill Sommerfeld wrote:>On Wed, 2006-05-03 at 13:40, Nicolas Williams wrote: > > >>On Wed, May 03, 2006 at 01:32:27PM -0400, Bill Sommerfeld wrote: >> >> >>> 5) I assume that new zfs and zpool subcommands will need to specify >>>whether or not they create a zpool history entry; is there a general >>>rule here going forward? Working backwards from your list of "logged" >>>vs "not logged", how about: >>> - A command which could cause a permanent change to the configuration >>>of the pool MUST be logged. >>> - Events which are operationally interesting (such as the start of a >>>scrub) MAY be logged. >>> >>> >>And since the latter may be common, maybe old items should be pruned. >> >> > >That''s in the proposal already: > >In the kernel, the log is implemented as a ring buffer. The maximum >size >is 32MB (enough space for a single indirect block with 128KB blocks). >For >smaller pools, the maximum size is capped at 1% of the pool size (size >determined at pool creation time), but the size is always at least >128KB. >The administrator doesn''t have direct control of how big the ring buffer >is. > > >Right, one thing i''d like to add is that the original ''zpool create'' command is never overwritten (true of pools created after these changes are integrated of course). eric
On Wed, 2006-05-03 at 11:10 -0700, eric kustarz wrote:> > 2) structured or un-structured log information? there are at least two > >fields now, and potentially more later (if you add the hostname, > >username, etc.). If it''s unstructured, why? if it''s structured, how do > >I get at individual fields of each record? > > > > > > Its really just a simple plain text file of strings of the following > entries: > > <string type><string value><delimiter> > > The delimiter is the ''\0'' character. So right now we record time > (string type ''t'') and command (string type ''c''). > > That will look like: > <''\0''><''t''><time encoded as string><''\0''><''c''><"zfs pool create hewitt > c1t1d0s3"><''\0''> > > The extra NULL character preceding the time string is done so i can > always find the beginning of a "record" (where a record contains several > entries). This is important since its a ring buffer and data is > overwritten (though the ''zpool create'' never is). > > So to find individual fields (entries) you need to parse through the > strings. > > In the future we will have something like: > <''\0''><''t''><time string><''\0''><''c''><"zfs pool create hewitt > c1t1d0s3"><''\0''><''h''><"super_server"><''\0''><''u''><"ek110237"><''\0''>Why not use a terse XML format? It could be extended later as needed without affecting tools and can easily accomodate the argv[] array. For that matter, shouldn''t it just be in a hidden file (.zfs-history) so we don''t have the circular buffer issues? -- richard
Richard Elling wrote:>On Wed, 2006-05-03 at 11:10 -0700, eric kustarz wrote: > > >>>2) structured or un-structured log information? there are at least two >>>fields now, and potentially more later (if you add the hostname, >>>username, etc.). If it''s unstructured, why? if it''s structured, how do >>>I get at individual fields of each record? >>> >>> >>> >>> >>Its really just a simple plain text file of strings of the following >>entries: >> >><string type><string value><delimiter> >> >>The delimiter is the ''\0'' character. So right now we record time >>(string type ''t'') and command (string type ''c''). >> >>That will look like: >><''\0''><''t''><time encoded as string><''\0''><''c''><"zfs pool create hewitt >>c1t1d0s3"><''\0''> >> >>The extra NULL character preceding the time string is done so i can >>always find the beginning of a "record" (where a record contains several >>entries). This is important since its a ring buffer and data is >>overwritten (though the ''zpool create'' never is). >> >>So to find individual fields (entries) you need to parse through the >>strings. >> >>In the future we will have something like: >><''\0''><''t''><time string><''\0''><''c''><"zfs pool create hewitt >>c1t1d0s3"><''\0''><''h''><"super_server"><''\0''><''u''><"ek110237"><''\0''> >> >> > >Why not use a terse XML format? It could be extended later as needed >without affecting tools and can easily accomodate the argv[] array. > >I didn''t see a need for it. Do you have a specific example where it works but what i proposed doesn''t?>For that matter, shouldn''t it just be in a hidden file (.zfs-history) >so we don''t have the circular buffer issues? > -- richard > > >Something like .zfs/history/history sounds nice, but remember this is information per pool, not file system. And i believe we want a circular buffer, i would hate for my storage to slowly get eaten over time without a way of getting it back (yes, the amount of space taken should be small, but there might be a few who switch properties on thousands of file systems weekly or something else weird). And i definitely don''t want to add a way that the admin can delete history on purpose. eric
On Wed, 2006-05-03 at 14:10, eric kustarz wrote:> Hmm, i''m just taking the argv array and taking each of its entries to > form one string and pass that down. So that''s going to potentially > change the whitespace. Why does that matter?potential ambiguity if arguments can contain whitespace. maybe actual ambiguity can''t arise right now in correct commands but will this always be true going forward? Currently we disallow whitespace in filesystem names, but if that constraint were to be relaxed, this would get much more important. (Apple seems to like putting spaces in file names..) The shell gives you an argv[] array; I think it''s best to just log what it hands you as is (to use your notation: as a sequence of <''c''>argv[i]<''\0''> fields instead of a single one, perhaps?) rather than attempt to splice it back into a string -- that way even if the initial version of zpool history just splices it back into a string without further processing you can still forensically reconstruct what happened to the pool and a later improvement can get the quoting right.> I haven''t looked at audit, do you have a pointer?I haven''t much looked at it myself, I just know it''s there... the data returned by getaudit(2) look potentially relevant and useful for associating the current process with a particular login session. Talk to Gary Winiger; he seems to know something about audit. I suspect the right answer here is for the whatever kernel mechanism you use to drop the history record into the audit trail to pick information equivalent to what would be available via getaudit() from the current process..> I would like events either logged or not (no MAYBEs).Agreed.> Sure thing, we''ll look into [boot/shutdown events] when we see what > internal events we want to log.Great.> So its really both - the subcommand successfully executes when its > actually written to disk and txg group is synced.I found myself backtracking while reading that sentence due to the ambiguity in the first half -- did you mean the write of the literal text of the command itself to the audit trail, or the intended changes to the pool produced by the subcommand? (just a wordsmithing thing, really.)> Still thinking about this one. I''m leaning towards logging just when > the scrub starts in this first phase (where the first phase is all about > knowing what the admin did). When we add internal events, then we can > log when the scrub finishes.and that''s consistent with the other commands since "scrub", "replace", "attach", etc., just start a background thread to actually do the work.. - Bill
On Wed, May 03, 2006 at 11:58:43AM -0700, eric kustarz wrote:> >Why not use a terse XML format? It could be extended later as needed > >without affecting tools and can easily accomodate the argv[] array. > > I didn''t see a need for it. Do you have a specific example where it > works but what i proposed doesn''t?Or an NVLIST! ZFS already uses those... I''ve found that it''s easier to use NVLISTs than it is to write packing/unpacking code from scratch.
On Wed, May 03, 2006 at 01:58:10PM -0400, Bill Sommerfeld wrote:> On Wed, 2006-05-03 at 13:40, Nicolas Williams wrote: > > On Wed, May 03, 2006 at 01:32:27PM -0400, Bill Sommerfeld wrote: > > > 5) I assume that new zfs and zpool subcommands will need to specify > > > whether or not they create a zpool history entry; is there a general > > > rule here going forward? Working backwards from your list of "logged" > > > vs "not logged", how about: > > > - A command which could cause a permanent change to the configuration > > > of the pool MUST be logged. > > > - Events which are operationally interesting (such as the start of a > > > scrub) MAY be logged. > > > > And since the latter may be common, maybe old items should be pruned. > > That''s in the proposal already:What I meant is that events that "cause a permanent change..." should not be deleted from the circular log if there are "old" (older?) "operationally interesting" events that could be deleted instead. I.e., if the log can keep only so much info then I''d rather have the history of a pool''s configuration than any but the most recent scrub/ resilver operations.
eric kustarz wrote:> The following case is about to go to PSARC. Comments are welcome. > > eric >To piggyback on earlier comments re: adding hostname and user: What is the need for zpool history to distinguish zfs commands that were executed by priviledged users in non-global zones for those datasets under ngz ownership? /jason
Jason Schroeder wrote:> eric kustarz wrote: > >> The following case is about to go to PSARC. Comments are welcome. >> >> eric >> > To piggyback on earlier comments re: adding hostname and user: > > What is the need for zpool history to distinguish zfs commands that > were executed by priviledged users in non-global zones for those > datasets under ngz ownership? >I personally don''t see a need to distinguish between zones. However, with delegated administration, it would be nice to know who did (say) destroy that file system - the local root or some remote user. eric
Nicolas Williams wrote:>On Wed, May 03, 2006 at 01:58:10PM -0400, Bill Sommerfeld wrote: > > >>On Wed, 2006-05-03 at 13:40, Nicolas Williams wrote: >> >> >>>On Wed, May 03, 2006 at 01:32:27PM -0400, Bill Sommerfeld wrote: >>> >>> >>>> 5) I assume that new zfs and zpool subcommands will need to specify >>>>whether or not they create a zpool history entry; is there a general >>>>rule here going forward? Working backwards from your list of "logged" >>>>vs "not logged", how about: >>>> - A command which could cause a permanent change to the configuration >>>>of the pool MUST be logged. >>>> - Events which are operationally interesting (such as the start of a >>>>scrub) MAY be logged. >>>> >>>> >>>And since the latter may be common, maybe old items should be pruned. >>> >>> >>That''s in the proposal already: >> >> > >What I meant is that events that "cause a permanent change..." should >not be deleted from the circular log if there are "old" (older?) >"operationally interesting" events that could be deleted instead. > >I.e., if the log can keep only so much info then I''d rather have the >history of a pool''s configuration than any but the most recent scrub/ >resilver operations. > >That perhaps should be implemented as two logs - then the kernel wouldn''t have to read in the buffer & parse it when its trying to just log a command. One log for modifying subcommands, one log for internal/interesting events. Makes me squeamish to have two (or more) though. eric
On Wed, May 03, 2006 at 02:47:57PM -0700, eric kustarz wrote:> Jason Schroeder wrote: > > >eric kustarz wrote: > > > >>The following case is about to go to PSARC. Comments are welcome. > >> > >>eric > >> > >To piggyback on earlier comments re: adding hostname and user: > > > >What is the need for zpool history to distinguish zfs commands that > >were executed by priviledged users in non-global zones for those > >datasets under ngz ownership? > > > I personally don''t see a need to distinguish between zones. However, > with delegated administration, it would be nice to know who did (say) > destroy that file system - the local root or some remote user.Keep in mind that one username (or uid) in a local zone is different from the same username in the global zone, since they can be running different name services. In the simplest example, you could have an entry that said something like: root zfs destroy tank/foo And if you were using datasets delegated to local zones, you wouldn''t know if that was ''root'' in the global zone or ''root'' in the local zone. If you are going to log a user at all, you _need_ to log the zone name as well. Even without usernames, it would probably be useful to know that a particular action was done in a particular zone. Imagine a service provider with several zones delegated to different users, and each user has their own portion of the namespace. At some point, you get a servicecall from a customer saying "someone deleted my filesystems!!!!" You could look at the zpool history, but without a zone name, you wouldn''t know if was your fault (from the global zone) or theirs (from the local zone). - Eric -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock
On Wed, May 03, 2006 at 03:05:25PM -0700, Eric Schrock wrote:> On Wed, May 03, 2006 at 02:47:57PM -0700, eric kustarz wrote: > > Jason Schroeder wrote: > > > > >eric kustarz wrote: > > > > > >>The following case is about to go to PSARC. Comments are welcome. > > >> > > >>eric > > >> > > >To piggyback on earlier comments re: adding hostname and user: > > > > > >What is the need for zpool history to distinguish zfs commands that > > >were executed by priviledged users in non-global zones for those > > >datasets under ngz ownership? > > > > > I personally don''t see a need to distinguish between zones. However, > > with delegated administration, it would be nice to know who did (say) > > destroy that file system - the local root or some remote user. > > Keep in mind that one username (or uid) in a local zone is different > from the same username in the global zone, since they can be running > different name services. In the simplest example, you could have an > entry that said something like: > > root zfs destroy tank/foo > > And if you were using datasets delegated to local zones, you wouldn''t > know if that was ''root'' in the global zone or ''root'' in the local zone. > If you are going to log a user at all, you _need_ to log the zone name > as well. Even without usernames, it would probably be useful to know > that a particular action was done in a particular zone. > > Imagine a service provider with several zones delegated to different > users, and each user has their own portion of the namespace. At some > point, you get a servicecall from a customer saying "someone deleted my > filesystems!!!!" You could look at the zpool history, but without a > zone name, you wouldn''t know if was your fault (from the global zone) or > theirs (from the local zone). > > - Eric >why don''t you see a need to distinguish between zones? in most cases (but not all) a zone administrator doesn''t deal with pools. they deal with datasets allocated to their zone, and for the same reasons that the global zone administrator might want access to zfs command histories, a zone administrator might want access to zfs command histories that apply to datasets allocated to their zones. which makes me wonder if perhaps zfs command history buffers should also be supported on datasets allocated to zones? or perhaps a zone administrator should should be able to view a subset of the zfs command history, specifically the transactions that affect datasets allocated to their zone? ed
I, too, am late to this thread but I caught something that didn''t seem right to me in this specific example. For the administration of the non-global zones, SunEducation (for whom I am an instructor) is stressing that the ng zones are "Software Virtualizations" (my quotes) and that the hardware and infrastructure are managed by the global zone admin. In this case, the ngz admins would not have access or permission to corrupt their filesystems at the zpool/zfs level. Unless zfs is to offer a different management model, I don''t suspect we will need to differentiate the (incapacitated) ngz admins from the gz admins. Regards, Craig On Wed, May 3, 2006 3:05 pm, Eric Schrock said:> On Wed, May 03, 2006 at 02:47:57PM -0700, eric kustarz wrote: >> Jason Schroeder wrote: >> >> >eric kustarz wrote: >> > >> >>The following case is about to go to PSARC. Comments are welcome. >> >> >> >>eric >> >> >> >To piggyback on earlier comments re: adding hostname and user: >> > >> >What is the need for zpool history to distinguish zfs commands that >> >were executed by priviledged users in non-global zones for those >> >datasets under ngz ownership? >> > >> I personally don''t see a need to distinguish between zones. However, >> with delegated administration, it would be nice to know who did (say) >> destroy that file system - the local root or some remote user. > > Keep in mind that one username (or uid) in a local zone is different > from the same username in the global zone, since they can be running > different name services. In the simplest example, you could have an > entry that said something like: > > root zfs destroy tank/foo > > And if you were using datasets delegated to local zones, you wouldn''t > know if that was ''root'' in the global zone or ''root'' in the local zone. > If you are going to log a user at all, you _need_ to log the zone name > as well. Even without usernames, it would probably be useful to know > that a particular action was done in a particular zone. > > Imagine a service provider with several zones delegated to different > users, and each user has their own portion of the namespace. At some > point, you get a servicecall from a customer saying "someone deleted my > filesystems!!!!" You could look at the zpool history, but without a > zone name, you wouldn''t know if was your fault (from the global zone) or > theirs (from the local zone). > > - Eric > > -- > Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Craig Cory wrote:>I, too, am late to this thread but I caught something that didn''t seem right >to me in this specific example. For the administration of the non-global >zones, SunEducation (for whom I am an instructor) is stressing that the ng >zones are "Software Virtualizations" (my quotes) and that the hardware and >infrastructure are managed by the global zone admin. In this case, the ngz >admins would not have access or permission to corrupt their filesystems at the >zpool/zfs level. Unless zfs is to offer a different management model, I don''t >suspect we will need to differentiate the (incapacitated) ngz admins from the >gz admins. > >Regards, > >Craig > > >Actually, with the current capabilities, a global zone priviledged user is able to choose to delegate a zfs dataset to a particular zone, in which case a ngz priviledged user would have the choice to set various zfs properties and create/delete zfs subsets within the zone''s delegated dataset <you will have to define your view of corrupt here ;-) >. So, in that case - a ngz priviledged user would trigger events that would seem to be captured in the global zone''s view of the overall zpool history. Thus the suggestion that there may be a need for further distinction. /jason
On Wed, May 03, 2006 at 03:52:39PM -0700, Craig Cory wrote:> I, too, am late to this thread but I caught something that didn''t seem right > to me in this specific example. For the administration of the non-global > zones, SunEducation (for whom I am an instructor) is stressing that the ng > zones are "Software Virtualizations" (my quotes) and that the hardware and > infrastructure are managed by the global zone admin. In this case, the ngz > admins would not have access or permission to corrupt their filesystems at the > zpool/zfs level. Unless zfs is to offer a different management model, I don''t > suspect we will need to differentiate the (incapacitated) ngz admins from the > gz admins.Regardless of whether global zone admins _should_ corrupt non-global zone software state, it will forever remain that they _can_. For example, nothing prevents a global zone admin today from removing /<zonepath>/root/etc/passwd. Nor should this level of restriction be enforced - as long as the global zone adminsitrator has all privileges, there is nothing he or she can or cannot do to destroy local zone state. Similarly, the global zone administrator can choose to export devices to the local zone that would give them access to physical hardware. It''s just not the recommended method of operation. The pool history provides a means for verifying that this philosophical boundary hasn''t been crossed. Even ignoring this fact, the hierarchical nature of ZFS datasets allows for changes to parent filesystems to be reflected in the local zone dataset. Imagine the global zone administrator turns compression on for the whole pool - now the local zone administrator will have compression on unless they''ve explicitly set it locally. - Eric -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock
Bill Sommerfeld wrote: ...>>So its really both - the subcommand successfully executes when its >>actually written to disk and txg group is synced. >> >> > >I found myself backtracking while reading that sentence due to the >ambiguity in the first half -- did you mean the write of the literal >text of the command itself to the audit trail, or the intended changes >to the pool produced by the subcommand? (just a wordsmithing thing, >really.) > >Sorry, the "subcommand" refers to whatever writes that currently happen today to "put" something like ''zfs create pool/fs'' on disk. The "log I/O" refers to the new writes that happen with the proposed changes for command history. So we''re trying to determine what happens when the newly introduced writes (due to the command history) fail. eric
> # zpool history jen > History for ''jen'': > 2006-04-27T10:38:36 zpool create jen mirror ...I have two suggestions which are just minor nits compared with the rest of this discussion: 1. Why do you print a "T" between the date and the time? I think a space would be more readable. 2. When printing the history for a specific pool, I don''t think we should print the "History for <pool>" line. It seems unnecessary, and that way every line of output will have the same format (better for later machine parsing). --matt This message posted from opensolaris.org
> What I meant is that events that "cause a permanent change..." should > not be deleted from the circular log if there are "old" (older?) > "operationally interesting" events that could be deleted instead. > > I.e., if the log can keep only so much info then I''d rather have the > history of a pool''s configuration than any but the most recent scrub/ > resilver operations.Well, then it''s really not a log. There''s no reason to truncate it unless you''re really low on disk space, and with 500G disks on the shelf at Fry''s, there''s really no reason to optimize for that case. It just makes the feature harder to explain. Jeff
> Why not use a terse XML format?I suppose we could, but I''m not convinced that XML is stable enough to be part of a 30-year on-disk format. 15 years ago PostScript was going to be stable forever, but today many PostScript readers barf on Adobe-PS-1.0 files -- which were supposed to be the most portable. They rely on %% structure comments (which were optional in 1.0), they can''t handle a missing ''showpage'', and so on. I know, XML is supposed to be standard. THIS TIME they won''t screw me. I just see way too many hands the XML cookie jar to trust it. You just can''t argue with 8-bit-clean ASCII for long-term stability. Jeff
On Wed, 3 May 2006, Matthew A. Ahrens wrote:> > # zpool history jen > > History for ''jen'': > > 2006-04-27T10:38:36 zpool create jen mirror ... > > I have two suggestions which are just minor nits compared with the rest of this discussion: > > 1. Why do you print a "T" between the date and the time? I think a space would be more readable.Perhaps its an ISO 8601 formatted date? Or is there another preferred format that is popular to timestamps?> 2. When printing the history for a specific pool, I don''t think we should print the "History for <pool>" line. It seems unnecessary, and that way every line of output will have the same format (better for later machine parsing). >Regards, Al Hopper Logical Approach Inc, Plano, TX. al at logical-approach.com Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT OpenSolaris.Org Community Advisory Board (CAB) Member - Apr 2005
Al Hopper wrote:>On Wed, 3 May 2006, Matthew A. Ahrens wrote: > > > >>># zpool history jen >>>History for ''jen'': >>>2006-04-27T10:38:36 zpool create jen mirror ... >>> >>> >>I have two suggestions which are just minor nits compared with the rest of this discussion: >> >>1. Why do you print a "T" between the date and the time? I think a space would be more readable. >> >> > >Perhaps its an ISO 8601 formatted date? > >Yep that''s the reason.>Or is there another preferred format that is popular to timestamps? > > > >>2. When printing the history for a specific pool, I don''t think we should print the "History for <pool>" line. It seems unnecessary, and that way every line of output will have the same format (better for later machine parsing). >> >>I agree. eric