Works great! Even tested to create boot environments with live upgrade, fast, easy and elegant! Anyone knows what the deal with /export/home is? I though /home was the default home directory in Solaris? ( I put up some screenshots of the installation process for those interested: http://sparcv9.blogspot.com ) Regards Henrik
On Wed, 4 Jun 2008, Henrik Johansson wrote:> Anyone knows what the deal with /export/home is? I though /home was > the default home directory in Solaris?Nope, /export/home has always been the *physical* location for users'' home directories. They''re usually automounted under /home, though. -- Rich Teer, SCSA, SCNA, SCSECA CEO, My Online Home Inventory URLs: http://www.rite-group.com/rich http://www.linkedin.com/in/richteer http://www.myonlinehomeinventory.com
On Jun 5, 2008, at 12:05 AM, Rich Teer wrote:> On Wed, 4 Jun 2008, Henrik Johansson wrote: > >> Anyone knows what the deal with /export/home is? I though /home was >> the default home directory in Solaris? > > Nope, /export/home has always been the *physical* location for > users'' home directories. They''re usually automounted under /home, > though.You are right, it was my own old habit of creating my own physical directory under /home for stand alone machines that got me confused. But filesystem(5) says: "/home Default root of a subtree for user directories." and useradd:s base_dir defaults to /home, where it tries to create a directory if used only with the -m flag. I know this doesn''t work when the automounter is available, but it can be disabled or reconfigured. When i think about it /export/home has been created earlier also, with UFS. It''s fun how old things can get one confused in a new context ;) Regards Henrik
On Wed, 4 Jun 2008, Henrik Johansson wrote:> > Anyone knows what the deal with /export/home is? I though /home was > the default home directory in Solaris?It seems that they expect you to use the automounter to mount it. That allows the same automount map to be used for all systems. It has been that way as long as I can remember.> ( I put up some screenshots of the installation process for those > interested: http://sparcv9.blogspot.com )Did you actually choose to keep / and /var combined? Is there any reason to do that with a ZFS root since both are sharing the same pool and so there is no longer any disk space advantage? If / and /var are not combined can they have different assigned quotas without one inheriting limits from the other? Bob =====================================Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
On Wed, 4 Jun 2008, Bob Friesenhahn wrote:> Did you actually choose to keep / and /var combined? Is there anyTHat''s what I''d do...> reason to do that with a ZFS root since both are sharing the same pool > and so there is no longer any disk space advantage? If / and /var are > not combined can they have different assigned quotas without one > inheriting limits from the other?Why would one do that? Just keep an eye on the root pool and all is good. -- Rich Teer, SCSA, SCNA, SCSECA CEO, My Online Home Inventory URLs: http://www.rite-group.com/rich http://www.linkedin.com/in/richteer http://www.myonlinehomeinventory.com
Rich Teer wrote:> On Wed, 4 Jun 2008, Bob Friesenhahn wrote >> Did you actually choose to keep / and /var combined? Is there any >> > > THat''s what I''d do... > >> reason to do that with a ZFS root since both are sharing the same pool >> and so there is no longer any disk space advantage? If / and /var are >> not combined can they have different assigned quotas without one >> inheriting limits from the other? >> > > Why would one do that? Just keep an eye on the root pool and all is good. >It''s the old debate around possibly filling the / partition, should you get some sort of explosion in /var (crash dumps are a big culprit here). IMHO, even though with ZFS making / and /var separate filesystems in the same zpool is trivial, I don''t see the benefit - it''s just something more to manage and pay attention to. If you have something very large going on in /var (like mail spools, or whatever), you''ll be creating a new zpool for that purpose. Otherwise, filling /var can be _bad_ (even if on a different ZFS filesystem), so I don''t see much benefit. But, with ZFS, the counter (it''s so simple, why not?) is also valid. It''s just personal whim, now, really. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800)
I''d expect it''s the old standard. if /var/tmp is filled, and that''s part of /, then bad things happen. there are often other places in /var that are writable by more than root, and always the possibility that something barfs heavily into syslog. Since the advent of reasonably sized disks, I know many don''t consider this an issue these days, but I''d still be inclined to keep /var (and especially /var/tmp) separated from / In ZFS, this is, of course, just two filesystems in the same pool, with differing quotas... :) Nathan. Rich Teer wrote:> On Wed, 4 Jun 2008, Bob Friesenhahn wrote: > >> Did you actually choose to keep / and /var combined? Is there any > > THat''s what I''d do... > >> reason to do that with a ZFS root since both are sharing the same pool >> and so there is no longer any disk space advantage? If / and /var are >> not combined can they have different assigned quotas without one >> inheriting limits from the other? > > Why would one do that? Just keep an eye on the root pool and all is good. >-- ////////////////////////////////////////////////////////////////// // Nathan Kroenert nathan.kroenert at aus.sun.com // // Technical Support Engineer Phone: +61 3 9869-6255 // // Sun Services Fax: +61 3 9869-6288 // // Level 3, 476 St. Kilda Road // // Melbourne 3004 Victoria Australia // //////////////////////////////////////////////////////////////////
In addition to the standard "containing the carnage" arguments used to justify splitting /var/tmp, /var/mail, /var/adm (process accounting etc), is there an interesting use-case where would one split out /var for "compression reasons" (as in, turn on compression for /var so that process accounting, network flow, and other such fun logs can be kept on a compressed filesystem (while keeping / (and thereby /usr etc.) uncompressed?).... The ZFSBOOT-FAQ document doesn''t really show how to break out multiple filesystems with jumpstart profiles... An example there might be helpful... (as its clear this is a frequently asked question :-) Also a "compression on", "ditto-data-bits on", (or perhaps a generic place to insert zpool/zfs parameters) as part of the jumpstart profiles could also be useful... If SSD is coming fast and furious, being able to use compression, shared free-space (quotas etc) to keep the boot-images small enough so they''ll fit and accommodate live-upgrade patching, will become increasingly important. http://www.networkworld.com/news/2008/060308-sun-flash-storage.html?page =1 Rock on guys, -- MikeE -----Original Message----- From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Rich Teer Sent: Thursday, June 05, 2008 12:19 AM To: Bob Friesenhahn Cc: ZFS discuss Subject: Re: [zfs-discuss] ZFS root finally here in SNV90 On Wed, 4 Jun 2008, Bob Friesenhahn wrote:> Did you actually choose to keep / and /var combined? Is there anyTHat''s what I''d do...> reason to do that with a ZFS root since both are sharing the same pool> and so there is no longer any disk space advantage? If / and /var are> not combined can they have different assigned quotas without one > inheriting limits from the other?Why would one do that? Just keep an eye on the root pool and all is good. -- Rich Teer, SCSA, SCNA, SCSECA CEO, My Online Home Inventory URLs: http://www.rite-group.com/rich http://www.linkedin.com/in/richteer http://www.myonlinehomeinventory.com _______________________________________________ zfs-discuss mailing list zfs-discuss at opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
On Thu, 5 Jun 2008, Ellis, Mike wrote:> > If SSD is coming fast and furious, being able to use compression, shared > free-space (quotas etc) to keep the boot-images small enough so they''ll > fit and accommodate live-upgrade patching, will become increasingly > important. > > http://www.networkworld.com/news/2008/060308-sun-flash-storage.html?page=1In the server market, the main value for SSD is to increase transaction rates (particularly read rates). It is mildly interesting that Sun will be introducing SSDs in their server products. It would be much more interesting to see if SSDs are rolled up and plugged into legacy SATA/SAS slots, or if Sun has the hootzbah to be more original and use a much faster path to the OS. It is doubtful that the existing disk I/O path (optimized for long-latency devices) will be able to deliver the IOPS performance of the hardware. A new driver stack is necessary which is optimized for low-latency block devices. ZFS is impacted as well since various ZFS read/write algorithms take latency into consideration. The tunings for slow disks won''t be suitable for SSDs. Bob =====================================Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
On Jun 5, 2008, at 5:09 AM, Bob Friesenhahn wrote:> On Wed, 4 Jun 2008, Henrik Johansson wrote: >> >> Anyone knows what the deal with /export/home is? I though /home was >> the default home directory in Solaris? > > It seems that they expect you to use the automounter to mount it. > That allows the same automount map to be used for all systems. It > has been that way as long as I can remember.Yes, as i wrote earlier, i was a bit confused in all my excitement and thew away 10 years of assumptions that are still true today ;)>> ( I put up some screenshots of the installation process for those >> interested: http://sparcv9.blogspot.com ) > > Did you actually choose to keep / and /var combined? Is there any > reason to do that with a ZFS root since both are sharing the same > pool and so there is no longer any disk space advantage? If / and / > var are not combined can they have different assigned quotas without > one inheriting limits from the other?I dint not spend any time thinking about this, all i have done so far is installation on my workstation and a test-machines. In a production environment i would spend some time thinking about this, but we are at least half a year away from that. As already disused it might be useful for turning on compression on log-files etc. I usually keep / and /var together, but with ZFS you no longer have the issues with sizing the slices... Regards Henrik
Nathan Kroenert wrote:> I''d expect it''s the old standard. > > if /var/tmp is filled, and that''s part of /, then bad things happen. >Such as? If you find a part of Solaris that cannot deal with a full file system, then please (pretty please... with a cherry on top) file a bug.> there are often other places in /var that are writable by more than > root, and always the possibility that something barfs heavily into syslog. > > Since the advent of reasonably sized disks, I know many don''t consider > this an issue these days, but I''d still be inclined to keep /var (and > especially /var/tmp) separated from / > > In ZFS, this is, of course, just two filesystems in the same pool, with > differing quotas... >I agree. If there is a need for different policies (compression, quota, etc.) then it might make sense to have different file systems. OTOH, this complicates your ability to backup/restore the system, so you need to look at all of the implications of doing so. -- richard
Cindy.Swearingen at Sun.COM
2008-Jun-05 17:56 UTC
[zfs-discuss] ZFS root finally here in SNV90
Mike, As we discussed, you can''t currently break out other datasets besides /var. I''ll add this issue to the FAQ. Thanks, Cindy Ellis, Mike wrote:> In addition to the standard "containing the carnage" arguments used to > justify splitting /var/tmp, /var/mail, /var/adm (process accounting > etc), is there an interesting use-case where would one split out /var > for "compression reasons" (as in, turn on compression for /var so that > process accounting, network flow, and other such fun logs can be kept on > a compressed filesystem (while keeping / (and thereby /usr etc.) > uncompressed?).... > > The ZFSBOOT-FAQ document doesn''t really show how to break out multiple > filesystems with jumpstart profiles... An example there might be > helpful... (as its clear this is a frequently asked question :-) > > Also a "compression on", "ditto-data-bits on", (or perhaps a generic > place to insert zpool/zfs parameters) as part of the jumpstart profiles > could also be useful... > > If SSD is coming fast and furious, being able to use compression, shared > free-space (quotas etc) to keep the boot-images small enough so they''ll > fit and accommodate live-upgrade patching, will become increasingly > important. > > http://www.networkworld.com/news/2008/060308-sun-flash-storage.html?page > =1 > > Rock on guys, > > -- MikeE > > > > > -----Original Message----- > From: zfs-discuss-bounces at opensolaris.org > [mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Rich Teer > Sent: Thursday, June 05, 2008 12:19 AM > To: Bob Friesenhahn > Cc: ZFS discuss > Subject: Re: [zfs-discuss] ZFS root finally here in SNV90 > > On Wed, 4 Jun 2008, Bob Friesenhahn wrote: > > >>Did you actually choose to keep / and /var combined? Is there any > > > THat''s what I''d do... > > >>reason to do that with a ZFS root since both are sharing the same pool > > >>and so there is no longer any disk space advantage? If / and /var are > > >>not combined can they have different assigned quotas without one >>inheriting limits from the other? > > > Why would one do that? Just keep an eye on the root pool and all is > good. >
On Thu, 5 Jun 2008, Richard Elling wrote:> Nathan Kroenert wrote: >> I''d expect it''s the old standard. >> >> if /var/tmp is filled, and that''s part of /, then bad things happen. > > Such as? If you find a part of Solaris that cannot deal with a full > file system, then please (pretty please... with a cherry on top) file a > bug.I expect that it matters when the filesystem gets filled up. The functionality offered by /var/run would become quite broken if it was full during boot. No nameservice. No daemon PID files. Doors broken. Boo-Hoo. Bob =====================================Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
On Thu, 5 Jun 2008, Bob Friesenhahn wrote:> I expect that it matters when the filesystem gets filled up. The > functionality offered by /var/run would become quite broken if it was > full during boot. No nameservice. No daemon PID files. Doors > broken. Boo-Hoo./var/run is memory-resident, not disk-resident. -- Rich Teer, SCSA, SCNA, SCSECA CEO, My Online Home Inventory URLs: http://www.rite-group.com/rich http://www.linkedin.com/in/richteer http://www.myonlinehomeinventory.com
Nicolas Williams
2008-Jun-05 18:51 UTC
[zfs-discuss] Full filesystems (Re: ZFS root finally here in SNV90)
On Thu, Jun 05, 2008 at 01:40:21PM -0500, Bob Friesenhahn wrote:> On Thu, 5 Jun 2008, Richard Elling wrote: > > > Nathan Kroenert wrote: > >> I''d expect it''s the old standard. > >> > >> if /var/tmp is filled, and that''s part of /, then bad things happen. > > > > Such as? If you find a part of Solaris that cannot deal with a full > > file system, then please (pretty please... with a cherry on top) file a > > bug. > > I expect that it matters when the filesystem gets filled up. The > functionality offered by /var/run would become quite broken if it was > full during boot. No nameservice. No daemon PID files. Doors > broken. Boo-Hoo./var/run is a tmpfs filesystem. You can cap it of course, but since not much is stored there being out of room in /var/run is typically indicative of being out of memory. Being out of memory is a whole different issue than being out of space in / or /var. I''m pretty sure that years ago there were things that broke when / (and/or /var) filled up, but nowadays the system is much more robust. Nico --
Bob Friesenhahn wrote:> On Thu, 5 Jun 2008, Richard Elling wrote: > > >> Nathan Kroenert wrote: >> >>> I''d expect it''s the old standard. >>> >>> if /var/tmp is filled, and that''s part of /, then bad things happen. >>> >> Such as? If you find a part of Solaris that cannot deal with a full >> file system, then please (pretty please... with a cherry on top) file a >> bug. >> > > I expect that it matters when the filesystem gets filled up. The > functionality offered by /var/run would become quite broken if it was > full during boot. No nameservice. No daemon PID files. Doors > broken. Boo-Hoo. >/var/run is mounted in virtual memory and not disk. So it doesn''t matter if the /var filesystem is full or not. OTOH, if you are out of virtual memory, then I agree there is lots of stuff that would fail. So, we had this argument about 7-8 years ago and to prove it we setup a server providing many services (http, NFS) and wrote a script which would keep / full (not as trivial as you might think). We ran for 6 months before we decided to cancel the experiment (point made, and we got bored :-). We have not repeated the experiment with Solaris 10 or NV, though, so I can''t say that there aren''t any bugs that might have sneaked in. If you find one, then please file a bug. Note: off-alias, someone pointed out a potential bug, so please keep those cards and letters coming! We want Solaris to be much more robust! -- richard
So can I jumpstart and setup the ZFS root config? Anyone have example profile? This message posted from opensolaris.org
On Thu, Jun 05, 2008 at 02:30:51PM -0700, Vincent Fox wrote:> So can I jumpstart and setup the ZFS root config? > > Anyone have example profile? >See TFM - http://opensolaris.org/os/community/zfs/boot/zfsbootFAQ/#jumpinstall has been mentioned in the last day or so on list. vh Mads Toftum -- http://soulfood.dk
Ellis, Mike
2008-Jun-05 22:05 UTC
[zfs-discuss] LiveUpgrade Bug? -- ZFS root finally here in SNV90
If you jumpstart a system, and have it "default" to a shared / and /var, you can do the following: lucreate -n lu1 lustatus Due to a bug you now have to edit: /bootpool/boot/menu.lst Once that''s done, you''re in good shape and both environments are bootable (review boot -L) -- If we do the same thing with a separate / and /var using jumpstart profile entry: bootenv installbe bename zfsboot dataset /var Things appear to be working well.... Lucreate appears to do all the snapshots/clones, and sets some special parameters for /var But when you try to boot from that BE, things die pretty early on in the boot process, likely related to the fact that it didn''t actually mount /var.... Is this a known bug? (is this the proper way to validate, and potentially file bugs for ZFS boot?) Thanks, -- MikeE -----Original Message----- From: Cindy.Swearingen at Sun.COM [mailto:Cindy.Swearingen at Sun.COM] Sent: Thursday, June 05, 2008 1:56 PM To: Ellis, Mike Cc: ZFS discuss Subject: Re: [zfs-discuss] ZFS root finally here in SNV90 Mike, As we discussed, you can''t currently break out other datasets besides /var. I''ll add this issue to the FAQ. Thanks, Cindy
On Wed, Jun 4, 2008 at 11:18 PM, Rich Teer <rich.teer at rite-group.com> wrote:> Why would one do that? Just keep an eye on the root pool and all is good.The only good argument I have for separating out some of /var is for boot environment management. I grew tired of repeating my arguments and suggestions and wrote a blog entry. http://mgerdts.blogspot.com/2008/03/future-of-opensolaris-boot-environment.html -- Mike Gerdts http://mgerdts.blogspot.com/
Mike Gerdts wrote:> On Wed, Jun 4, 2008 at 11:18 PM, Rich Teer <rich.teer at rite-group.com> wrote: > >> Why would one do that? Just keep an eye on the root pool and all is good. >> > > The only good argument I have for separating out some of /var is for > boot environment management. I grew tired of repeating my arguments > and suggestions and wrote a blog entry. > > http://mgerdts.blogspot.com/2008/03/future-of-opensolaris-boot-environment.html > >Sorry it''s taken me so long to weigh in on this. The reason that the install program permits the user to set up a separate /var dataset is because some production environments require it. More exactly, some production environments require that /var have its own slice so that unrestrained growth in /var can''t fill up the root file system. (I have no idea whether this is actually a good or sufficient policy. I just know that some customer environments have such a policy.) With zfs, we don''t actually have to put /var in its own slice. We can achieve the same goal by putting it in its own dataset and assigning a quota to that dataset. That''s really the only reason we offered this option. Lori -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080623/11e8f94e/attachment.html>
Wouldnt it be nice to break out all file systems in separate zfs file systems? Then you could snapshot each file system individually. Just like each user has his own filesystem, and I can snapshot that filesystem independently from other users. Because of now, if I do a snapshot of /, then everything gets snapshotted, even /var which changes a lot. I dont want to snapshot /var. I only want to snapshot /usr. This message posted from opensolaris.org
On Mon, Jun 23, 2008 at 4:04 PM, Orvar Korvar <knatte_fnatte_tjatte at yahoo.com> wrote:> Wouldnt it be nice to break out all file systems in separate zfs file systems? Then you could snapshot each file system individually. Just like each user has his own filesystem, and I can snapshot that filesystem independently from other users. > > Because of now, if I do a snapshot of /, then everything gets snapshotted, even /var which changes a lot. I dont want to snapshot /var. I only want to snapshot /usr. >Some things in /var are likely appropriate to snapshot with /. For example, /var/sadm has lots of information about which packages and patches are installed. There is a lot of other stuff that shouldn''t be snapshotted with it. I have proposed /var/share to cope with this. http://mgerdts.blogspot.com/2008/03/future-of-opensolaris-boot-environment.html -- Mike Gerdts http://mgerdts.blogspot.com/
On Mon, Jun 23, 2008 at 11:18:21AM -0600, Lori Alt wrote:> Sorry it''s taken me so long to weigh in on this.You''re busy with important things, we''ll forgive you. ;)> With zfs, we don''t actually have to put /var in its own > slice. We can achieve the same goal by putting it > in its own dataset and assigning a quota to that dataset. > > That''s really the only reason we offered this option.And thank you for doing so. I will always put /var in it''s own "area" even if the definition of that area has changed with the use of ZFS. Rampant writes to /var can *still* run / out of space even on ZFS, being able to keep that from happening is never a bad idea as far as I''m concerned. :) -brian -- "Coding in C is like sending a 3 year old to do groceries. You gotta tell them exactly what you want or you''ll end up with a cupboard full of pop tarts and pancake mix." -- IRC User (http://www.bash.org/?841435)
Brian Hechinger wrote:> On Mon, Jun 23, 2008 at 11:18:21AM -0600, Lori Alt wrote: > >> Sorry it''s taken me so long to weigh in on this. >> > > You''re busy with important things, we''ll forgive you. ;) > > >> With zfs, we don''t actually have to put /var in its own >> slice. We can achieve the same goal by putting it >> in its own dataset and assigning a quota to that dataset. >> >> That''s really the only reason we offered this option. >> > > And thank you for doing so. I will always put /var in it''s own "area" > even if the definition of that area has changed with the use of ZFS. > > Rampant writes to /var can *still* run / out of space even on ZFS, being > able to keep that from happening is never a bad idea as far as I''m > concerned. :) > >I think the ability to have different policies for file systems is pure goodness -- though you pay for it on the backup/ restore side. A side question though, my friends who run Windows, Linux, or OSX don''t seem to have this bias towards isolating /var. Is this a purely Solaris phenomenon? If so, how do we fix it? -- richard
On Mon, Jun 23, 2008 at 8:45 PM, Richard Elling <Richard.Elling at sun.com> wrote:> Brian Hechinger wrote: > > On Mon, Jun 23, 2008 at 11:18:21AM -0600, Lori Alt wrote: > > > >> Sorry it''s taken me so long to weigh in on this. > >> > > > > You''re busy with important things, we''ll forgive you. ;) > > > > > >> With zfs, we don''t actually have to put /var in its own > >> slice. We can achieve the same goal by putting it > >> in its own dataset and assigning a quota to that dataset. > >> > >> That''s really the only reason we offered this option. > >> > > > > And thank you for doing so. I will always put /var in it''s own "area" > > even if the definition of that area has changed with the use of ZFS. > > > > Rampant writes to /var can *still* run / out of space even on ZFS, being > > able to keep that from happening is never a bad idea as far as I''m > > concerned. :) > > > > > > I think the ability to have different policies for file systems > is pure goodness -- though you pay for it on the backup/ > restore side. > > A side question though, my friends who run Windows, > Linux, or OSX don''t seem to have this bias towards isolating > /var. Is this a purely Solaris phenomenon? If so, how do we > fix it?I don''t think it''s a Solaris phenomenon, and it''s not really a /var thing. UNIX heads have always had to contend with the disaster that is a full / filesystem. /var was always the most common culprit for causing it to run out of space. If you talk to the really paranoid among us, we run a read-only root filesystem. The real way to "fix" it, in zfs terms, is to reserve a minimum amount of space in / - thereby guaranteeing that you don''t fill up your root filesystem.> > -- richard > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-- chris -at- microcozm -dot- net === Si Hoc Legere Scis Nimium Eruditionis Habes -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080623/aebd60e0/attachment.html>
On Mon, Jun 23, 2008 at 05:45:45PM -0700, Richard Elling wrote:> > I think the ability to have different policies for file systems > is pure goodness -- though you pay for it on the backup/ > restore side.That''s a price I for one am willing to pay. ;)> A side question though, my friends who run Windows, > Linux, or OSX don''t seem to have this bias towards isolating > /var. Is this a purely Solaris phenomenon? If so, how do we > fix it?This is not a purely Solaris phenomenon, this is a UNIX phenomenon. People who run Linux or OSX (I can''t speak for Windows users) tend to be "new to the game" and feel that "This 40/80/500GB disk will never fill up" and so don''t believe that seperating /var is needed. It doesn''t matter how big your disk is, a rampant process can fill up any disk of any size, it''s just a matter of how long it takes. It isn''t just /var that can cause trouble either, it''s just that /var is the "usual suspect" since it''s the filesystem that tends to get written to by the largest number of different processes. /tmp on a system that doesn''t do tmpfs (BSD for example) is another likely candidate. Keeping / as far away from everything else as possible is never a bad idea. ZFS only makes this task easier (IMHO) since you can set quotas and reserves on different filesystems, thus protecting yourself from damage, and also at the same time not "wasting" disk space that could be better used elsewhere. Opinionated? Me? Yes. ;) -brian -- "Coding in C is like sending a 3 year old to do groceries. You gotta tell them exactly what you want or you''ll end up with a cupboard full of pop tarts and pancake mix." -- IRC User (http://www.bash.org/?841435)
Hi All, the separating of /var is something that comes from the Unix tradition. Much of the Unix tradition of systems administration is based on making sure systems with many users remain stable and so administrators are prepared to work to make the system more reliable. Common Windows, Linux and OS X practices are dominated by the concept of a personal computer ie you only hurt yourself so ease is a priority to them. The original filesystem layout separated / /var /tmp /usr onto separate filessytems. In the bad old days every time there is a write there is risk that the filesystem may be made unstable so the aim was to minimise writes to / as without / booting to a minimal environment is a serious trial. /tmp was used for data that is not required to persist over reboots. /var was used for data that should persist over reboots The other filesystems were used to store user files / non-minimal boot programs etc By separating the filesystems it is possible to make a far more recoverable system in the event of: - a user deciding to fill up all of one piece of temporary storage (ramdisk /tmp was one of those optimisations that sun made that had some serious negative consequences; many admins on large shared systems make it back into a disk backed filesystem) - high write rate to other filesystems reduces risk of boot affecting writes from being made So keeping /var and /tmp separate make life much easier. Some of us have even been known to run with a read-only root filesystem. Linux and windows users appear to value the flexibility of not having to make system use decisions ie how big /var and /tmp should be at installation and being able to use the disk as they see fit; however, they are typically not managing systems for others and so they have made a choice of convenience which can be seriously inconvenient in a shared environment. Maurice Castro On 24/06/2008, at 10:45 AM, Richard Elling wrote:> > I think the ability to have different policies for file systems > is pure goodness -- though you pay for it on the backup/ > restore side. > > A side question though, my friends who run Windows, > Linux, or OSX don''t seem to have this bias towards isolating > /var. Is this a purely Solaris phenomenon? If so, how do we > fix it? > -- richard
Maurice Castro wrote:> Hi All, > the separating of /var is something that comes from the Unix > tradition. Much of the Unix tradition of systems administration is > based on making sure systems with many users remain stable and so > administrators are prepared to work to make the system more reliable. > Common Windows, Linux and OS X practices are dominated by the concept > of a personal computer ie you only hurt yourself so ease is a priority > to them. >So the consensus is that if we are to compete with them at the desktop, then a simpler, easier to maintain file system structure is cool.> The original filesystem layout separated > / > /var > /tmp > /usr > onto separate filessytems. In the bad old days every time there is a > write there is risk that the filesystem may be made unstable so the > aim was to minimise writes to / as without / booting to a minimal > environment is a serious trial. >Actually, no. There was /. /var didn''t show up until SunOS 4 circa 1988. /usr was made separable when it began to grow bigger than the disks available at the time, circa 1986 or so. In any case, well after UNIX was established.> /tmp was used for data that is not required to persist over reboots. > /var was used for data that should persist over reboots > > The other filesystems were used to store user files / non-minimal boot > programs etc > > By separating the filesystems it is possible to make a far more > recoverable system in the event of: > - a user deciding to fill up all of one piece of temporary storage > (ramdisk /tmp was one of those optimisations that sun made that had > some serious negative consequences; many admins on large shared > systems make it back into a disk backed filesystem) > - high write rate to other filesystems reduces risk of boot affecting > writes from being made >The reason for separating was very different, though this was also a side-product. In the days of diskless systems, you could share parts of the OS as read-only. /usr originally contained /usr/tmp and /usr/Richard (or whatever). To make /usr be read-only, user home directories and tmp had to be moved out to /var. / also was unique to each diskless client, so while they could share much of the stuff in /usr, each had to have its own /. Some people also took advantage of the fact that /usr/spool is now in /var/spool, where the printer files and mail collected and separated these out so that you wouldn''t have to back them up (tapes were 60MBytes or so at the time). So, there you have it: /, /var, /usr, and /export/home. Each has a different policy, which is the key here. NB. UFS, by default, reserves 10% which only root can write. So a regular user could not directly impact a running UNIX system using UFS. ZFS does not have such reserve, so if you want to implement it, you will end up with a separate file system somewhere. -- richard> So keeping /var and /tmp separate make life much easier. Some of us > have even been known to run with a read-only root filesystem. > > Linux and windows users appear to value the flexibility of not having > to make system use decisions ie how big /var and /tmp should be at > installation and being able to use the disk as they see fit; however, > they are typically not managing systems for others and so they have > made a choice of convenience which can be seriously inconvenient in a > shared environment. > > Maurice Castro > > On 24/06/2008, at 10:45 AM, Richard Elling wrote: > > >> I think the ability to have different policies for file systems >> is pure goodness -- though you pay for it on the backup/ >> restore side. >> >> A side question though, my friends who run Windows, >> Linux, or OSX don''t seem to have this bias towards isolating >> /var. Is this a purely Solaris phenomenon? If so, how do we >> fix it? >> -- richard >> > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
On Mon, Jun 23, 2008 at 8:06 PM, Brian Hechinger <wonko at 4amlunch.net> wrote:> This is not a purely Solaris phenomenon, this is a UNIX phenomenon. > People who run Linux or OSX (I can''t speak for Windows users) tend to > be "new to the game" and feel that "This 40/80/500GB disk will never > fill up" and so don''t believe that seperating /var is needed.Why is having a full /var so much better than having a full /? I''ve had a number of Solaris systems fail to boot because it can''t update /var/adm/utmpx, but I''ve never had one fail to boot because / was full. As best as I can deduce, the "root file system corruption when it gets full" is a combination of ancient history and urban legend. I''ve brought this up on a lengthy thread over at sysadmin-discuss a while back and have had no one refute my assertion with credible data. http://mail.opensolaris.org/pipermail/sysadmin-discuss/2007-September/001668.html I''ve also shared more detailed thoughts on file system sprawl at... http://mail.opensolaris.org/pipermail/sysadmin-discuss/2007-September/001641.html Really it boils down to lots of file systems to hold the OS adds administrative complexity and rarely saves more work than it creates. I believe this especially holds true for enterprise server environments where downtime is really expensive. I much prefer to ask for a 3 hour outage to patch than a 5 hour outage to relayout file systems then patch. Of course today''s development work will make the 3 hour outage for patching a thing of ancient history as well. -- Mike Gerdts http://mgerdts.blogspot.com/
Mike Gerdts writes:> On Mon, Jun 23, 2008 at 8:06 PM, Brian Hechinger <wonko at 4amlunch.net> wrote: >> This is not a purely Solaris phenomenon, this is a UNIX phenomenon. >> People who run Linux or OSX (I can''t speak for Windows users) tend to >> be "new to the game" and feel that "This 40/80/500GB disk will never >> fill up" and so don''t believe that seperating /var is needed. >WIth ZFS boot, the point is moot.> Why is having a full /var so much better than having a full /? I''ve > had a number of Solaris systems fail to boot because it can''t update > /var/adm/utmpx, but I''ve never had one fail to boot because / was > full. As best as I can deduce, the "root file system corruption when > it gets full" is a combination of ancient history and urban legend. > I''ve brought this up on a lengthy thread over at sysadmin-discuss a > while back and have had no one refute my assertion with credible data. >We can only hope that ZFS boot will consign this never ending layout argument to the dust of history. Ian
On Mon, Jun 23, 2008 at 10:25:09PM -0500, Mike Gerdts wrote:> > Really it boils down to lots of file systems to hold the OS adds > administrative complexity and rarely saves more work than it creates.Some of us want to use different mount options on /var than on /. That''s why they need to be different filesystems. For example, our policy is that anything containing user-writable directories must be mounted without setuid and without devices. /tmp and /home must also be separate filesystes, but this is generally already the case. -- -Gary Mills- -Unix Support- -U of M Academic Computing and Networking-
On Tue, Jun 24, 2008 at 04:35:30PM +1200, ian at ianshome.com wrote:> > >On Mon, Jun 23, 2008 at 8:06 PM, Brian Hechinger <wonko at 4amlunch.net> > >wrote: > >>This is not a purely Solaris phenomenon, this is a UNIX phenomenon. > >>People who run Linux or OSX (I can''t speak for Windows users) tend to > >>be "new to the game" and feel that "This 40/80/500GB disk will never > >>fill up" and so don''t believe that seperating /var is needed. > > WIth ZFS boot, the point is moot.Actually, it''s not. ZFS doesn''t magically make your disks not run out of space. Even with ZFS Boot you can still run your / or /var out of space. If you think otherwise, you are only fooling yourself.> We can only hope that ZFS boot will consign this never ending layout > argument to the dust of history.The layout of disks and filesystems will always be a personal preference and will never go away. You are likely to never agree with the way I feel servers should be setup, and I''ll likely never agree with the way you feel servers should be setup. -brian -- "Coding in C is like sending a 3 year old to do groceries. You gotta tell them exactly what you want or you''ll end up with a cupboard full of pop tarts and pancake mix." -- IRC User (http://www.bash.org/?841435)
On Tue, Jun 24, 2008 at 7:24 AM, Gary Mills <mills at cc.umanitoba.ca> wrote:> On Mon, Jun 23, 2008 at 10:25:09PM -0500, Mike Gerdts wrote: >> >> Really it boils down to lots of file systems to hold the OS adds >> administrative complexity and rarely saves more work than it creates. > > Some of us want to use different mount options on /var than on /. > That''s why they need to be different filesystems. For example, our > policy is that anything containing user-writable directories must > be mounted without setuid and without devices. /tmp and /home must > also be separate filesystes, but this is generally already the case.In your case, having separate file systems makes sense to me. You are using multiple file systems to implement a requirement that presumably came out of a risk analysis. I can respect that. My advice against lots of file systems is aimed at those that separate file systems because that''s what the installer''s defaults were or because of fear of some bug that may or may not have existed sometime last century. In the end, sysadmins that have a strong preference and those that have a requirement for separate file systems are not going to change in the near future. While the defaults may change (and have changed) over time, the option to support such configurations is a clear requirement going forward. -- Mike Gerdts http://mgerdts.blogspot.com/
On Mon, 23 Jun 2008, Richard Elling wrote:> A side question though, my friends who run Windows, > Linux, or OSX don''t seem to have this bias towards isolating > /var. Is this a purely Solaris phenomenon? If so, how do we > fix it?In addition to the other answers given, I think another reason why this is common among Solaris sysadmins (though its not a practise I endorse) is the fact that we''ve been around longer than the usual Linux and Windoze "admins". Back when we started our careers, disks were relatively tiny, so one had no option but to split up /, /usr, /var, and so on if meaningful uptimes were to be achieved. With today''s huge boot disks, I think the practise is somewhat deprecated. The untrue myth about systems panicing when / can''t be written to because its full doesn''t help, either... -- Rich Teer, SCSA, SCNA, SCSECA CEO, My Online Home Inventory URLs: http://www.rite-group.com/rich http://www.linkedin.com/in/richteer http://www.myonlinehomeinventory.com
On 6/23/08 7:45 PM, "Richard Elling" <Richard.Elling at Sun.COM> wrote:> I think the ability to have different policies for file systems > is pure goodness -- though you pay for it on the backup/ > restore side.And another reason why Automated Data Migration is the way to go. "Backup" and "replication" schemes are problematic for a number of reasons, not the least of which is "remembering what filesystems to back up." If the filesystem itself is taking care of things, you''re probably more likely to "get it right." This is especially true if, like ZFS object propoerties, ADM properties are inherited. E.g., set / to back up, and everything else is automatically protected. Go "custom" wherever appropriate. Having already been saved by a Time Machine backup that occurred just 20 minutes before my laptop hard drive died, I''m convinced Apple got it right in making the default a "back up everything" approach. While their solution isn''t integrated into the filesystem, as I expect ADM will be with ZFS, there''s something to be said for the "whole system approach."> A side question though, my friends who run Windows, > Linux, or OSX don''t seem to have this bias towards isolating > /var. Is this a purely Solaris phenomenon? If so, how do we > fix it? > -- richardWell, having spent a lot of time on the IRIX side, yeah, it''s just a "Solaris thing." And S10U5 at least now only defaults to TWO partitions (bigger / than before, and /export/home). Baby steps, I suppose :) Charles
Brian Hechinger wrote:> On Tue, Jun 24, 2008 at 04:35:30PM +1200, ian at ianshome.com wrote: >> We can only hope that ZFS boot will consign this never ending layout >> argument to the dust of history. >> > > The layout of disks and filesystems will always be a personal preference > and will never go away. You are likely to never agree with the way I > feel servers should be setup, and I''ll likely never agree with the way you > feel servers should be setup. > >But with ZFS, the layout is a simple matter of creating filesystems, not slicing up drives. The filesystems are in a shared pool, so one doesn''t have to know how big a particular slice will grow. Once we can break out more datasets, the wheel will have turned full circle and we will be able to assign different properties to individual "slices", such as read only /usr. Ian
On Wed, Jun 25, 2008 at 04:41:56PM +1200, Ian Collins wrote:> > But with ZFS, the layout is a simple matter of creating filesystems, not > slicing up drives. The filesystems are in a shared pool, so one doesn''t > have to know how big a particular slice will grow.Yes, we are agreeing on that part. ;)> Once we can break out more datasets, the wheel will have turned full > circle and we will be able to assign different properties to individual > "slices", such as read only /usr.I look forward to that day. Actually, a more granulated /var would thrill me. I gave up on the read-only /usr thing years ago. ;) -brian -- "Coding in C is like sending a 3 year old to do groceries. You gotta tell them exactly what you want or you''ll end up with a cupboard full of pop tarts and pancake mix." -- IRC User (http://www.bash.org/?841435)