Hi, I''m still debating wether I should use ZFS or not and how. Here is my scenario. I want to run a server with a lot of storage, that gets disks added/upgraded from time to time to expand space. I''d want to store large files on it, 15mb - 5gb per file, and they''d only need to be accessible via NFS/FTP and maybe CIFS. The machine I have dedicated for this job would be a slow (2800+ S754) AMD64 with about 1.5G of slow ram (166Mhz (DDR333), though I could put in 200Mhz ram if it makes a BIG difference). The storage pool would look as follows, 2x 160GB, preferably as mirror or similar for OS +initial storage, IDE. 4x 500GB on a sil2124 Sata2 PCI/PCI-X controller, and later 4x300GB IDE via HPT374 PCI controller. Now I''ve been reading a lot about ZFS and it seems like it''s the ''one FS to rule them all''. So here comes, should I use it? Traditionally I would run a software raid1/5 on each of the collection of disks, and LVM2 them together with reiserfs/jfs/xfs or even ext on top of it. This has worked reasonably well in the past but expanding/replacing/redistributing storage etc has been tricky. Ideally I''d like to use ZFS on Gentoo, but since my only option there is using the slow fuse, I''m also worried about stability. Is it ''ready'' for reliable usage so to speak. I know ZFS cant'' be used as a boot partition (yet) so i''d be ok with using 10 - 25gb from the 2 160''s in mirror for the OS/Swap and all of the rest with ZFS. Also the support for 64bit in linux/gentoo is readily available, which I heard is ''best'' for ZFS? My second idea then was to use FreeBSD, not cause I know it all that well, but there is a Gentoo/BSD which would make things easier for me. Unfortunately Gentoo/freeBSD64 doesn''t exist yet :/ and if I have to learn ''regular'' freeBSD I might as well take the extra step and learn opensolaris, not something i''d want to do learning new stuff isn''t something i''m getting excited about at the moment :p However, I found on the liveDVD/CD that nexentia and beleniX both don''t come in x86_64 flavors? Or does solaris autodetect and auto run in 64bit mode at boottime? Finally, I hope it''s safe to assume that once I have one of these 2/3 options up and running, I can always change right? Say I set up an opensolaris server, but decide 2 years down the line I''d want to switch to a linux (which would then hopefully support native ZFS) it would be just a matter of exporting/importing the pool? Or is ZFS not right for me and I should choose one of the traditional methods, and why? Thanks for your time, Oliver
Oliver Schinagl wrote:> However, I found on the liveDVD/CD that nexentia and beleniX both don''t > come in x86_64 flavors? Or does solaris autodetect and auto run in 64bit > mode at boottime?Yes Solaris auto detects, but note that some Live systems and install images only do 32bit but will install 64 bit which is what will run on boot of the installed version. -- Darren J Moffat
>However, I found on the liveDVD/CD that nexentia and beleniX both don''t >come in x86_64 flavors? Or does solaris autodetect and auto run in 64bit >mode at boottime?Solaris autodetects the CPU type and boots in 64 bit mode on 64 bit CPUs and in 32 bit mode on 32 bit CPUs. large parts of the runtime (executables) exist only in 32 bit varieties to save space; but libraries by and large come in both 32 and 64 bit variants as do device drivers and kernel modules.>Finally, I hope it''s safe to assume that once I have one of these 2/3 >options up and running, I can always change right? Say I set up an >opensolaris server, but decide 2 years down the line I''d want to switch >to a linux (which would then hopefully support native ZFS) it would be >just a matter of exporting/importing the pool?If the disk labeling chosen is supported by both OSes. Casper
Casper.Dik at Sun.COM wrote:>> However, I found on the liveDVD/CD that nexentia and beleniX both don''t >> come in x86_64 flavors? Or does solaris autodetect and auto run in 64bit >> mode at boottime? >> > > Solaris autodetects the CPU type and boots in 64 bit mode on 64 bit > CPUs and in 32 bit mode on 32 bit CPUs. > > large parts of the runtime (executables) exist only in 32 bit varieties > to save space; but libraries by and large come in both 32 and 64 bit > variants as do device drivers and kernel modules. > >once I boot it in 64bit mode, i''d have to run emulation libraries to run 32bit bins right? (As I''m a solaris newbie, and only know a little about 64bit stuff from the Linux world, this is all I know) Since it will be a very light server (only bare binaries to run my req.s) they probably end up most likely all being 64bit?>> Finally, I hope it''s safe to assume that once I have one of these 2/3 >> options up and running, I can always change right? Say I set up an >> opensolaris server, but decide 2 years down the line I''d want to switch >> to a linux (which would then hopefully support native ZFS) it would be >> just a matter of exporting/importing the pool? >> > > If the disk labeling chosen is supported by both OSes. >Since this is all new terrain to me, I''m sorry if this is a dumb question. But what are disklabels? or more importantly, which ones are compatible? Googling for it doesn''t help much, either you do actually mean labels that you stick on a CD/DVD :p I doubt it''s what the fdisk world would call partition type, as that can easly be changed with fdisk. The linux kernel does have an ''Advanced partition selection'' where you can choose among many types, one being UFS (that''s what the BSD''s and Solaris use if i''m not mistaken). I assume ZFS would create a label even if i give it the ''entire'' disk without creating any partitions at all? Or is there a magic ''ideal'' label that would work well (performance wise) with both/three OSes?> Casper > >Oliver
Oliver Schinagl wrote:> Casper.Dik at Sun.COM wrote: > >>> However, I found on the liveDVD/CD that nexentia and beleniX both don''t >>> come in x86_64 flavors? Or does solaris autodetect and auto run in 64bit >>> mode at boottime? >>> >>> >> Solaris autodetects the CPU type and boots in 64 bit mode on 64 bit >> CPUs and in 32 bit mode on 32 bit CPUs. >> >> large parts of the runtime (executables) exist only in 32 bit varieties >> to save space; but libraries by and large come in both 32 and 64 bit >> variants as do device drivers and kernel modules. >> >> >> > once I boot it in 64bit mode, i''d have to run emulation libraries to run > 32bit bins right?No. Solaris has both 32 and 64 bit libraries. Ian
Ian Collins wrote:> Oliver Schinagl wrote: > >> Casper.Dik at Sun.COM wrote: >> >> >>>> However, I found on the liveDVD/CD that nexentia and beleniX both don''t >>>> come in x86_64 flavors? Or does solaris autodetect and auto run in 64bit >>>> mode at boottime? >>>> >>>> >>>> >>> Solaris autodetects the CPU type and boots in 64 bit mode on 64 bit >>> CPUs and in 32 bit mode on 32 bit CPUs. >>> >>> large parts of the runtime (executables) exist only in 32 bit varieties >>> to save space; but libraries by and large come in both 32 and 64 bit >>> variants as do device drivers and kernel modules. >>> >>> >>> >>> >> once I boot it in 64bit mode, i''d have to run emulation libraries to run >> 32bit bins right? >> > No. Solaris has both 32 and 64 bit libraries. > > Ian > >so you are saying that i can run both 32bit and 64bit code simultaneously, natively with the solaris kernel? That''s pretty damn cool
Oliver Schinagl wrote:> Ian Collins wrote: > >> Oliver Schinagl wrote: >>> once I boot it in 64bit mode, i''d have to run emulation libraries to run >>> 32bit bins right? >>> >> No. Solaris has both 32 and 64 bit libraries. >> > so you are saying that i can run both 32bit and 64bit code > simultaneously, natively with the solaris kernel? That''s pretty damn cool >I wouldn''t consider an OS where I couldn''t! Ian
Ian Collins wrote:> Oliver Schinagl wrote: > >> Ian Collins wrote: >> >> >>> Oliver Schinagl wrote: >>> >>>> once I boot it in 64bit mode, i''d have to run emulation libraries to run >>>> 32bit bins right? >>>> >>>> >>> No. Solaris has both 32 and 64 bit libraries. >>> >>> >> so you are saying that i can run both 32bit and 64bit code >> simultaneously, natively with the solaris kernel? That''s pretty damn cool >> >> > I wouldn''t consider an OS where I couldn''t! > > Ian >not to start a flamewar or the like, but linux can run 32bit bins, just not nativly afaik, you need some sort of emu library. But since I use gentoo, and pretty much everything is compiled from source anyway, I only have stupid closed src bins that would need to work in 32bit mode to begin with. So to me, that wouldn''t really be an issue :)
>once I boot it in 64bit mode, i''d have to run emulation libraries to run >32bit bins right? (As I''m a solaris newbie, and only know a little about >64bit stuff from the Linux world, this is all I know)No; you run the exact same binaries and libraries under 32 and 64 bit. It''s not emulation; it''s basically two syscall entry tables one for 32 bit and one for 64 bit, mostly sharing the same code except where pointer sizes matter.>Since it will be a very light server (only bare binaries to run my >req.s) they probably end up most likely all being 64bit?No, most binaries are 32 bit only those that need to be 64 bit are 64 bit.>>> Finally, I hope it''s safe to assume that once I have one of these 2/3 >>> options up and running, I can always change right? Say I set up an >>> opensolaris server, but decide 2 years down the line I''d want to switch >>> to a linux (which would then hopefully support native ZFS) it would be >>> just a matter of exporting/importing the pool? >>> >> >> If the disk labeling chosen is supported by both OSes. >> >Since this is all new terrain to me, I''m sorry if this is a dumb >question. But what are disklabels? or more importantly, which ones are >compatible? Googling for it doesn''t help much, either you do actually >mean labels that you stick on a CD/DVD :pThe disklabel is the first bit of the disk which describes what types of bits are on the disk: typically you have FDISK "labels" on PCs but you can also have EFI labels.>I doubt it''s what the fdisk world would call partition type, as that can >easly be changed with fdisk. The linux kernel does have an ''Advanced >partition selection'' where you can choose among many types, one being >UFS (that''s what the BSD''s and Solaris use if i''m not mistaken). I >assume ZFS would create a label even if i give it the ''entire'' disk >without creating any partitions at all? Or is there a magic ''ideal'' >label that would work well (performance wise) with both/three OSes?It''s not the partition type which matters; it''s what bits are used to tell what partitions are on the disk. Casper
>so you are saying that i can run both 32bit and 64bit code >simultaneously, natively with the solaris kernel? That''s pretty damn coolCorrect. There are several reasons for an OS which generally comes in binary distributions to do so: - maintain complete binary compatibility with old applications (and yes, closed source does matter, even to a lot of Linux customers) - allows a single distribution to work on both 32 and 64 bit systems of the same architecture. Casper
Oliver Schinagl wrote:> not to start a flamewar or the like, but linux can run 32bit bins, just not nativly afaik, you need some sort of emu library. But since I use gentoo, and pretty much everything is compiled from source anyway, I only have stupid closed src bins that would need to work in 32bit mode to begin with. So to me, that wouldn''t really be an issue :)Actually, the x86_64 linux kernel will run 32bit applications natively. The X86_64 instruction set changes in 64bit mode which cause problems are all in the privileged instruction set and this is purely a kernel domain. What the kernel has to cope with, however, are the different data sizes. As with Solaris, Linux running 32bit apps with a 64bit kernel needs to have the 32bit libraries so that the programs can interface with the kernel correctly. It''s just the same. The problem is that there a number of religious zealots in the Linux world who would have you believe that a 64bit operating system is only ideologically pure if all the apps are 64bit as well and hence don''t supply 32bit libraries. For the most part, 32bit binaries will run faster (my benchmarks on both Linux and Solaris generally get a x2 speed improvement) than their 64bit counterparts, probably due to the extra memory bandwidth and cache space they need, on the same system and kernel. Only a very few programs need to access more than 3GB of memory. This is why, by default, applications on Solaris are built as 32bit binaries even on 64bit hardware. It also makes the binaries more portable. Anyway, this is getting way off the topic of ZFS. Solaris makes an excellent NFS/CIFS server (with Samba for the CIFS part). ZFS itself is excellent, in my experience, as long as you don''t want fine-grained control over user space usage. As I''ve just discovered, the current work-around for this is to use the underlying, excellent, zpool technology to manage the storage itself and overlay another filesystem in a zvol for those circumstances where ZFS isn''t there yet. I hope that helps. Steve -- --------------------------------------------------------------------------- Computer Systems Administrator, E-Mail:-steve at earth.ox.ac.uk Department of Earth Sciences, Tel:- +44 (0)1865 282110 University of Oxford, Parks Road, Oxford, UK. Fax:- +44 (0)1865 272072
Stephen Usher <Stephen.Usher at earth.ox.ac.uk> wrote:> Oliver Schinagl wrote: > > not to start a flamewar or the like, but linux can run 32bit bins, just not nativly afaik, you need some sort of emu library. But since I use gentoo, and pretty much everything is compiled from source anyway, I only have stupid closed src bins that would need to work in 32bit mode to begin with. So to me, that wouldn''t really be an issue :) > > Actually, the x86_64 linux kernel will run 32bit applications natively. > The X86_64 instruction set changes in 64bit mode which cause problems > are all in the privileged instruction set and this is purely a kernel > domain. What the kernel has to cope with, however, are the different > data sizes. > > As with Solaris, Linux running 32bit apps with a 64bit kernel needs to > have the 32bit libraries so that the programs can interface with the > kernel correctly. It''s just the same. The problem is that there a number > of religious zealots in the Linux world who would have you believe that > a 64bit operating system is only ideologically pure if all the apps are > 64bit as well and hence don''t supply 32bit libraries.I am not sure about the current state, but 2 years ago, Linux was only able to run a few simple proramg in 32 bit mode because the drivers did not support 32 bit ioctl interfaces. This made e.g. a 32 bit cdrecord on 64 bit Linux impossible.> For the most part, 32bit binaries will run faster (my benchmarks on both > Linux and Solaris generally get a x2 speed improvement) than their 64bitMy tests show a typical 30% speed improvement for 64 bit code caused by the fact that the x64 architecture offers twice as many registers than x86. On Sparc systems, 64 bit code typically runs 10-20% slower probably because of higher memory requirements. J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily
Joerg Schilling wrote:> I am not sure about the current state, but 2 years ago, Linux was only able to > run a few simple proramg in 32 bit mode because the drivers did not support > 32 bit ioctl interfaces. This made e.g. a 32 bit cdrecord on 64 bit Linux > impossible.I''ve never had a 32bit binary not work under Mandriva Linux. Then again, the code I''m running doesn''t do anything low-level (other than normal tty ioctls).>> For the most part, 32bit binaries will run faster (my benchmarks on both >> Linux and Solaris generally get a x2 speed improvement) than their 64bit > > My tests show a typical 30% speed improvement for 64 bit code caused by the fact > that the x64 architecture offers twice as many registers than x86.Funnily enough I''ve never, ever had code run faster when compiled 64 bit, either with gcc or the Sun Studio compilers on x86_64 machines.> On Sparc systems, 64 bit code typically runs 10-20% slower probably because of > higher memory requirements.I concur with those findings. Steve -- --------------------------------------------------------------------------- Computer Systems Administrator, E-Mail:-steve at earth.ox.ac.uk Department of Earth Sciences, Tel:- +44 (0)1865 282110 University of Oxford, Parks Road, Oxford, UK. Fax:- +44 (0)1865 272072
Stephen Usher <Stephen.Usher at earth.ox.ac.uk> wrote:> Joerg Schilling wrote: > > I am not sure about the current state, but 2 years ago, Linux was only able to > > run a few simple proramg in 32 bit mode because the drivers did not support > > 32 bit ioctl interfaces. This made e.g. a 32 bit cdrecord on 64 bit Linux > > impossible. > > I''ve never had a 32bit binary not work under Mandriva Linux. Then again, > the code I''m running doesn''t do anything low-level (other than normal > tty ioctls).Then you did only run simple programs that call syscalls and simple ioctl''s like TTY related ioctls nut not cdrecord.> >> For the most part, 32bit binaries will run faster (my benchmarks on both > >> Linux and Solaris generally get a x2 speed improvement) than their 64bit > > > > My tests show a typical 30% speed improvement for 64 bit code caused by the fact > > that the x64 architecture offers twice as many registers than x86. > > Funnily enough I''ve never, ever had code run faster when compiled 64 > bit, either with gcc or the Sun Studio compilers on x86_64 machines.The right programs will run twice as fast (e.g. programs that examine data using (long long *). J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily