Hello, the ZFS best practices guide at http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide tells:> * Run ZFS on a system that runs a 64-bit kernelbesides performance aspects, what`s the con`s of running zfs on 32 bit ? -- This message posted from opensolaris.org
> besides performance aspects, what`s the con`s of > running zfs on 32 bit ?The default 32 bit kernel can cache a limited amount of data (< 512MB) - unless you lower the "kernelbase" parameter. In the end the small cache size on 32 bit explains the inferior performance compared to the 64 bit kernel. -- This message posted from opensolaris.org
J?rgen Keil wrote:>> besides performance aspects, what`s the con`s of >> running zfs on 32 bit ? >> > > The default 32 bit kernel can cache a limited amount of data > (< 512MB) - unless you lower the "kernelbase" parameter. > In the end the small cache size on 32 bit explains the inferior > performance compared to the 64 bit kernel. >It''s been a long time, but I seem to recall that the ZFS internals were written using values (ints, longs, etc) as found on 64-bit architectures, and that there was the possibility that many of them wouldn''t operate properly in a 32-bit environment (i.e. size assumption mismatches on values that might silently drop/truncate or screw up calculations). I don''t know if that''s still correct (or if I''m getting it completely wrong), but the word was (2 years ago), that 32-bit ZFS might not just have performance problems, but might possibly be silently screwing you. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800)
There is a 32-bit and 64-bit version of the file system module available on x86. Given the quality of the development team, I''d be *very* surprised if such issues as suggested in your message exist. Jurgen''s comment highlights the major issue - the lack of space to cache data when in 32-bit mode. Jim Litchfield ------------------- Erik Trimble wrote:> J?rgen Keil wrote: >>> besides performance aspects, what`s the con`s of >>> running zfs on 32 bit ? >>> >> >> The default 32 bit kernel can cache a limited amount of data >> (< 512MB) - unless you lower the "kernelbase" parameter. >> In the end the small cache size on 32 bit explains the inferior >> performance compared to the 64 bit kernel. >> > It''s been a long time, but I seem to recall that the ZFS internals > were written using values (ints, longs, etc) as found on 64-bit > architectures, and that there was the possibility that many of them > wouldn''t operate properly in a 32-bit environment (i.e. size > assumption mismatches on values that might silently drop/truncate or > screw up calculations). I don''t know if that''s still correct (or if > I''m getting it completely wrong), but the word was (2 years ago), that > 32-bit ZFS might not just have performance problems, but might > possibly be silently screwing you. >
This sounds like FUD. There''s a comprehensive test suite, and it apparently passes. -- This message posted from opensolaris.org
Daniel Carosone wrote:> This sounds like FUD. > > There''s a comprehensive test suite, and it apparently passes.It''s not exactly FUD. If you search the list archives, you''ll find messages about multiple bugs in the 32-bit code. I strongly suspect that these have been fixed in the interim, but it _used_ to be a real problem. -- Carson
so, besides performance there COULD be some stability issues. thanks for the answers - i think i`ll stay with 32bit, even if there COULD be issues. (i`m happy to report and help fixing those) i don`t have free 64bit hardware around for building storage boxes. -- This message posted from opensolaris.org
Ive asked the same question about 32bit. I created a thread and asked. It were something like "does 32bit ZFS fragments RAM?" or something similar. As I remember it, 32 bit had some issues. Mostly due to RAM fragmentation or something similar. The result was that you had to restart your server after a while. But I shuts down my desktopPC every night so I never had any issues. -- This message posted from opensolaris.org
I had a 32 bit zfs server up for months with no such issue Performance is not great but it''s no buggier than anything else. War stories from the initial zfs drops notwithstanding khbkhb at gmail.com | keith.bierman at quantum.com Sent from my iPod On Jun 15, 2009, at 3:59 PM, Orvar Korvar <no-reply at opensolaris.org> wrote:> Ive asked the same question about 32bit. I created a thread and > asked. It were something like "does 32bit ZFS fragments RAM?" or > something similar. As I remember it, 32 bit had some issues. Mostly > due to RAM fragmentation or something similar. The result was that > you had to restart your server after a while. But I shuts down my > desktopPC every night so I never had any issues. > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
one of my disaster recovery servers has been running on 32bit hardware (ancient northwood chip) for about a year. the only problems i''ve run into are: slow (duh) and will not take disks that are bigger than 1tb. that is kind of a bummer and means i''ll have to switch to a 64bit base soon. everything else has been fine. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090615/ae5c521a/attachment.html>
>Ive asked the same question about 32bit. I created a thread and asked. >It were something like "does 32bit ZFS fragments RAM?" or something similar.>As I remember it, 32 bit had some issues. Mostly due to RAM fragmentation or >something similar. The result was that you had to restart your server >after a while. But I shuts down my desktopPC every night so I never had >any issues.It''s the problem of virtual memory and that only hits when you have more memory than the amount of kernel virtual memory. You can lower kernelbase that helps a bit, but not when you have multiple GBs installed. Caasper
>the only problems i''ve run into are: slow (duh) and will not >take disks that are bigger than 1tbdo you think that 1tb limit is due to 32bit solaris ? -- This message posted from opensolaris.org
yeah, i get a nice clean zfs error message about disk size limits when i try to add the disk. On Tue, Jun 16, 2009 at 4:26 PM, roland<no-reply at opensolaris.org> wrote:>>the only problems i''ve run into are: slow (duh) and will not >>take disks that are bigger than 1tb > > do you think that 1tb limit is due to 32bit solaris ? > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
so, we have a 128bit fs, but only support for 1tb on 32bit? i`d call that a bug, isn`t it ? is there a bugid for this? ;) -- This message posted from opensolaris.org
On 06/16/09 02:39 PM, roland wrote:> so, we have a 128bit fs, but only support for 1tb on 32bit? > > i`d call that a bug, isn`t it ? is there a bugid for this? ;) >Well, opinion is welcome. I''d call it an RFE. With 64 bit versions of the CPU chips so inexpensive these days, how much money do you want me to invest in moving modern features and support to old versions of the OS? I mean, Microsoft could, on a technical level, backport all new features from Vista and Windows Seven to Windows 95. But if they did that, their current offering would lag, since all the engineers would be working on the older stuff. Heck, you can buy a 64 bit CPU motherboard very very cheap. The staff that we do have are working on modern features for the 64bit version, rather than spending all their time "in the rear-view mirror". Live life forward. Upgrade. Changing all the data structures in the 32 bit OS to handle super larger disks, is, well, sorta like trying to get a Pentium II to handle HD Video. I''m sure, with enough time and money, you might find a way. But is it worth it? Or is it cheaper to buy a new pump? Neal
On Tue, 16 Jun 2009, roland wrote:> so, we have a 128bit fs, but only support for 1tb on 32bit? > > i`d call that a bug, isn`t it ? is there a bugid for this? ;)I''d say the bug in this instance is using a 32-bit platform in 2009! :-) -- Rich Teer, SCSA, SCNA, SCSECA URLs: http://www.rite-group.com/rich http://www.linkedin.com/in/richteer
yeah i pretty much agree with you on this. the fact that no one has brought this up before is a pretty good indication of the demand. there are about 1000 things i''d rather see fixed/improved than max disk size on a 32bit platform. On Tue, Jun 16, 2009 at 5:55 PM, Neal Pollack<Neal.Pollack at sun.com> wrote:> On 06/16/09 02:39 PM, roland wrote: >> >> so, we have a 128bit fs, but only support for 1tb on 32bit? >> >> i`d call that a bug, isn`t it ? ?is there a bugid for this? ;) >> > > Well, opinion is welcome. > I''d call it an RFE. > > With 64 bit versions of the CPU chips so inexpensive these days, > how much money do you want me to invest in moving modern features > and support to old versions of the OS? > > I mean, Microsoft could, on a technical level, backport all new features > from > Vista and Windows Seven to Windows 95. ?But if they did that, their current > offering > would lag, since all the engineers would be working on the older stuff. > > Heck, you can buy a 64 bit CPU motherboard very very cheap. ?The staff that > we do have > are working on modern features for the 64bit version, rather than spending > all their time > "in the rear-view mirror". ? Live life forward. ?Upgrade. > Changing all the data structures in the 32 bit OS to handle super larger > disks, is, well, sorta > like trying to get a Pentium II to handle HD Video. ?I''m sure, with enough > time and money, > you might find a way. ?But is it worth it? ?Or is it cheaper to buy a new > pump? > > Neal > > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
On Tue, Jun 16, 2009 at 03:16:09PM -0700, milosz wrote:> yeah i pretty much agree with you on this. the fact that no one has > brought this up before is a pretty good indication of the demand. > there are about 1000 things i''d rather see fixed/improved than max > disk size on a 32bit platform.I''d say a lot of folks out there have plenty of enterprise-class 32-bit hardware still in production in their datacenters. I know I do. Several IBM BladeCenters with 32-bit blades and attached storage... It would be "nice" to be able to do ZFS on these platforms (>1TB that is), but I understand if it''s not a priority. But there''s certainly a lot of life left in 32-bit hardware, and not all of it is cheap to replace. Ray> > On Tue, Jun 16, 2009 at 5:55 PM, Neal Pollack<Neal.Pollack at sun.com> wrote: > > On 06/16/09 02:39 PM, roland wrote: > >> > >> so, we have a 128bit fs, but only support for 1tb on 32bit? > >> > >> i`d call that a bug, isn`t it ? ?is there a bugid for this? ;) > >> > > > > Well, opinion is welcome. > > I''d call it an RFE. > > > > With 64 bit versions of the CPU chips so inexpensive these days, > > how much money do you want me to invest in moving modern features > > and support to old versions of the OS? > > > > I mean, Microsoft could, on a technical level, backport all new features > > from > > Vista and Windows Seven to Windows 95. ?But if they did that, their current > > offering > > would lag, since all the engineers would be working on the older stuff. > > > > Heck, you can buy a 64 bit CPU motherboard very very cheap. ?The staff that > > we do have > > are working on modern features for the 64bit version, rather than spending > > all their time > > "in the rear-view mirror". ? Live life forward. ?Upgrade. > > Changing all the data structures in the 32 bit OS to handle super larger > > disks, is, well, sorta > > like trying to get a Pentium II to handle HD Video. ?I''m sure, with enough > > time and money, > > you might find a way. ?But is it worth it? ?Or is it cheaper to buy a new > > pump? > > > > Neal
On 16-Jun-09, at 6:22 PM, Ray Van Dolson wrote:> On Tue, Jun 16, 2009 at 03:16:09PM -0700, milosz wrote: >> yeah i pretty much agree with you on this. the fact that no one has >> brought this up before is a pretty good indication of the demand. >> there are about 1000 things i''d rather see fixed/improved than max >> disk size on a 32bit platform. > > I''d say a lot of folks out there have plenty of enterprise-class 32- > bit > hardware still in production in their datacenters. I know I do. > Several IBM BladeCenters with 32-bit blades and attached storage... > > It would be "nice" to be able to do ZFS on these platforms (>1TB that > is), but I understand if it''s not a priority. But there''s certainly a > lot of life left in 32-bit hardware, and not all of it is cheap to > replace.I bet 1+ TB drives in the right format (e.g. SCSI) aren''t exactly cheap or even available... Let''s be reminded that this is about maximum size of a single drive (or slice?) not dataset or pool.> milosz wrote,> the fact that no one has > brought this up before is a pretty good indication of the demand. > there are about 1000 things i''d rather see fixed/improved than max > disk size on a 32bit platform. >+1 --Toby> > Ray > >> >> On Tue, Jun 16, 2009 at 5:55 PM, Neal >> Pollack<Neal.Pollack at sun.com> wrote: >>> On 06/16/09 02:39 PM, roland wrote: >>>> >>>> so, we have a 128bit fs, but only support for 1tb on 32bit? >>>> >>>> i`d call that a bug, isn`t it ? is there a bugid for this? ;) >>>> >>> >>> Well, opinion is welcome. >>> I''d call it an RFE. >>> >>> With 64 bit versions of the CPU chips so inexpensive these days, >>> how much money do you want me to invest in moving modern features >>> and support to old versions of the OS? >>> >>> I mean, Microsoft could, on a technical level, backport all new >>> features >>> from >>> Vista and Windows Seven to Windows 95. But if they did that, >>> their current >>> offering >>> would lag, since all the engineers would be working on the older >>> stuff. >>> >>> Heck, you can buy a 64 bit CPU motherboard very very cheap. The >>> staff that >>> we do have >>> are working on modern features for the 64bit version, rather than >>> spending >>> all their time >>> "in the rear-view mirror". Live life forward. Upgrade. >>> Changing all the data structures in the 32 bit OS to handle super >>> larger >>> disks, is, well, sorta >>> like trying to get a Pentium II to handle HD Video. I''m sure, >>> with enough >>> time and money, >>> you might find a way. But is it worth it? Or is it cheaper to >>> buy a new >>> pump? >>> >>> Neal > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
On Tue, Jun 16, 2009 at 04:25:58PM -0700, Toby Thain wrote:> > On 16-Jun-09, at 6:22 PM, Ray Van Dolson wrote: > > > On Tue, Jun 16, 2009 at 03:16:09PM -0700, milosz wrote: > >> yeah i pretty much agree with you on this. the fact that no one has > >> brought this up before is a pretty good indication of the demand. > >> there are about 1000 things i''d rather see fixed/improved than max > >> disk size on a 32bit platform. > > > > I''d say a lot of folks out there have plenty of enterprise-class 32- > > bit > > hardware still in production in their datacenters. I know I do. > > Several IBM BladeCenters with 32-bit blades and attached storage... > > > > It would be "nice" to be able to do ZFS on these platforms (>1TB that > > is), but I understand if it''s not a priority. But there''s certainly a > > lot of life left in 32-bit hardware, and not all of it is cheap to > > replace. > > > I bet 1+ TB drives in the right format (e.g. SCSI) aren''t exactly > cheap or even available... > > Let''s be reminded that this is about maximum size of a single drive > (or slice?) not dataset or pool. >Ah, missed that part. That''s what I get for jumping in in the middle of a thread. :-) Ray
On 06/16/09 03:22 PM, Ray Van Dolson wrote:> On Tue, Jun 16, 2009 at 03:16:09PM -0700, milosz wrote: > >> yeah i pretty much agree with you on this. the fact that no one has >> brought this up before is a pretty good indication of the demand. >> there are about 1000 things i''d rather see fixed/improved than max >> disk size on a 32bit platform. >> > > I''d say a lot of folks out there have plenty of enterprise-class 32-bit > hardware still in production in their datacenters. I know I do. > Several IBM BladeCenters with 32-bit blades and attached storage... > > It would be "nice" to be able to do ZFS on these platforms (>1TB that > is), but I understand if it''s not a priority. But there''s certainly a > lot of life left in 32-bit hardware, and not all of it is cheap to > replace. >Not sure I understand all this concern. 32 bit can use 1.0 TB disks as data drives. ZFS can use more than 1 disk. So if you hook up 48 of the 1.0 TB disks using ZFS on a 32 bit system, where is the problem? If someone running a 32bit system is angry because they can''t waste a 1.5 TB seagate disk as the boot drive, then I''ll admit I don''t understand something in their requirements. What is the specific complaint please? Neal> Ray > > >> On Tue, Jun 16, 2009 at 5:55 PM, Neal Pollack<Neal.Pollack at sun.com> wrote: >> >>> On 06/16/09 02:39 PM, roland wrote: >>> >>>> so, we have a 128bit fs, but only support for 1tb on 32bit? >>>> >>>> i`d call that a bug, isn`t it ? is there a bugid for this? ;) >>>> >>>> >>> Well, opinion is welcome. >>> I''d call it an RFE. >>> >>> With 64 bit versions of the CPU chips so inexpensive these days, >>> how much money do you want me to invest in moving modern features >>> and support to old versions of the OS? >>> >>> I mean, Microsoft could, on a technical level, backport all new features >>> from >>> Vista and Windows Seven to Windows 95. But if they did that, their current >>> offering >>> would lag, since all the engineers would be working on the older stuff. >>> >>> Heck, you can buy a 64 bit CPU motherboard very very cheap. The staff that >>> we do have >>> are working on modern features for the 64bit version, rather than spending >>> all their time >>> "in the rear-view mirror". Live life forward. Upgrade. >>> Changing all the data structures in the 32 bit OS to handle super larger >>> disks, is, well, sorta >>> like trying to get a Pentium II to handle HD Video. I''m sure, with enough >>> time and money, >>> you might find a way. But is it worth it? Or is it cheaper to buy a new >>> pump? >>> >>> Neal >>> > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090616/047fa555/attachment.html>
roland wrote:> so, we have a 128bit fs, but only support for 1tb on 32bit? > > i`d call that a bug, isn`t it ? is there a bugid for this? ;) >Not a ZFS bug. IIRC, the story goes something like this: a SMI label only works to 1 TByte, so to use > 1 TByte, you need an EFI label. For older x86 systems -- those which are 32-bit -- you probably have a BIOS which does not handle EFI labels. This will become increasingly irritating since 2 TByte disks are now hitting the store shelves, but it doesn''t belong in a ZFS category. -- richard
> On Tue, 16 Jun 2009, roland wrote: > >> so, we have a 128bit fs, but only support for 1tb on 32bit? >> >> i`d call that a bug, isn`t it ? is there a bugid for this? ;) > > I''d say the bug in this instance is using a 32-bit platform in 2009! :-)Rich, a lot of embedded industrial solutions are 32-bit and very up to date in terms of features. Thus : $ uname -a SunOS aequitas 5.11 snv_115 i86pc i386 i86pc $ isainfo -v 32-bit i386 applications ahf sse2 sse fxsr mmx cmov sep cx8 tsc fpu $ isalist -v i486 i386 i86 $ psrinfo -pv The physical processor has 1 virtual processor (0) x86 (CentaurHauls 6A9 family 6 model 10 step 9 clock 1200 MHz) VIA Esther processor 1200MHz Also, some of the very very small little PC units out there, those things called eePC ( or whatever ) are probably 32-bit only. Dennis
On Wed, Jun 17, 2009 at 8:32 AM, Neal Pollack<Neal.Pollack at sun.com> wrote:> Not sure I understand all this concern.? 32 bit can use 1.0 TB disks as data > drives. ZFS can use more than 1 disk.? So if you hook up 48 of the 1.0 TB disks > using ZFS on a 32 bit system, where is the problem?+1. Even if someone needs larger disk space, then it means system going to withstand bigger/larger stream, thus they''d better probably switch to 64bit and think about better hardware anyway. Just try to imagine $100 price Atom-based Eee PC, running 1.6GHz clockspeed with 1GB RAM, yet with 48 USB external hard drives hooked... :-)> If someone running a 32bit system is angry because they can''t waste a 1.5 TB > seagate disk as the boot drive, then I''ll admit I don''t understand something > in their requirements.? What is the specific complaint please?LOL The only specific complaint must be related to preventing people from defective Slashdot brainwash that sways them towards amateurish presumptions... -- Kind regards, bm
> Not a ZFS bug. [SMI vs EFI labels vs BIOS booting]and so also only a problem for disks that are members of the root pool. ie, I can have >1Tb disks as part of a non-bootable data pool, with EFI labels, on a 32-bit machine? -- This message posted from opensolaris.org
Dennis is correct in that there are significant areas where 32-bit systems will remain the norm for some time to come. And choosing a 32-bit system in these areas is completely correct. That said, I think the issue is that (unlike Linux), Solaris is NOT a super-duper-plays-in-all-possible-spaces OS. It''s a specialized OS, intended for specific market segments. Embedded is not really one of them; nor are ultra-low-end netbooks or appliances such as set-top boxes (though, with the increasing functionality of DVRs, that may change soon). Because of that, I don''t think it''s appropriate for ZFS/Solaris folks to stray too far from their target audience, given the limited resources. Now, should someone else outside Sun pick up more development, then Solaris may move into different markets than it is now. Solaris/ZFS really is intended for a 64-bit machine which is not RAM or CPU constrained. And, frankly, making it work on things such as removable USB thumb drives, legacy 32-bit systems, or with consumer-grade widgets is (in my opinion) pure sugar-coating to the central goal of a solid, high-performance, flexible, enterprise-class OS and filesystem (which, hopefully, will someday include being a true cluster filesystem :-) Note that I don''t speak for Sun on this, it''s just my personal reading of the tea leaves. -Erik Dennis Clarke wrote:>> On Tue, 16 Jun 2009, roland wrote: >> >> >>> so, we have a 128bit fs, but only support for 1tb on 32bit? >>> >>> i`d call that a bug, isn`t it ? is there a bugid for this? ;) >>> >> I''d say the bug in this instance is using a 32-bit platform in 2009! :-) >> > > Rich, a lot of embedded industrial solutions are 32-bit and very up to > date in terms of features. > > Thus : > > $ uname -a > SunOS aequitas 5.11 snv_115 i86pc i386 i86pc > $ isainfo -v > 32-bit i386 applications > ahf sse2 sse fxsr mmx cmov sep cx8 tsc fpu > $ isalist -v > i486 i386 i86 > > $ psrinfo -pv > The physical processor has 1 virtual processor (0) > x86 (CentaurHauls 6A9 family 6 model 10 step 9 clock 1200 MHz) > VIA Esther processor 1200MHz > > Also, some of the very very small little PC units out there, those things > called eePC ( or whatever ) are probably 32-bit only. > > Dennis > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800)
>roland wrote: >> so, we have a 128bit fs, but only support for 1tb on 32bit? >> >> i`d call that a bug, isn`t it ? is there a bugid for this? ;) >> > >Not a ZFS bug. IIRC, the story goes something like this: a SMI >label only works to 1 TByte, so to use > 1 TByte, you need an >EFI label. For older x86 systems -- those which are 32-bit -- you >probably have a BIOS which does not handle EFI labels. This >will become increasingly irritating since 2 TByte disks are now >hitting the store shelves, but it doesn''t belong in a ZFS category.Slightly more complicated than that. 32 bit Solaris can use at most 2^31 as disk address; a disk block is 512bytes, so in total it can address 2^40 bytes. A SMI label found in Solaris 10 (update 8?) and OpenSolaris has been enhanced and can address 2TB but only on a 64 bit system. I''m not sure which system uses EFI. Casper
So you better post the nice and clean zfs error message that you got on your screen, instead of posting about things that you might ignore. To give the correct information, leads to your correct solution. In your case possible, the patchlevel, or /format -e/ issue. Think about it ! milosz schrieb:> yeah, i get a nice clean zfs error message about disk size limits when > i try to add the disk. > > On Tue, Jun 16, 2009 at 4:26 PM, roland<no-reply at opensolaris.org> wrote: > >>> the only problems i''ve run into are: slow (duh) and will not >>> take disks that are bigger than 1tb >>> >> do you think that 1tb limit is due to 32bit solaris ? >> -- >> This message posted from opensolaris.org >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> >> > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-- Moutacim LACHHAB Service Engineer Software Technical Services Center Sun Microsystems Inc. Email moutacim.lachhab at Sun.COM <mailto:moutacim.lachhab at sun.com> +33(0)134030594 x31457 For knowledge and support: http://sunsolve.sun.com
>$ psrinfo -pv >The physical processor has 1 virtual processor (0) > x86 (CentaurHauls 6A9 family 6 model 10 step 9 clock 1200 MHz) > VIA Esther processor 1200MHz > >Also, some of the very very small little PC units out there, those things >called eePC ( or whatever ) are probably 32-bit only.It''s true for most of the Intel Atom family (Zxxx and Nxxx but not the 230 and 330 as those are 64 bit) Those are new systems. Casper
>> Not a ZFS bug. [SMI vs EFI labels vs BIOS booting] > >and so also only a problem for disks that are members of the root pool. > >ie, I can have >1Tb disks as part of a non-bootable data pool, with EFI labels, on a 32-bit machine?No; the daddr_t is only 32 bits. Casper
Casper.Dik at Sun.COM wrote:> > It''s true for most of the Intel Atom family (Zxxx and Nxxx but not the > 230 and 330 as those are 64 bit) Those are new systems. > > Casper > > _______________________________________________I''ve actually just started to build my home raid using the Atom 330 (D945GCLF2): Status of virtual processor 0 as of: 06/17/2009 16:25:55 on-line since 09/17/2008 14:32:04. The i386 processor operates at 1600 MHz, and has an i387 compatible floating point processor. Status of virtual processor 1 as of: 06/17/2009 16:25:55 on-line since 09/17/2008 14:32:24. The i386 processor operates at 1600 MHz, and has an i387 compatible floating point processor. Status of virtual processor 2 as of: 06/17/2009 16:25:55 on-line since 09/17/2008 14:32:24. The i386 processor operates at 1600 MHz, and has an i387 compatible floating point processor. Status of virtual processor 3 as of: 06/17/2009 16:25:55 on-line since 09/17/2008 14:32:26. The i386 processor operates at 1600 MHz, and has an i387 compatible floating point processor. and booted 64 bit just fine. (I thought uname -a showed that, but apparently it does not). The only annoyance is that the onboard ICH7 is the $27c0, and not the $27c1 (with ahci mode for hot-swapping). But I always were planning on adding a SATA PCI card since I need more than 2 HDDs. But to stay on-topic, it sounds like Richard Elling summed it up nicely, which is something Richard is really good at. Lund -- Jorgen Lundman | <lundman at lundman.net> Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work) Shibuya-ku, Tokyo | +81 (0)90-5578-8500 (cell) Japan | +81 (0)3 -3375-1767 (home)
> Not a ZFS bug. IIRC, the story goes something like this: a SMI > label only works to 1 TByte, so to use > 1 TByte, you need an > EFI label. For older x86 systems -- those which are 32-bit -- you > probably have a BIOS which does not handle EFI labels. This > will become increasingly irritating since 2 TByte disks are now > hitting the store shelves, but it doesn''t belong in a ZFS category.Hasn''t the 1TB limit for SMI labels been fixed (= limit raised to 2TB) by "PSARC/2008/336 Extended VTOC" ? http://www.opensolaris.org/os/community/on/flag-days/pages/2008091102/ But there still is a 1TB limit for 32-bit kernel, the PSARC case includes this: The following functional limitations are applicable: * 32-bit kernel will not support disks > 1 TB. ... Btw. on older Solaris releases the install media always booted into a 32-bit kernel, even on systems that are capable to run the 64-bit kernel. Seems to have been changed with the latest opensolaris releases and that PSARC case, so that 64-bit systems can install to a disk > 1TB. -- This message posted from opensolaris.org
Casper.Dik at Sun.COM wrote:> >ie, I can have >1Tb disks as part of a non-bootable data pool, with EFI labels, on a 32-bit machine? > > No; the daddr_t is only 32 bits.This looks like a left over problem problem from former times when UFS was limited to 1 TB anyway. J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
>> Not a ZFS bug. IIRC, the story goes something like this: a SMI >> label only works to 1 TByte, so to use > 1 TByte, you need an >> EFI label. For older x86 systems -- those which are 32-bit -- you >> probably have a BIOS which does not handle EFI labels. This >> will become increasingly irritating since 2 TByte disks are now >> hitting the store shelves, but it doesn''t belong in a ZFS category. > >Hasn''t the 1TB limit for SMI labels been fixed >(= limit raised to 2TB) by "PSARC/2008/336 Extended VTOC" ? >http://www.opensolaris.org/os/community/on/flag-days/pages/2008091102/ > >But there still is a 1TB limit for 32-bit kernel, the PSARC case includes this: > > The following functional limitations are applicable: > * 32-bit kernel will not support disks > 1 TB. > ... > > >Btw. on older Solaris releases the install media always >booted into a 32-bit kernel, even on systems that are >capable to run the 64-bit kernel. Seems to have >been changed with the latest opensolaris releases and >that PSARC case, so that 64-bit systems can install to >a disk > 1TB.That was changed many builds ago. Casper
Erik Trimble wrote:> Dennis is correct in that there are significant areas where 32-bit > systems will remain the norm for some time to come. And choosing a > 32-bit system in these areas is completely correct. > > That said, I think the issue is that (unlike Linux), Solaris is NOT a > super-duper-plays-in-all-possible-spaces OS. It''s a specialized OS, > intended for specific market segments. Embedded is not really one of > them; nor are ultra-low-end netbooks or appliances such as set-top boxes > (though, with the increasing functionality of DVRs, that may change soon).http://opensolaris.org/os/project/osarm/ -- Darren J Moffat
thank you, caspar. to sum up here (seems to have been a lot of confusion in this thread): the efi vs. smi thing that richard and a few other people have talked about is not the issue at the heart of this. this:> 32 bit Solaris can use at most 2^31 as disk address; a disk block is > 512bytes, so in total it can address 2^40 bytes. > > A SMI label found in Solaris 10 (update 8?) and OpenSolaris has been enhanced > and can address 2TB but only on a 64 bit system.is what the problem is. so 32-bit zfs cannot use disks larger than 1(.09951)tb regardless of whether it''s for the root pool or not. let me repeat that i do not consider this a "bug" and do not want to see it "fixed".
> > 32 bit Solaris can use at most 2^31 as disk address; a disk block is > > 512bytes, so in total it can address 2^40 bytes. > > > > A SMI label found in Solaris 10 (update 8?) and OpenSolaris has been enhanced > > and can address 2TB but only on a 64 bit system. > > is what the problem is. so 32-bit zfs cannot use disks larger than > 1(.09951)tb regardless of whether it''s for the root pool or not.I think this isn''t a problem with the 32-bit zfs module, but with all of the 32-bit Solaris kernel. The daddr_t type is used in a *lot* of places, and is defined as a signed 32-bit integer ("long") in the 32-bit kernel. It seems that there already are 64-bit disk address types defined, diskaddr_t and lldaddr_t (that could be used in the 32-bit kernel, too), but a lot of the existing kernel code doesn''t use them. And redefining the existing daddr_t type to 64-bit "long long" for the 32-bit kernel won''t work, because it would break binary compatibility. -- This message posted from opensolaris.org
>Solaris is NOT a super-duper-plays-in-all-possible-spaces OS.yes, i know - but it`s disappointing that not even 32bit and 64bit x86 hardware is handled the same. 1TB limit on 32bit, less stable on 32bit. sorry, but if you are used to linux, solaris is really weird. issue here, limitation there.... doh! -- This message posted from opensolaris.org
roland wrote:>> Solaris is NOT a super-duper-plays-in-all-possible-spaces OS. >> > > yes, i know - but it`s disappointing that not even 32bit and 64bit x86 hardware is handled the same. > 1TB limit on 32bit, less stable on 32bit. > > sorry, but if you are used to linux, solaris is really weird. > > issue here, limitation there.... > > doh! >Which is true, but so is it for Linux - you''re just familiar with Linux''s limitations, so they don''t seem like "limitations" anymore. E.g.: linux handles 32-bit programs under 64-bit Linux is a much less clean way than Solaris does. Also, 2.4 (x86) kernels have a 1TB block device size limit, while 2.6 Linux x86 is limited to 16TB block devices. Solaris handles many more (i.e. maximum number of) CPUs than even the latest Linux. It''s a transitional adjustment - you don''t expect a Semi to drive the same is Hummer to drive the same as a Porche, do you? Each OS has its strengths and weaknesses; pick your poison. It''s actually NOT a good idea for all OSes to have the same feature set. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800)
>>>>> "djm" == Darren J Moffat <darrenm at opensolaris.org> writes: >>>>> "cd" == Casper Dik <Casper.Dik at Sun.COM> writes:djm> http://opensolaris.org/os/project/osarm/ yeah. many of those ARM systems will be low-power builtin-crypto-accel builtin-gigabit-MAC based on Orion and similar, NAS (NSLU2-ish) things begging for ZFS. cd> It''s true for most of the Intel Atom family (Zxxx and Nxxx but cd> not the 230 and 330 as those are 64 bit) Those are new cd> systems. the 64-bit atom are desktop, and the 32-bit are laptop. They are both current chips right now---the 64-bit are not newer than 32-bit. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090617/38cd527d/attachment.bin>
>yeah. many of those ARM systems will be low-power >builtin-crypto-accel builtin-gigabit-MAC based on Orion and similar, >NAS (NSLU2-ish) things begging for ZFS.So what''s the boot environment they use?> cd> It''s true for most of the Intel Atom family (Zxxx and Nxxx but > cd> not the 230 and 330 as those are 64 bit) Those are new > cd> systems. > >the 64-bit atom are desktop, and the 32-bit are laptop. They are both >current chips right now---the 64-bit are not newer than 32-bit.I know; I''m not sure about the recently Pineview boxes, though. Casper
>>>>> "cd" == Casper Dik <Casper.Dik at Sun.COM> writes:>> yeah. many of those ARM systems will be low-power >> builtin-crypto-accel builtin-gigabit-MAC based on Orion and >> similar, NAS (NSLU2-ish) things begging for ZFS. cd> So what''s the boot environment they use? i think it is called U-Boot: http://forum.openwrt.org/viewtopic.php?pid=60387 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090618/552a8ba7/attachment.bin>
On Thu, Jun 18, 2009 at 4:28 AM, Miles Nordin<carton at ivy.net> wrote:> ? djm> http://opensolaris.org/os/project/osarm/ > > yeah. ?many of those ARM systems will be low-power > builtin-crypto-accel builtin-gigabit-MAC based on Orion and similar, > NAS (NSLU2-ish) things begging for ZFS.Are they feasible targets for zfs? The N610N that I have (BCM3302, 300MHz, 64MB) isn''t even powerful enough to saturate either the gigabit wired or 802.11n wireless. It only goes about 25Mbps. Last time I test on EEPC 2G''s Celeron, zfs is slow to the point of unusable. Will it be usable enough on most ARMs? -- Fajar
Fajar A. Nugraha wrote:> On Thu, Jun 18, 2009 at 4:28 AM, Miles Nordin<carton at ivy.net> wrote: > >> djm> http://opensolaris.org/os/project/osarm/ >> >> yeah. many of those ARM systems will be low-power >> builtin-crypto-accel builtin-gigabit-MAC based on Orion and similar, >> NAS (NSLU2-ish) things begging for ZFS. >> > > Are they feasible targets for zfs? > > The N610N that I have (BCM3302, 300MHz, 64MB) isn''t even powerful > enough to saturate either the gigabit wired or 802.11n wireless. It > only goes about 25Mbps. > > Last time I test on EEPC 2G''s Celeron, zfs is slow to the point of > unusable. Will it be usable enough on most ARMs? > >Well, given that ARM processors use a completely different ISA (ie. they''re not x86-compatible), OpenSolaris won''t run on them currently. If you''d like to do the port.... <wink> I can''t say as to the entire Atom line of stuff, but I''ve found the Atoms are OK for desktop use, and not anywhere powerful enough for even a basic NAS server. The demands of wire-speed Gigabit, ZFS, and encryption/compression are hard on the little Atom guys. Plus, it seems to be hard to find an Atom motherboard which supports more than 2GB of RAM, which is a serious problem. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA
On Fri, Jun 19, 2009 at 11:16 AM, Erik Trimble<Erik.Trimble at sun.com> wrote:> I can''t say as to the entire Atom line of stuff, but I''ve found the Atoms > are OK for desktop use, and not anywhere powerful enough for even a basic > NAS server. ?The demands of wire-speed Gigabit, ZFS, and > encryption/compression are hard on the little Atom guys.+1. I wanted to skip it, but will reply. I have two Asus EeePC Box 202 / 2GB. These are running numerous zones (snv_111b) for me with various services on them and still are very usable and fast enough. Additionally, I overclocked each up to 1.75GHz, did some corrections to Solaris''s TCP/IP stack, removed some unnecessary services and they are just fine.> Plus, it seems to be hard to find an Atom motherboard which supports > more than 2GB of RAM, which is a serious problem.Well, let''s don''t forget that Atom is also smallest low-power processor and is designed for cheap and small nettops/netbooks that are don''t need 4GB RAM ever. Despite of that: http://www.mini-itx.com/store/?c=53 -- Kind regards, BM Things, that are stupid at the beginning, rarely ends up wisely.
Erik Trimble wrote:> Fajar A. Nugraha wrote: >> Are they feasible targets for zfs? >> >> The N610N that I have (BCM3302, 300MHz, 64MB) isn''t even powerful >> enough to saturate either the gigabit wired or 802.11n wireless. It >> only goes about 25Mbps. >> >> Last time I test on EEPC 2G''s Celeron, zfs is slow to the point of >> unusable. Will it be usable enough on most ARMs? >> >> > Well, given that ARM processors use a completely different ISA (ie. > they''re not x86-compatible), OpenSolaris won''t run on them currently. > > If you''d like to do the port.... > > <wink> > > I can''t say as to the entire Atom line of stuff, but I''ve found the > Atoms are OK for desktop use, and not anywhere powerful enough for > even a basic NAS server. The demands of wire-speed Gigabit, ZFS, and > encryption/compression are hard on the little Atom guys. Plus, it > seems to be hard to find an Atom motherboard which supports more than > 2GB of RAM, which is a serious problem. >Open mouth, insert foot. The ARM port is now functional (and available). I would assume (though I can''t verify) that ZFS support is part of the port. There are a wide variety of ARM chips, in all sorts of stuff. Given the performance characteristics of some of the stuff I''ve been playing with over the last decade (and a pre-look at an ARM-based netbook), I''d have to say that any currently-available single-chip ARM-based system isn''t going to be good to run OpenSolaris/ZFS on. That said, I can certainly see some really, really good uses for ARM-based microcontrollers as the guts of an HBA. They''re likely good enough to do something like a tiny computer-on-a-board setup. Think something like a Sun 7110-style system shrunk down to a PCI-E controller - you have a simple host-based control program, hook a disk (or storage system) to the ARM HBA, and you could have a nice little embedded ZFS system. Either that, or if someone would figure out a way to have multiple-chip ARM implementations (where they could spread out the load efficiently). -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA
>>>>> "fan" == Fajar A Nugraha <fajar at fajar.net> writes: >>>>> "et" == Erik Trimble <Erik.Trimble at Sun.COM> writes:fan> The N610N that I have (BCM3302, 300MHz, 64MB) isn''t even fan> powerful enough to saturate either the gigabit wired I can''t find that device. Did you misspell it or something? BCM probably means Broadcom, and Broadcom is probably MIPS---it''s TI (omap) and Marvell (orion) that are selling arm. Anyway I don''t think saturating gigabit is the minimum acceptable performance considering the external storage people actually use right now. That said, ARM is interesting because the chips just recently got a lot faster at the same power/price point, like >1GHz. There are a whole batch of new netbooks (i''ve been calling them HypeBooks because they will probably fail) based on these new fast omap chips. Also the next Orion stepping is supposed to have crypto accel which makes a big difference in AES per watt. I will be trying ZFS crypto once it''s released, and my understanding from Linux dmcrypt users is, on ordinary CPU''s it''s a serious bottleneck/powerhog. Right now it makes more sense to me to do the crypto on Linux iSCSI targets, where I can do it on hardware-accel Via C7 (also 32-bit), and put several C7 chips into one zpool since they are device-granularity. The 64MB may be a show-stopper for ZFS on the whole ARM platform though. I brought it up because arm is a 32-bit platform. et> a Sun 7110-style system shrunk down to a PCI-E controller - et> you have a simple host-based control program, hook a disk (or et> storage system) to the ARM HBA, and you could have a nice et> little embedded ZFS system. haha yeah! Oxford 911 firewire-to-?ATA bridges already have an ARM core inside them. If such a thing is ever made, I hope it''s not sold by Sun so that I can demand CDDL source. Otherwise it will probably be treated like 7000---people will be meant to buy the card to get access to a special closed-source stable branch that has more bugfixes than sol10 but fewer regressions than SXCE. et> Either that, or if someone would figure out a way to have et> multiple-chip ARM implementations (where they could spread out et> the load efficiently). yeah seriously though, this is a good chip. it''s interesting in the same way SPARC is interesting---gate count per throughput, watts per throughput. The downside is that it doesn''t have the stone-squeezing high-end proprietary C compiler and fancy Java runtime with mature JIT that Sun has for SPARC. The upside is the price point is orders of magnitude off the T2000 which means it can seep into all kinds of weird fun markets. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090619/e97894fd/attachment.bin>
On Sat, Jun 20, 2009 at 2:53 AM, Miles Nordin<carton at ivy.net> wrote:>>>>>> "fan" == Fajar A Nugraha <fajar at fajar.net> writes: >>>>>> "et" == Erik Trimble <Erik.Trimble at Sun.COM> writes: > > ? fan> The N610N that I have (BCM3302, 300MHz, 64MB) isn''t even > ? fan> powerful enough to saturate either the gigabit wired > > I can''t find that device. ?Did you misspell it or something? ?BCM > probably means Broadcom, and Broadcom is probably MIPS---it''s TI > (omap) and Marvell (orion) that are selling arm.Correct, it''s MIPS. My point is the embedded device and cheap netbook I''ve used aren''t likely to be powerful enough for zfs. I have the impression that common ARM-based appliances today (like DLink''s DNS-323 NAS, 500 Mhz Marvell 88F5181 proprietary Feroceon ARM) would have similar performace characteristic and was wondering whether they are truly feasible targets for opensolaris and zfs.> That said, ARM is interesting because the chips just recently got a > lot faster at the same power/price point, like >1GHz.using zfs on THAT might make more sense :D -- Fajar
>Dennis is correct in that there are significant areas where 32-bit >systems will remain the norm for some time to come.think of that hundreds of thousands of VMWare ESX/Workstation/Player/Server installations on non VT capable cpu`s - even if the cpu has 64bit capability, a VM cannot run in 64bit mode the cpu is missing VT support. And VT isn`t available for so long, and still there are even recent CPUs which don`t have VT support.... -- This message posted from opensolaris.org
It''s actually worse than that--it''s not just "recent CPUs" without VT support. Very few of Intel''s current low-price processors, including the Q8xxx quad-core desktop chips, have VT support. On Wed, Jun 24, 2009 at 12:09 PM, roland<no-reply at opensolaris.org> wrote:>>Dennis is correct in that there are significant areas where 32-bit >>systems will remain the norm for some time to come. > > think of that hundreds of thousands of VMWare ESX/Workstation/Player/Server installations on non VT capable cpu`s - even if the cpu has 64bit capability, a VM cannot run in 64bit mode the cpu is missing VT support. And VT isn`t available for so long, and still there are even recent CPUs which don`t have VT support.... > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >