zfs discuss - Mar 2011 - best migration path from Solaris 10

We''ve been running Solaris 10 for the past couple of years, primarily
to
leverage zfs to provide storage for about 40,000 faculty, staff, and 
students as well as about 1000 groups. Access is provided via NFSv4, 
CIFS (by samba), and http/https (including a local module allowing 
filesystem acl''s to be respected via web access). This has worked 
reasonably well barring some ongoing issues with scalability 
(approximately a 2 hour reboot window on an x4500 with ~8000 zfs 
filesystems, complete breakage of live upgrade) and acl/chmod 
interaction madness.

We were just about to start working on a cutover to OpenSolaris (for the 
in-kernel CIFS server, and quicker access to new features/developments) 
when Oracle finished assimilating Sun and killed off the OpenSolaris 
distribution. We''ve been sitting pat for a while to see how things
ended
up shaking out, and at this point want to start reevaluating our best 
migration option to move forward from Solaris 10.

There''s really nothing else available that is comparable to zfs
(perhaps
btrfs someday in the indefinite future, but who knows when that day 
might come), so our options would appear to be Solaris 11 Express, 
Nexenta (either NexentaStor or NexentaCore), and OpenIndiana (FreeBSD is 
occasionally mentioned as a possibility, but I don''t really see that as
suitable for our enterprise needs).

Solaris 11 is the official successor to OpenSolaris, has commercial 
support, and the backing of a huge corporation which historically has 
contributed the majority of Solaris forward development. However, that 
corporation is Oracle, and frankly, I don''t like doing business with 
Oracle. With no offense intended to the no doubt numerous talented and 
goodhearted people that might work there, Oracle is simply evil. We''ve 
dealt with Oracle for a long time (in addition to their database itself, 
we''re a PeopleSoft shop) and a positive interaction with them is quite 
rare. Since they took over Sun, costs on licensing, support contracts, 
and hardware have increased dramatically, at least in the cases where 
we''ve actually been able to get a quote. Arguably, we are not their 
target market, and they make that quite clear ;). There''s also been 
significant brain drain of prior Sun employees since the takeover, so 
while they might still continue to contribute the most money into 
Solaris development, they might not be the future source of the most 
innovation. Given our needs, and our budget, I really don''t consider 
this a viable option.

Nexenta, on the other hand, seems to be the kind of company I''d like to
deal with. Relatively small, nimble, with a ton of former Sun zfs talent 
working for them, and what appears to be actual consideration for the 
needs of their customers. I think I''d more likely get my needs
addressed
through Nexenta, they''ve already started work on adding aclmode back
and
I''ve had some initial discussion with one of their engineers on the 
possibility of adding additional options such as denying or ignoring 
attempted chmod updates on objects with acls. It looks like they only 
offer commercial support for NexentaStor, not NexentaCore. Commercial 
support isn''t a strict requirement, a sizable chunk of our 
infrastructure runs on a non-commercial linux distribution and open 
source software, but it can make management happier. NexentaStor seems 
positioned as a storage appliance, which isn''t really what we need.
I''m
not particularly interested in a web gui or cli interface that hides the 
underlying complexity of the operating system and zfs, on the contrary, 
I want full access to the guts :). We have our zfs deployment integrated 
into our identity management system, which automatically provisions, 
destroys, and maintains filespace for our user/groups, as well as 
providing an API for end-users and administrators to manage quotas and 
other attributes. We also run apache with some custom modules. I still 
need to investigate further, but I''m not even sure if NexentaStor 
provides access into the underlying OS or encapsulates everything and 
only allows control through its own administrative functionality.

NexentaCore is more of the raw operating system we''re probably looking 
for, but with only community-based support. Given that NexentaCore and 
OpenIndiana are now both going to be based off of the illumos core, I''m
not quite certain what''s going to distinguish them. NexentaCore will 
continue to leverage debian packaging as opposed to IPS, and currently 
defaults to a GNU userland rather than native Solaris userland. However, 
I''ve read that the next version is going to switch to the solaris 
userland, so will no longer be different from OpenIndiana in that respect.

OpenIndiana was initially positioned to be a binary compatible 
open-source equivalent to Solaris 11, such as CentOS is to Red Hat. 
However, given that Oracle has not released any of their internal 
development work on which Solaris 11 express is based, and there''s 
really no certainty that it ever will, I''m not sure how that''s
going to
be accomplished. Particularly in the case of zfs, Oracle has moved 
forward with new pool versions which are not supported. Assuming Oracle 
does not release the corresponding source code, is the intention to 
duplicate the functionality with separately written code? And what will 
happen if for example an innovative new zfs feature is developed on the 
open source side, requiring a new pool version? Oracle seems to own that 
namespace.

My first inclination is to prefer OpenIndiana, which would seem to be 
the most flexible option. I particularly like to be able to fix 
something in the code myself and immediately deploy it and then try to 
get it accepted upstream rather than spending ages trying to get some 
support person to agree it should be done at all, let alone having an 
engineer update proprietary source code to implement it. I''ve got Sun 
(Oracle) support tickets that have been open for over a year waiting to 
get things fixed that I could have fixed myself.

Nexenta on the other hand has the only product ready for short term 
production deployment (well, ruling out Oracle, which I mostly have), as 
I don''t think there is a firm timeline yet on a production ready
version
of OpenIndiana. Of course, given the major changes in NexentaCore 4 
dropping a new deployment of NexentaCore 3 into place right now might 
not be the best idea either, it would probably be better to wait for 4 
to come out.

Sorry, I think I''m starting to ramble :). So, for those people with 
similar deployment needs as myself, with a large number of filesystems, 
a large population, and access via multiple protocols, what are you 
currently running? What do you plan to be running in the short to mid 
term? And why :)?


-- 
Paul B. Henson  |  (909) 979-6361  |  http://www.csupomona.edu/~henson/
Operating Systems and Network Analyst  |  henson at csupomona.edu
California State Polytechnic University  |  Pomona CA 91768

Thanks for thinking about us, Paul.

A few quick thoughts:

a) Nexenta Core Platform is a bare-bones OS.  No GUI, in other words (no
X11.)  It might well suit you.

b) NCP 3 will not have an upgrade path to NCP 4.  Its simply too much
change in the underlying packaging.

c) NCP 4 is still 5-6 months away.  We''re still developing it.

d) NCP 4 will make much more use of the illumos userland, and only use
Debian when illumos doesn''t have an equivalent.

e) NCP comes entirely unsupported.  NexentaStor is a commercial product
with real support behind it, though.

f) *Today*, NexentaStor 3 has newer code in it than NCP.  That will be
changing, as we will be keeping the two much more closely in sync
starting with 3.1.

g) If you want to self support, OpenIndiana or NCP are both good
options.  NCP has debian packaging, and lacks a bunch of the GUI
goodies.  NCP 3 is not as new as OI, but is probably a bit more proven.

Hopefully the additional information is helpful to you.

	- Garrett

I''m in a similar position, so I''ll be curious what kinds of
responses you
get.  I can give you a thumbnail sketch of what I''ve looked at so far:

I evaluated FreeBSD, and ruled it out because I need NFSv4, and
FreeBSD''s
NFSv4 support is still in an early stage.  The NFS stability and performance
just isn''t there yet, in my opinion.

Nexenta Core looked promising, but locked up in bonnie++ NFS testing with
our RedHat nodes, so its stability is a bit of a question mark for me.

I haven''t gotten the opportunity to thoroughly evaluate OpenIndiana,
yet.
 It''s only available as a DVD ISO, and my test machine currently has
only a
CD-ROM drive.  Changing that is on my to-do list, but other things keep
slipping in ahead of it.

For now I''m running OpenSolaris, with a locally-compiled version of
Samba.
 (The OpenSolaris Samba package is very old and has several unpatched
security holes, at this point.)

-- 
David Brodbeck
System Administrator, Linguistics
University of Washington
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20110318/f866d1a8/attachment.html>

On 18/03/11 5:56 PM, Paul B. Henson wrote:
> We''ve been running Solaris 10 for the past couple of years,
primarily to
> leverage zfs to provide storage for about 40,000 faculty, staff, and
> students ... and at this point want to start reevaluating our best
> migration option to move forward from Solaris 10.
> 
> There''s really nothing else available that is comparable to zfs
(perhaps
> btrfs someday in the indefinite future, but who knows when that day
> might come), so our options would appear to be Solaris 11 Express,
> Nexenta (either NexentaStor or NexentaCore), and OpenIndiana (FreeBSD is
> occasionally mentioned as a possibility, but I don''t really see
that as
> suitable for our enterprise needs).
> 
You''re not the only institution asking this question; here''s a
couple of
blog posts by Chris Siebenmann:
 * http://utcc.utoronto.ca/~cks/space/blog/solaris/OurFutureWithSolaris
 * http://utcc.utoronto.ca/~cks/space/blog/solaris/OurSolarisAlternatives

regards
--Toby

I think we all feel the same pain with Oracle''s purchase of Sun.

FreeBSD that has commercial support for ZFS maybe?

Not here quite yet, but it is something being looked at by an F500 that I am
currently on contract with.

www.freenas.org, www.ixsystems.com.

Not saying this would be the right solution by any means, but for that
''corporate barrier'', sometimes the option to get both the
hardware and ZFS from the same place, with support, helps out.

- mike


On Mar 18, 2011, at 2:56 PM, Paul B. Henson wrote:
> We''ve been running Solaris 10 for the past couple of years,
primarily to leverage zfs to provide storage for about 40,000 faculty, staff,
and students as well as about 1000 groups. Access is provided via NFSv4, CIFS
(by samba), and http/https (including a local module allowing filesystem
acl''s to be respected via web access). This has worked reasonably well
barring some ongoing issues with scalability (approximately a 2 hour reboot
window on an x4500 with ~8000 zfs filesystems, complete breakage of live
upgrade) and acl/chmod interaction madness.
> 
> We were just about to start working on a cutover to OpenSolaris (for the
in-kernel CIFS server, and quicker access to new features/developments) when
Oracle finished assimilating Sun and killed off the OpenSolaris distribution.
We''ve been sitting pat for a while to see how things ended up shaking
out, and at this point want to start reevaluating our best migration option to
move forward from Solaris 10.
> 
> There''s really nothing else available that is comparable to zfs
(perhaps btrfs someday in the indefinite future, but who knows when that day
might come), so our options would appear to be Solaris 11 Express, Nexenta
(either NexentaStor or NexentaCore), and OpenIndiana (FreeBSD is occasionally
mentioned as a possibility, but I don''t really see that as suitable for
our enterprise needs).
> 
> Solaris 11 is the official successor to OpenSolaris, has commercial
support, and the backing of a huge corporation which historically has
contributed the majority of Solaris forward development. However, that
corporation is Oracle, and frankly, I don''t like doing business with
Oracle. With no offense intended to the no doubt numerous talented and
goodhearted people that might work there, Oracle is simply evil. We''ve
dealt with Oracle for a long time (in addition to their database itself,
we''re a PeopleSoft shop) and a positive interaction with them is quite
rare. Since they took over Sun, costs on licensing, support contracts, and
hardware have increased dramatically, at least in the cases where we''ve
actually been able to get a quote. Arguably, we are not their target market, and
they make that quite clear ;). There''s also been significant brain
drain of prior Sun employees since the takeover, so while they might still
continue to contribute the most money into Solaris development, they might not
be the future source of the most innovation. Given our needs, and our budget, I
really don''t consider this a viable option.
> 
> Nexenta, on the other hand, seems to be the kind of company I''d
like to deal with. Relatively small, nimble, with a ton of former Sun zfs talent
working for them, and what appears to be actual consideration for the needs of
their customers. I think I''d more likely get my needs addressed through
Nexenta, they''ve already started work on adding aclmode back and
I''ve had some initial discussion with one of their engineers on the
possibility of adding additional options such as denying or ignoring attempted
chmod updates on objects with acls. It looks like they only offer commercial
support for NexentaStor, not NexentaCore. Commercial support isn''t a
strict requirement, a sizable chunk of our infrastructure runs on a
non-commercial linux distribution and open source software, but it can make
management happier. NexentaStor seems positioned as a storage appliance, which
isn''t really what we need. I''m not particularly interested in
a web gui or cli interface that hides the underlying complexity of the operating
system and zfs, on the contrary, I want full access to the guts :). We have our
zfs deployment integrated into our identity management system, which
automatically provisions, destroys, and maintains filespace for our user/groups,
as well as providing an API for end-users and administrators to manage quotas
and other attributes. We also run apache with some custom modules. I still need
to investigate further, but I''m not even sure if NexentaStor provides
access into the underlying OS or encapsulates everything and only allows control
through its own administrative functionality.
> 
> NexentaCore is more of the raw operating system we''re probably
looking for, but with only community-based support. Given that NexentaCore and
OpenIndiana are now both going to be based off of the illumos core, I''m
not quite certain what''s going to distinguish them. NexentaCore will
continue to leverage debian packaging as opposed to IPS, and currently defaults
to a GNU userland rather than native Solaris userland. However, I''ve
read that the next version is going to switch to the solaris userland, so will
no longer be different from OpenIndiana in that respect.
> 
> OpenIndiana was initially positioned to be a binary compatible open-source
equivalent to Solaris 11, such as CentOS is to Red Hat. However, given that
Oracle has not released any of their internal development work on which Solaris
11 express is based, and there''s really no certainty that it ever will,
I''m not sure how that''s going to be accomplished. Particularly
in the case of zfs, Oracle has moved forward with new pool versions which are
not supported. Assuming Oracle does not release the corresponding source code,
is the intention to duplicate the functionality with separately written code?
And what will happen if for example an innovative new zfs feature is developed
on the open source side, requiring a new pool version? Oracle seems to own that
namespace.
> 
> My first inclination is to prefer OpenIndiana, which would seem to be the
most flexible option. I particularly like to be able to fix something in the
code myself and immediately deploy it and then try to get it accepted upstream
rather than spending ages trying to get some support person to agree it should
be done at all, let alone having an engineer update proprietary source code to
implement it. I''ve got Sun (Oracle) support tickets that have been open
for over a year waiting to get things fixed that I could have fixed myself.
> 
> Nexenta on the other hand has the only product ready for short term
production deployment (well, ruling out Oracle, which I mostly have), as I
don''t think there is a firm timeline yet on a production ready version
of OpenIndiana. Of course, given the major changes in NexentaCore 4 dropping a
new deployment of NexentaCore 3 into place right now might not be the best idea
either, it would probably be better to wait for 4 to come out.
> 
> Sorry, I think I''m starting to ramble :). So, for those people
with similar deployment needs as myself, with a large number of filesystems, a
large population, and access via multiple protocols, what are you currently
running? What do you plan to be running in the short to mid term? And why :)?
> 
> 
> -- 
> Paul B. Henson  |  (909) 979-6361  |  http://www.csupomona.edu/~henson/
> Operating Systems and Network Analyst  |  henson at csupomona.edu
> California State Polytechnic University  |  Pomona CA 91768
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Hi David,

Caught your note about bonnie, actually do some testing myself over the weekend.

All on older hardware for fun - dual opteron 285 with 16GB RAM.  Disk systems is
off a pair of SuperMicro SATA cards, with a combination of WD enterprise and
Seagate ES 1TB drives.  No ZIL, no L2ARC, no tuning at all from base FreeNAS
install.

10 drives total, I''m going to be running tests as below, mostly curious
about IOPS and to sort out a little debate with a co-worker.

- all 10 in one raidz2 (running now)
- 5 by 2-way mirrors
- 2 by 5-disk raidz1

Script is as below - if folks would find the data I collect be useful
information at all, let me know and I will post it publicly somewhere.




freenas# cat test.sh
#!/bin/sh

# Basic test for file I/O.  We run lots and lots of the tradditional
# ''bonnie'' tool at 50GB file size, starting one every minute. 
Resulting
# data should give us a good work mixture in the middle given all the different
# tests that bonnnie runs, 100 instances running at the same time, and at
different
# stages of their processing.


MAX=100
COUNT=0

FILESYSTEM=testrz2
LOG=${FILESYSTEM}.log


date > ${LOG}
echo "Test with file system named ${FILESYSTEM} and Configuration
of..." >> ${LOG}
zpool status >> ${LOG}

# DEMAN grab zfs and regular dev iostats every 10 minutes during test
zpool iostat -v 600 >>  ${LOG} &
iostat -w 600 ada0 ada1 ada2 ada3 ada4 ada5 ada6 ada7 ada8 ada9 >
${LOG}.iostat &


while [ $COUNT -le $MAX ]; do
	echo kicking off bonnie
	bonnie -d /mnt/${FILESYSTEM} -s 50000 &
	sleep 60;
	COUNT=$((count+1));
done;



On Mar 18, 2011, at 3:26 PM, David Brodbeck wrote:
> I''m in a similar position, so I''ll be curious what kinds
of responses you get.  I can give you a thumbnail sketch of what I''ve
looked at so far:
> 
> I evaluated FreeBSD, and ruled it out because I need NFSv4, and
FreeBSD''s NFSv4 support is still in an early stage.  The NFS stability
and performance just isn''t there yet, in my opinion.
> 
> Nexenta Core looked promising, but locked up in bonnie++ NFS testing with
our RedHat nodes, so its stability is a bit of a question mark for me.
> 
> I haven''t gotten the opportunity to thoroughly evaluate
OpenIndiana, yet.  It''s only available as a DVD ISO, and my test
machine currently has only a CD-ROM drive.  Changing that is on my to-do list,
but other things keep slipping in ahead of it.
> 
> For now I''m running OpenSolaris, with a locally-compiled version
of Samba.  (The OpenSolaris Samba package is very old and has several unpatched
security holes, at this point.)
> 
> -- 
> David Brodbeck
> System Administrator, Linguistics
> University of Washington
> 
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

> I think we all feel the same pain with Oracle''s purchase of Sun.
> 
> FreeBSD that has commercial support for ZFS maybe?
Fbsd currently has a very old zpool version, not suitable for running with
SLOGs, since if you lose it, you may lose the pool, which isn''t very
amusing...

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 97542685
roy at karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er
et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av
idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og
relevante synonymer p? norsk.

Newer versions of FreeBSD have newer ZFS code.

That said, ZFS on FreeBSD is kind of a 2nd class citizen still.  FreeBSD
still gives equal (or higher) priority to ufs, and so some of the
changes in Solaris and derivatives (illumos) to make certain things like
NFS, CIFS, and COMSTAR/iSCSI work better with ZFS won''t be present in
FreeBSD.

There are vendors who offer NexentaStor on hardware with full commercial
support from a single vendor (granted they get backline support from
Nexenta, but do you think ixSystems engineers personally fix bugs in
FreeBSD?)  Such vendors include PogoLinux and AreaData.

I''ve also started conversations with Pogo about offering an OpenIndiana
based workstation, which might be another option if you prefer more of a
general purpose solution.

	- Garrett

On Sat, 2011-03-19 at 02:16 +0100, Roy Sigurd Karlsbakk
wrote:
> > I think we all feel the same pain with Oracle''s purchase of
Sun.
> > 
> > FreeBSD that has commercial support for ZFS maybe?
> 
> Fbsd currently has a very old zpool version, not suitable for running with
SLOGs, since if you lose it, you may lose the pool, which isn''t very
amusing...
> 
> Vennlige hilsener / Best regards
> 
> roy
> --
> Roy Sigurd Karlsbakk
> (+47) 97542685
> roy at karlsbakk.net
> http://blogg.karlsbakk.net/
> --
> I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det
er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av
idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og
relevante synonymer p? norsk.
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

ZFSv28 is in HEAD now and will be out in 8.3.

ZFS + HAST in 9.x means being able to cluster off different hardware.

In regards to OpenSolaris and Indiana - can somebody clarify the relationship
there?  It was clear with OpenSolaris that the latest/greatest ZFS would always
be available since it was a guinea-pig product for cost conscious folks and
served as an excellent area for Sun to get marketplace feedback and bug fixes
done before rolling updates into full Solaris.

To me it seems that Open Indiana is basically a green branch off of a dead tree
- if I am wrong, please enlighten me.

On Mar 18, 2011, at 6:16 PM, Roy Sigurd Karlsbakk wrote:
>> I think we all feel the same pain with Oracle''s purchase of
Sun.
>> 
>> FreeBSD that has commercial support for ZFS maybe?
> 
> Fbsd currently has a very old zpool version, not suitable for running with
SLOGs, since if you lose it, you may lose the pool, which isn''t very
amusing...
> 
> Vennlige hilsener / Best regards
> 
> roy
> --
> Roy Sigurd Karlsbakk
> (+47) 97542685
> roy at karlsbakk.net
> http://blogg.karlsbakk.net/
> --
> I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det
er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av
idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og
relevante synonymer p? norsk.

On Mar 18, 2011, at 21:16, Roy Sigurd Karlsbakk wrote:
>> I think we all feel the same pain with Oracle''s purchase of
Sun.
>> 
>> FreeBSD that has commercial support for ZFS maybe?
> 
> Fbsd currently has a very old zpool version, not suitable for running with
SLOGs, since if you lose it, you may lose the pool, which isn''t very
amusing...
For commercial FreeBSD support:

	http://www.freebsd.org/commercial/commercial.html

ZFSv28 has been commited to HEAD / 9.x:

	http://lists.freebsd.org/pipermail/freebsd-current/2011-February/023132.html

and there''s patches available for 8.x:

	http://people.freebsd.org/~mm/patches/zfs/v28/
	http://lists.freebsd.org/pipermail/freebsd-fs/

Heck, ZFS on Mac OS X may be available (again) by this summer:

	http://tinyurl.com/68eg6bf
	http://arstechnica.com/apple/news/2011/03/how-zfs-is-slowly-making-its-way-to-mac-os-x.ars

Oracle has said that they "will distribute updates to approved CDDL or
other open source- licensed code following full releases of our enterprise
Solaris operating system."

	http://unixconsole.blogspot.com/2010/08/internal-oracle-memo-leaked-on-solaris.html

I guess we''ll see what happens once Solaris 11 comes out officially.

On 03/19/11 12:17 AM, Toby Thain wrote:
> On 18/03/11 5:56 PM, Paul B. Henson wrote:
>> We''ve been running Solaris 10 for the past couple of years,
primarily to
>> leverage zfs to provide storage for about 40,000 faculty, staff, and
>> students ... and at this point want to start reevaluating our best
>> migration option to move forward from Solaris 10.
>>
>> There''s really nothing else available that is comparable to
zfs (perhaps
>> btrfs someday in the indefinite future, but who knows when that day
>> might come), so our options would appear to be Solaris 11 Express,
>> Nexenta (either NexentaStor or NexentaCore), and OpenIndiana (FreeBSD
is
>> occasionally mentioned as a possibility, but I don''t really
see that as
>> suitable for our enterprise needs).
Questions are: Do you care of your OS being open and not tight to only
one company, and do you care for software and packaging compatibility
and do you need payed support or not and do you need it right now or in
the future?
Do you want to tie yourself with Oracle and closed Solaris products?
(even if unofficially there were saying that they might open code after
S11 release)
If you used closed product before, that might be your enterprise upgrade
path. Just prepare to cache Oracle out and that is it.

If you want to use free open source with ability to buy suport and all
you want to use is zfs,
then Nexenta is your way with their both free to use releases and
commercially supported ones.
Nexenta support development of Illumos that is future base of
OpenIndiana, too.
So Nexenta is something like what Sun previously was doing, they are
actively developing it and you can have support for less money then from
Oracle, I suppose.

OpenIndiana is and will contiue to be closest you can get to Oracle
Solaris releases. It shares software consolidations (and packaging,
IPS,pkg) with closed brother. OpenIndiana has stable release in mind in
near future, that might suit your needs.
Dev OpenIndiana releases are (slowly) following path of OpenSolaris dev
releases,
so OpenIndiana can be right now Solaris 10 replacement (many people just
continued to use OI dev) and in the future, with transition to Illumos
base ahead in mind.

I think that best thing you can do is to install OpenSolaris snv_134 (or
134b) and from that point you can see where you can go: To OpenIndiana
dev and then follow Illumos development and wait for OpenIndiana stable
, And try even closed Solaris Express 11.
(with No zfs upgrade to Solaris Express version (!) - Be sure Not to do
zfs and zpool upgrade to closed Solaris 11 express version, because you
will be then locked-in in Oracle zfs versions.)
I do not know how Nexenta could be installed in the same zpool in new BE
but I suppose it can, since I know upgrading Nexenta use zfs BE''s, too.

That way, with multiple installs and sharing zfs between them, you are
on safe ground of being able to test and choose to what will come in
future and ,beside Oracle, there are at least 2 solutions now and in the
future, that you can consider.

I would personally like if one could buy support from Nexenta and
continue to use OpenIndiana or Nexenta :) But Nexenta is more
server-like and OpenIndiana is shooting to all-around solution.

On Fri, Mar 18, 2011 at 06:26:37PM -0700, Michael DeMan
wrote:
> ZFSv28 is in HEAD now and will be out in 8.3.
> 
> ZFS + HAST in 9.x means being able to cluster off different hardware.
> 
> In regards to OpenSolaris and Indiana - can somebody clarify the
relationship there?  It was clear with OpenSolaris that the latest/greatest ZFS
would always be available since it was a guinea-pig product for cost conscious
folks and served as an excellent area for Sun to get marketplace feedback and
bug fixes done before rolling updates into full Solaris.
> 
> To me it seems that Open Indiana is basically a green branch off of a dead
tree - if I am wrong, please enlighten me.
> 
Illumos project was started as a fork of OpenSolaris when Oracle was still
publishing OpenSolaris sources.

Then Oracle closed OpenSolaris development, and decided to call upcoming
(closed) versions "Solaris 11 Express",
with no source included.

Illumos project continued the development based on the latest published
OpenSolaris sources,
and a bit later OpenIndiana *distribution* was announced to deliver a binary
distro based on OpenSolaris/Illumos.

So in short Illumos is the development project, which hosts the new sources, and
OpenIndiana is a binary distro based on it.


-- Pasi
> On Mar 18, 2011, at 6:16 PM, Roy Sigurd Karlsbakk wrote:
> 
> >> I think we all feel the same pain with Oracle''s purchase
of Sun.
> >> 
> >> FreeBSD that has commercial support for ZFS maybe?
> > 
> > Fbsd currently has a very old zpool version, not suitable for running
with SLOGs, since if you lose it, you may lose the pool, which isn''t
very amusing...
> > 
> > Vennlige hilsener / Best regards
> > 
> > roy
> > --
> > Roy Sigurd Karlsbakk
> > (+47) 97542685
> > roy at karlsbakk.net
> > http://blogg.karlsbakk.net/
> > --
> > I all pedagogikk er det essensielt at pensum presenteres
intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng?
eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller
eksisterer adekvate og relevante synonymer p? norsk.
> 
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Nexenta are a great company (I''m no way affiliated with them btw), if
for no
other reason being willing to invest in Illumos and by that OpenIndiana and
NCP (for which they charge nothing). If you need a large enterprise
commercially backed storage server system, NextentaStor is the answer.

If you want a CLI OS, NCP or OpenIndiana text only will fulfill that spot
(there is also illumos-extra which is even more stripped down to bare
minimum), if a GUI or desktop is required OpenIndiana has that covered.

Whilst there isn''t yet official commercial support for OpenIndiana,
that is
something that has been brought up and the feeling is that it is something
that would like to be offered at some point in time, it is just the
logistics and organization of that support has to be setup. If it became a
deal breaker I''d suggest talking to a senior person of OI, as it
possible
something could be arranged.

I expect we are no more than 6 months away from Illumos being the defacto
open source foundation for all the major distributions, which will then
start freeing up a lot of developer resources to continue and improve
things, beyond this initial get everything together period.

ZFS open source future is being hammered out, with hopefully a cross
platform working body to promote and move it forward appearing. I know
everyone involved is both eager to make sure it''s truly open and
portable
(with support for Illumos/OI, FreeBSD, Linux and OS-X in working or
in-progress) and that it has a future regardless of any code drops from
Oracle (they are of course welcome if they come, but clearly at this point
it can''t be relied upon).

Personally I think OpenIndiana with Illumos foundations, is a great
enterprise quality open source OS with a great future and a bunch of really
committed guys and gals behind it.

My 2P,
Deano

-----Original Message-----
From: zfs-discuss-bounces at opensolaris.org
[mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Garrett
D''Amore
Sent: 18 March 2011 22:15
To: Paul B. Henson
Cc: openindiana-discuss at openindiana.org; zfs-discuss at opensolaris.org
Subject: Re: [zfs-discuss] best migration path from Solaris 10

Thanks for thinking about us, Paul.

A few quick thoughts:

a) Nexenta Core Platform is a bare-bones OS.  No GUI, in other words (no
X11.)  It might well suit you.

b) NCP 3 will not have an upgrade path to NCP 4.  Its simply too much
change in the underlying packaging.

c) NCP 4 is still 5-6 months away.  We''re still developing it.

d) NCP 4 will make much more use of the illumos userland, and only use
Debian when illumos doesn''t have an equivalent.

e) NCP comes entirely unsupported.  NexentaStor is a commercial product
with real support behind it, though.

f) *Today*, NexentaStor 3 has newer code in it than NCP.  That will be
changing, as we will be keeping the two much more closely in sync
starting with 3.1.

g) If you want to self support, OpenIndiana or NCP are both good
options.  NCP has debian packaging, and lacks a bunch of the GUI
goodies.  NCP 3 is not as new as OI, but is probably a bit more proven.

Hopefully the additional information is helpful to you.

	- Garrett


_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

That?s not quite right, Illumos is open source continuation of ONNV, which
is the core foundation, however it doesn''t include other consolidation
that
made up OpenSolaris.

OpenIndiana does, it takes all those consolidations and produces a working
OS you can install. Of course the biggest and most important of those
consolidation is Illumos. The reason OI is only now slowly moving to
Illumos, it has many other consolidations that also required work before a
safe move could be made, and also keeping OpenIndiana in line with the
closed source variants.

There are other forks for both areas (ONNV replacements and installable OS
distributions), if for some reason Illumos and OpenIndiana aren''t
suitable.

HTH,
Deano



-----Original Message-----
From: Pasi K?rkk?inen [mailto:pasik at iki.fi] 
Sent: 19 March 2011 14:58
To: Michael DeMan
Cc: openindiana-discuss at openindiana.org; zfs-discuss at opensolaris.org
Subject: Re: [OpenIndiana-discuss] [zfs-discuss] best migration path from
Solaris 10

On Fri, Mar 18, 2011 at 06:26:37PM -0700, Michael DeMan
wrote:
> ZFSv28 is in HEAD now and will be out in 8.3.
> 
> ZFS + HAST in 9.x means being able to cluster off different hardware.
> 
> In regards to OpenSolaris and Indiana - can somebody clarify the
relationship there?  It was clear with OpenSolaris that the latest/greatest
ZFS would always be available since it was a guinea-pig product for cost
conscious folks and served as an excellent area for Sun to get marketplace
feedback and bug fixes done before rolling updates into full
Solaris.
> 
> To me it seems that Open Indiana is basically a green branch off of a dead
tree - if I am wrong, please enlighten me.
> 
Illumos project was started as a fork of OpenSolaris when Oracle was still
publishing OpenSolaris sources.

Then Oracle closed OpenSolaris development, and decided to call upcoming
(closed) versions "Solaris 11 Express",
with no source included.

Illumos project continued the development based on the latest published
OpenSolaris sources, 
and a bit later OpenIndiana *distribution* was announced to deliver a binary
distro based on OpenSolaris/Illumos.

So in short Illumos is the development project, which hosts the new sources,
and OpenIndiana is a binary distro based on it.


-- Pasi
> On Mar 18, 2011, at 6:16 PM, Roy Sigurd Karlsbakk wrote:
> 
> >> I think we all feel the same pain with Oracle''s purchase
of Sun.
> >> 
> >> FreeBSD that has commercial support for ZFS maybe?
> > 
> > Fbsd currently has a very old zpool version, not suitable for running
with SLOGs, since if you lose it, you may lose the pool, which isn''t
very
amusing...
> > 
> > Vennlige hilsener / Best regards
> > 
> > roy
> > --
> > Roy Sigurd Karlsbakk
> > (+47) 97542685
> > roy at karlsbakk.net
> > http://blogg.karlsbakk.net/
> > --
> > I all pedagogikk er det essensielt at pensum presenteres
intelligibelt.
Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv
anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller
eksisterer adekvate og relevante synonymer p? norsk.
> 
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
OpenIndiana-discuss mailing list
OpenIndiana-discuss at openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

On Fri, Mar 18, 2011 at 06:22:01PM -0700, Garrett D''Amore
wrote:
> Newer versions of FreeBSD have newer ZFS code.
Yes, we are at v28 at this point (the lastest open-source version).
> That said, ZFS on FreeBSD is kind of a 2nd class citizen still. [...]
That''s actually not true. There are more FreeBSD committers working on
ZFS than on UFS.
> There are vendors who offer NexentaStor on hardware with full commercial
> support from a single vendor (granted they get backline support from
> Nexenta, but do you think ixSystems engineers personally fix bugs in
> FreeBSD?) [...]
iXsystems works very closely with the FreeBSD project. They hire or
contract quite a few FreeBSD committers (FYI I''m not one of them), so
yes, they are definitely in position to fix bugs in FreeBSD, as well as
develop new stuff and they do that.

Just wanted to clarify few points:)

-- 
Pawel Jakub Dawidek                       http://www.wheelsystems.com
FreeBSD committer                         http://www.FreeBSD.org
Am I Evil? Yes, I Am!                     http://yomoli.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20110319/665dfaf7/attachment.bin>

On Sun, Mar 20, 2011 at 4:05 AM, Pawel Jakub Dawidek <pjd at freebsd.org>
wrote:
> On Fri, Mar 18, 2011 at 06:22:01PM -0700, Garrett D''Amore wrote:
>> Newer versions of FreeBSD have newer ZFS code.
>
> Yes, we are at v28 at this point (the lastest open-source version).
>
>> That said, ZFS on FreeBSD is kind of a 2nd class citizen still. [...]
>
> That''s actually not true. There are more FreeBSD committers
working on
> ZFS than on UFS.
How is the performance of ZFS under FreeBSD? Is it comparable to that
in Solaris, or still slower due to some needed compatibility layer?

-- 
Fajar

Probably, we need place a tag before zfs -- Opensource-ZFS or Oracle-ZFS after
Solaris11 release.
If it is true, these two ZFSes will definitely evolve into different directions.
BTW, Did Oracle unveil the actual release date? We are also at the cross road...

Thanks.

Fred
> -----Original Message-----
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Fajar A. Nugraha
> Sent: ???, ?? 20, 2011 14:55
> To: Pawel Jakub Dawidek
> Cc: openindiana-discuss at openindiana.org; zfs-discuss at opensolaris.org
> Subject: Re: [zfs-discuss] best migration path from Solaris 10
> 
> On Sun, Mar 20, 2011 at 4:05 AM, Pawel Jakub Dawidek <pjd at
freebsd.org>
> wrote:
> > On Fri, Mar 18, 2011 at 06:22:01PM -0700, Garrett D''Amore
wrote:
> >> Newer versions of FreeBSD have newer ZFS code.
> >
> > Yes, we are at v28 at this point (the lastest open-source version).
> >
> >> That said, ZFS on FreeBSD is kind of a 2nd class citizen still.
[...]
> >
> > That''s actually not true. There are more FreeBSD committers
working
> on
> > ZFS than on UFS.
> 
> How is the performance of ZFS under FreeBSD? Is it comparable to that
> in Solaris, or still slower due to some needed compatibility layer?
> 
> --
> Fajar
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Fred Liu <Fred_Liu at issi.com> wrote:
> Probably, we need place a tag before zfs -- Opensource-ZFS or Oracle-ZFS
after Solaris11 release.
> If it is true, these two ZFSes will definitely evolve into different
directions.
> BTW, Did Oracle unveil the actual release date? We are also at the cross
road...
The long term acceptance for ZFS depends on how Oracle will behave past the 
announced Solaris 11 is released. If they don''t Opensource the related
ZFS,
they will harm the future of ZFS. If they Opensource it again, there is still a 
problem with syncing the ZFS ve3rsions from the OSS OpenSolaris continuation 
projects.

The revision number introduced by Sun is only useful if there is no more than 
a single entity that introduces new features.

For a reliable future for a distributed ZFS development, we would need 
something like the POSIX method to introduce tar extensions:

	a combination of a textual name for the entity that introduced the
	fature and a textual name for the feature.

	e.g. SCHILY-zfs-encryption

J?rg

-- 
 EMail:joerg at schily.net                  (home) J?rg Schilling D-13353 Berlin
       js at cs.tu-berlin.de                (uni)  
       joerg.schilling at fokus.fraunhofer.de (work) Blog:
http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

On Mar 20, 2011, at 09:26, Joerg Schilling wrote:
> The long term acceptance for ZFS depends on how Oracle will behave past the
> announced Solaris 11 is released. If they don''t Opensource the
related ZFS,
> they will harm the future of ZFS. If they Opensource it again, there is
still a
> problem with syncing the ZFS ve3rsions from the OSS OpenSolaris
continuation
> projects.
For a while Apple was considering it, and if Ellison and Jobs can come to an
agreement, it would certainly become very popular very quickly.

Apple probably ships more UNIX(tm) devices than any other  vendor (often over 3M
units in a quarter). Using revenue as a metric gives similar results. And who
says the Unix "workstation" market is dead? :)

On Mar 20, 2011, at 14:33, Garrett D''Amore wrote:
> I hear from reliable sources that Apple is not doing anything with ZFS,
> so I would not look there for leadership.
Given that one of the prominent (?) file system guys at Apple left to form his
own ZFS company, I figured that was the case even before you stated the above:

http://tinyurl.com/4jznw48
http://arstechnica.com/apple/news/2011/03/how-zfs-is-slowly-making-its-way-to-mac-os-x.ars

The ZFS Working Group is awesome news. I hope to hear of a bright future for ZFS
on all operating systems.

On 3/18/2011 3:15 PM, Garrett D''Amore wrote:
> a) Nexenta Core Platform is a bare-bones OS.  No GUI, in other words
> (no X11.)  It might well suit you.
Indeed :), my servers are headless (well, as headless as you can get on
x86 hardware 8-/, they do have an ipmi remote console that still needs
to be used occasionally <sigh>) and I generally install a minimal set of
packages. We have the X client libraries installed on some of our linux
servers, as our DBA''s like to run the gui oracle installer, but I
don''t
recall ever needing to run X software on our storage servers. One of my
many spats with Oracle technical support (the database side, not the
operating system side) was trying to get them to justify why the
"xscreensaver" package was listed as a core dependency of running 10g
under RHEL 5 :(. Never did get an answer to that, they just closed the
ticket out from under me...
> c) NCP 4 is still 5-6 months away.  We''re still developing it.
By the time I do some initial evaluation, then some prototyping, I
don''t
anticipate migrating anything production wise until at the earliest
Christmas break, so that timing shouldn''t be a problem. Any thoughts on
how soon a beta might be available? As it sounds like there will be
significant changes, it might be better to evaluate with a beta of the
new stuff rather than the production version of the older stuff. Plus I
generally tend to break things in unexpected ways ;), so doing that in
the beta cycle might be beneficial.
> d) NCP 4 will make much more use of the illumos userland, and only
> use Debian when illumos doesn''t have an equivalent.
Given both NCP and OpenIndiana will be based off of illumos, and as of
version 4 NCP will be migrating as much as possible of the userland to
solaris as opposed to gnu, other than the differing packaging formats
what do you feel will distinguish NCP from openindiana? NCP is positioned as
a bare-bones server, whereas openindiana is trying to be more general
purpose including desktop use?
> e) NCP comes entirely unsupported.  NexentaStor is a commercial
> product with real support behind it, though.
Can you treat NexentaStor like a general purpose operating system, not
use the management gui, and configure everything from a shell prompt, or
is it more appliance like and you''re locked out from the OS? In other
words, would it be possible (although not necessarily cost-effective) to
pay for NexentaStor for the support but treat it like NCP?

Has your company considered basic support contracts for NCP? I''ve heard
from at least one other site that might be interested in something like
that. We don''t need much in the way of handholding, the majority of our
support calls end up being actual bugs or limitations in solaris. But if
one of our file servers panics, doesn''t import a pool when it boots,
and
crashes every time you try to import it by hand, it would be nice to
have an engineer available :).

Thanks...


-- 
Paul B. Henson  |  (909) 979-6361  |  http://www.csupomona.edu/~henson/
Operating Systems and Network Analyst  |  henson at csupomona.edu
California State Polytechnic University  |  Pomona CA 91768

On Mon, 2011-03-21 at 14:56 -0700, Paul B. Henson wrote:
> On 3/18/2011 3:15 PM, Garrett D''Amore wrote:
> 
> > c) NCP 4 is still 5-6 months away.  We''re still developing
it.
> 
> By the time I do some initial evaluation, then some prototyping, I
don''t
> anticipate migrating anything production wise until at the earliest
> Christmas break, so that timing shouldn''t be a problem. Any
thoughts on
> how soon a beta might be available? As it sounds like there will be
> significant changes, it might be better to evaluate with a beta of the
> new stuff rather than the production version of the older stuff. Plus I
> generally tend to break things in unexpected ways ;), so doing that in
> the beta cycle might be beneficial.
I *hate* talking about unreleased product schedules, but I think you can
expect a beta with a month or two, perhaps less.  We''ve already got an
alpha that we''ve handed out in limited quantities.
> 
> > d) NCP 4 will make much more use of the illumos userland, and only
> > use Debian when illumos doesn''t have an equivalent.
> 
> Given both NCP and OpenIndiana will be based off of illumos, and as of
> version 4 NCP will be migrating as much as possible of the userland to
> solaris as opposed to gnu, other than the differing packaging formats
> what do you feel will distinguish NCP from openindiana? NCP is positioned
as
> a bare-bones server, whereas openindiana is trying to be more general
> purpose including desktop use?
NCP is a core-technology thing.  Definitely not a general purpose OS at
all, and will be missing all the desktop stuff.

The idea behind NCP is that other distros build on top of, or people who
just want that bare bones OS use it.  It comes with debian packaging,
and we do have a bunch of the common server packages (Apache, etc.) set
up, but not everything that you might want.
> 
> > e) NCP comes entirely unsupported.  NexentaStor is a commercial
> > product with real support behind it, though.
> 
> Can you treat NexentaStor like a general purpose operating system, not
> use the management gui, and configure everything from a shell prompt, or
> is it more appliance like and you''re locked out from the OS? In
other
> words, would it be possible (although not necessarily cost-effective) to
> pay for NexentaStor for the support but treat it like NCP?
Once you dive under the controlled UI (which you can do), you basically
are breaking your support contract.

Going forward, NCP and NS will be more closely synchronized, so you''ll
be able to get the same OS, and probably receive patches to it, that you
get with NS, albeit without official support and without the proprietary
add-on features like HA clustering, the management UI,
auto-tiering/auto-sync, etc.
> 
> Has your company considered basic support contracts for NCP? I''ve
heard
> from at least one other site that might be interested in something like
> that. We don''t need much in the way of handholding, the majority
of our
> support calls end up being actual bugs or limitations in solaris. But if
> one of our file servers panics, doesn''t import a pool when it
boots, and
> crashes every time you try to import it by hand, it would be nice to
> have an engineer available :).
There have been some discussions, but figuring out how to make that
commercially worthwhile is challenging.  At some level, our engineers
are busy enough that we''d have to see enough commercial demand here to
justify adding engineers, because the number of calls we would take
would probably go significantly with such a change.

	- Garrett

On 3/18/2011 6:32 PM, David Magda wrote:
> Oracle has said that they "will distribute updates to approved CDDL
> or other open source- licensed code following full releases of our
> enterprise Solaris operating system."
>
>
http://unixconsole.blogspot.com/2010/08/internal-oracle-memo-leaked-on-solaris.html
Hmm, I dunno that I''d take a quote from a leaked internal memo as
gospel
;). For that matter, even if they flat out publicly announced it I
can''t
say I''d trust them to actually follow through...


--
Paul B. Henson  |  (909) 979-6361  |  http://www.csupomona.edu/~henson/
Operating Systems and Network Analyst  |  henson at csupomona.edu
California State Polytechnic University  |  Pomona CA 91768

On 3/21/2011 2:59 PM, Garrett D''Amore wrote:
> I *hate* talking about unreleased product schedules
:).
> but I think you can expect a beta with a month or two, perhaps less.
> We''ve already got an alpha that we''ve handed out in
limited
> quantities.
Actually, I read about that alpha; one of my coworkers was at SCALE 9x, 
if I''d known at the time I would have had him pick up a CD ;).
> Once you dive under the controlled UI (which you can do), you
> basically are breaking your support contract.
Meh :(, that rules it out for me; I need to run our own custom stuff to 
integrate it into our identity management platform.
> add-on features like HA clustering, the management UI,
> auto-tiering/auto-sync, etc.
HA clustering I would actually be interested in, depending on pricing; 
but unfortunately not in an appliance-only availability.
> There have been some discussions, but figuring out how to make that
> commercially worthwhile is challenging
Agreed. If not support contracts, what about engineering services 
available on a time/materials basis? That would cover my main concern of 
having expertise available in case of a critical failure. There might 
also be occasions where a specific bug has already been identified, but 
local resources lack sufficient time or knowledge to efficiently fix it. 
One of the people I''ve spoken to off-line mentioned a handful of known 
opensolaris bugs he''d really like to see resolved in NCP and would be 
willing to pay somebody to make it happen.

Thanks for the info...


-- 
Paul B. Henson  |  (909) 979-6361  |  http://www.csupomona.edu/~henson/
Operating Systems and Network Analyst  |  henson at csupomona.edu
California State Polytechnic University  |  Pomona CA 91768

> I''ve also started conversations with Pogo about offering an
OpenIndiana
> based workstation, which might be another option if you prefer more of
a
> general purpose solution.
> 
> 	- Garrett
Just to highlight a point that seems often lost here - not everyone uses
Solaris/ZFS as a "file storage appliance"/"home NAS and
workstation"/"workstation". Sometimes, people want to run
applications
on the machine too. :)

ZFS became such a focus and driver for Solaris that sometimes it feels
like the tail is wagging the dog. (Of course I''m writing this to the
ZFS
list, so...) 

I use ZFS for fileservers, sure. The primary application though is an
in-house database using ZFS to tie together 120TB worth of JBODs and 2TB
Constellations via LSI HBAs to present filesystems to a bunch of
processes on the box doing intensive data analysis. ZFS is great, ZFS is
good, indeed, ZFS is one of the drivers for why this is using Solaris
and not CentOS... but it matters that Solaris be a decent all-round OS,
not a tuned fileserver appliance.

Sometimes I''m left wondering if anyone uses the non-Oracle versions for
anything but file storage... ? 

-bacon

On Wed, Mar 23, 2011 at 7:33 AM, Jeff Bacon <bacon at walleyesoftware.com>
wrote:
>> I''ve also started conversations with Pogo about offering an
> OpenIndiana
>> based workstation, which might be another option if you prefer more of
> Sometimes I''m left wondering if anyone uses the non-Oracle
versions for
> anything but file storage... ?
Seeing that userland programs for *Solaris and derivatives (GUI,
daemons, tools, etc) is usually late compared to bleeding-edge Linux
distros (e.g. Ubuntu), with no particular dedicated team working on
improvement there, I''m guessing the answer will be "highly
unlikely".

-- 
Fajar

On Mar 22, 2011, at 21:09, Fajar A. Nugraha wrote:
> Seeing that userland programs for *Solaris and derivatives (GUI,
> daemons, tools, etc) is usually late compared to bleeding-edge Linux
> distros (e.g. Ubuntu), with no particular dedicated team working on
> improvement there, I''m guessing the answer will be "highly
unlikely".
Pkgsrc works pretty well:

	http://en.wikipedia.org/wiki/Pkgsrc

As does Blastwave and Sunfreeware. I like a stable base to work off, and
that''s why I''m also a fan of the BSDs: a well-done OS, with
fairly easy  to add third-party software.

Having pre-packaged binaries is useful, but I find that either the churn is too
fast (Fedora, Ubuntu), or it''s stable and therefore no different that
having Solaris/BSD (RHEL-based stuff). There''s also the fact that the
pre-packaged stuff often pulls in dependencies I have no use for (e.g., Avahi,
D-Bus).

Pluses and minuses to both.

I can deal with installing more up-to-date packages if need be. There is no
substitute for ZFS, zones/jails, DTrace, etc.

On Sun, Mar 20, 2011 at 01:54:54PM +0700, Fajar A. Nugraha
wrote:
> On Sun, Mar 20, 2011 at 4:05 AM, Pawel Jakub Dawidek <pjd at
freebsd.org> wrote:
> > On Fri, Mar 18, 2011 at 06:22:01PM -0700, Garrett D''Amore
wrote:
> >> Newer versions of FreeBSD have newer ZFS code.
> >
> > Yes, we are at v28 at this point (the lastest open-source version).
> >
> >> That said, ZFS on FreeBSD is kind of a 2nd class citizen still.
[...]
> >
> > That''s actually not true. There are more FreeBSD committers
working on
> > ZFS than on UFS.
> 
> How is the performance of ZFS under FreeBSD? Is it comparable to that
> in Solaris, or still slower due to some needed compatibility layer?
This compatibility layer is just a bunch of ugly defines, etc. to allow
for less code modifications. It introduces no overhead.

I made performance comparison between FreeBSD 9 with ZFSv28 and Solaris
11 Express, but I don''t think Solaris license allows me to publish the
results. But believe me, the results were very surprising:)

-- 
Pawel Jakub Dawidek                       http://www.wheelsystems.com
FreeBSD committer                         http://www.FreeBSD.org
Am I Evil? Yes, I Am!                     http://yomoli.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20110323/9c00f298/attachment-0001.bin>

On 03/23/11 09:07 AM, Pawel Jakub Dawidek wrote:
> On Sun, Mar 20, 2011 at 01:54:54PM +0700, Fajar A. Nugraha wrote:
>> On Sun, Mar 20, 2011 at 4:05 AM, Pawel Jakub Dawidek<pjd at
freebsd.org>  wrote:
>>> On Fri, Mar 18, 2011 at 06:22:01PM -0700, Garrett D''Amore
wrote:
>>>> Newer versions of FreeBSD have newer ZFS code.
>>>
>>> Yes, we are at v28 at this point (the lastest open-source version).
>>>
>>>> That said, ZFS on FreeBSD is kind of a 2nd class citizen still.
[...]
>>>
>>> That''s actually not true. There are more FreeBSD
committers working on
>>> ZFS than on UFS.
>>
>> How is the performance of ZFS under FreeBSD? Is it comparable to that
>> in Solaris, or still slower due to some needed compatibility layer?
>
> This compatibility layer is just a bunch of ugly defines, etc. to allow
> for less code modifications. It introduces no overhead.
>
> I made performance comparison between FreeBSD 9 with ZFSv28 and Solaris
> 11 Express, but I don''t think Solaris license allows me to publish
the
> results. But believe me, the results were very surprising:)
You can compare OpenIndiana oi_148 (and oi148a with IllumOS) and publish 
comparisons.
I think site: Phoronix.com already did comparisons with ZFS under 
several platforms and other (Linux) file systems without sweat.

On Wed, Mar 23, 2011 at 3:50 PM, Nikola M. <minikola at gmail.com>
wrote:
> I think site: Phoronix.com already did comparisons with ZFS under several
> platforms and other (Linux) file systems without sweat.
with single disk configuration no less (er, more) ;)

You may want to check this instead: http://www.zfsbuild.com/

OpenIndiana and others (i.e. Benunix) are distributions that actively
support full desktop workstations based on the Illumos base.

It is true, that the storage server application is a popular one and so has
supporters both commercially and others. ZFS is amazing and quite rightly it
stands out, it works even better when used with zones, crossbow, dtrace,
etc. and so its obvious to see what it''s a focus and often seems the
only
priority. 

However is isn''t the only interest, by a long shot.

The SFE package repositories has many packages available to install for when
the binary packaging aren''t up to date. OpenIndiana is hard at work
trying
to build bigger binary repositories with more apps and newer versions.
A simple "pkg install APPLICATION" is the aim for the majority of main
applications.

Is it not moving fast enough, or missing the packages you need?
Well that''s the beauty of Open Source, we welcome and have systems to
help
newcomers add and update the packages and applications they want, so we all
benefit. 

Ultimately I''d (and I''m sure many would) like to have a level
of binary
repositories similar to Debian, with stable and faster changing place repos
and support for many different applications, however that requires a lot of
work and manpower.

Bye,
Deano

-----Original Message-----
From: zfs-discuss-bounces at opensolaris.org
[mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Fajar A. Nugraha
Sent: 23 March 2011 01:09
To: Jeff Bacon
Cc: zfs-discuss at opensolaris.org
Subject: Re: [zfs-discuss] best migration path from Solaris 10

On Wed, Mar 23, 2011 at 7:33 AM, Jeff Bacon <bacon at walleyesoftware.com>
wrote:
>> I''ve also started conversations with Pogo about offering an
> OpenIndiana
>> based workstation, which might be another option if you prefer more of
> Sometimes I''m left wondering if anyone uses the non-Oracle
versions for
> anything but file storage... ?
Seeing that userland programs for *Solaris and derivatives (GUI,
daemons, tools, etc) is usually late compared to bleeding-edge Linux
distros (e.g. Ubuntu), with no particular dedicated team working on
improvement there, I''m guessing the answer will be "highly
unlikely".

-- 
Fajar
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

On 3/23/2011 6:14 AM, Deano wrote:
> OpenIndiana and others (i.e. Benunix) are distributions that actively
> support full desktop workstations based on the Illumos base.
>
> It is true, that the storage server application is a popular one and so has
> supporters both commercially and others. ZFS is amazing and quite rightly
it
> stands out, it works even better when used with zones, crossbow, dtrace,
> etc. and so its obvious to see what it''s a focus and often seems
the only
> priority.
>
> However is isn''t the only interest, by a long shot.
>
> The SFE package repositories has many packages available to install for
when
> the binary packaging aren''t up to date. OpenIndiana is hard at
work trying
> to build bigger binary repositories with more apps and newer versions.
> A simple "pkg install APPLICATION" is the aim for the majority of
main
> applications.
>
> Is it not moving fast enough, or missing the packages you need?
> Well that''s the beauty of Open Source, we welcome and have systems
to help
> newcomers add and update the packages and applications they want, so we all
> benefit.
>
> Ultimately I''d (and I''m sure many would) like to have a
level of binary
> repositories similar to Debian, with stable and faster changing place repos
> and support for many different applications, however that requires a lot of
> work and manpower.
>
> Bye,
> Deano
Honestly (and I say this from purely personal preferences and bias, not 
any official statement), I see the long-term future of Solaris (and 
IllumOS-based distros) as the new engine for appliances, supplanting 
Linux and the *BSDs in that space.

For a lot of reasons, Solaris has a long list of very superior 
functionality that make is very appealing for appliance makers.  Right 
now, we see that in two areas:  ZFS for storage, and high scaleability 
for DBs (the various Oracle ExaData stuff).   I''m expecting to see a 
whole raft of things start to show up - JVM container systems (Run Your 
App Server in SUPERMAN MODE! ), online backup devices, firewall 
appliances, spam and mail filter systems, intrusion detection systems, 
maybe even software routers, etc...

It''s here that I think Solaris'' strengths can beat its
competitors, and
where its weaknesses aren''t significant.

Sadly, I think Solaris'' future as a general-purpose OS is likely
finished.

Of course, that''s just my reading of the tea leaves...

-- 
Erik Trimble
Java System Support
Mailstop:  usca22-123
Phone:  x17195
Santa Clara, CA

On Wed, Mar 23, 2011 at 9:27 AM, Erik Trimble <erik.trimble at oracle.com>
wrote:
> For a lot of reasons, Solaris has a long list of very superior
functionality
> that make is very appealing for appliance makers. ?Right now, we see that
in
> two areas: ?ZFS for storage, and high scaleability for DBs (the various
> Oracle ExaData stuff). ? I''m expecting to see a whole raft of
things start
> to show up - JVM container systems (Run Your App Server in SUPERMAN MODE!
),
> online backup devices, firewall appliances, spam and mail filter systems,
> intrusion detection systems, maybe even software routers, etc...
>
> It''s here that I think Solaris'' strengths can beat its
competitors, and
> where its weaknesses aren''t significant.
>
> Sadly, I think Solaris'' future as a general-purpose OS is likely
finished.
    It has been a long time since I thought that Solaris made a good
"Workstation", SunRays not withstanding. The JDS spin of Gnome was an
attempt to get back into the Workstation space, but IMHO was not
really a player. Solaris'' strengths have been on the "server"
side and
some of the very serious innovation in Solaris 10 really solidified
that position (ZFS, dtrace, SMF, FMD, etc.). With this as the starting
point, it is easy to see how packaging Solaris into an appliance is
appealing.

    While I am mostly a Solaris admin, my desktop runs Linux and has
for over 5 years. The strength of the desktop tools consistently
available on Linux as part of the distribution was what converted me
over. Back in 1996 I had a dual CPU SPARC20 running
OpenLook/OpenWindows as my desktop and it was fantastic, but times
change.

-- 
{--------1---------2---------3---------4---------5---------6---------7---------}
Paul Kraus
-> Senior Systems Architect, Garnet River ( http://www.garnetriver.com/ )
-> Sound Coordinator, Schenectady Light Opera Company (
http://www.sloctheater.org/ )
-> Technical Advisor, RPI Players

I think on this, the big question is going to be whether Oracle continues to
release ZFS updates under CDDL after their commercial releases.

Overall, in the past it has obviously and necessarily been the case that FreeBSD
has been a ''2nd class citizen''.

Moving forward, that 2nd class idea becomes very mutable - and ironically it
becomes more so in regards to dealing with organizations that have longevity.

Moving forward...

If Oracle continues to release critical ZFS feature sets under CDDL to the
community, then:

A) They are no longer pre-releasing those features to OpenSolaris
B) FreeBSD gets them at the same time.

If Oracle does not continue to release ZFS features sets under CDDL, then then
game changes.  Pick your choice of operating systems - one that has a history of
surviving for nearly two decades on its own with community support, or the
''green leaf off the dead tree'' that just decided to jump into
the willy-nilly world without direct/giant corporate support.

2nd class citizen issue for FreeBSD disappears either way.  

The only remaining question would be the remaining crufts of legal disposition. 
I could for instance see NetApp or somebody try and sue ixSystems, but I have a
really, really rough time seeing Oracle/LarryEllison suing the FreeBSD
foundation overall or something?

Oh yeah - plus BTRFS on the horizon?

Honestly - I am not here to start a flame war - I am asking these questions
because businesses both big and small need to know what to do.

My hunch is, we all have to wait and see if Oracle releases ZFS updates after
Solaris 11, and if so, whether that is a subset of functionality or full
functionality.

- mike


On Mar 19, 2011, at 11:54 PM, Fajar A. Nugraha wrote:
> On Sun, Mar 20, 2011 at 4:05 AM, Pawel Jakub Dawidek <pjd at
freebsd.org> wrote:
>> On Fri, Mar 18, 2011 at 06:22:01PM -0700, Garrett D''Amore
wrote:
>>> Newer versions of FreeBSD have newer ZFS code.
>> 
>> Yes, we are at v28 at this point (the lastest open-source version).
>> 
>>> That said, ZFS on FreeBSD is kind of a 2nd class citizen still.
[...]
>> 
>> That''s actually not true. There are more FreeBSD committers
working on
>> ZFS than on UFS.
> 
> How is the performance of ZFS under FreeBSD? Is it comparable to that
> in Solaris, or still slower due to some needed compatibility layer?
> 
> -- 
> Fajar
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

On Mar 24, 2011, at 02:03, Michael DeMan wrote:
> The only remaining question would be the remaining crufts of legal
disposition.  I could for instance see NetApp or somebody try and sue ixSystems,
but I have a really, really rough time seeing Oracle/LarryEllison suing the
FreeBSD foundation overall or something?
Last time I checked NetApp actually used FreeBSD under the hood. As do Isilon
and Juniper AFAIK.

That''s not to say they couldn''t, but from a PR perspective
it''s going to stir things up.

Michael DeMan <solaris at deman.com> wrote:
> Moving forward...
>
> If Oracle continues to release critical ZFS feature sets under CDDL to the
community, then:
>
> A) They are no longer pre-releasing those features to OpenSolaris
> B) FreeBSD gets them at the same time.
>
> If Oracle does not continue to release ZFS features sets under CDDL, then
then game changes.  Pick your choice of operating systems - one that has a
history of surviving for nearly two decades on its own with community support,
or the ''green leaf off the dead tree'' that just decided to
jump into the willy-nilly world without direct/giant corporate support.
>
> 2nd class citizen issue for FreeBSD disappears either way.  
If Oracle does not continue to publish up to date ZFS sources under CDDL, then 
it may even be that Orcale becomes a second class citizen in the future as it 
may be that there is more ZFS development in the community than it is inside 
Oracle.
> Oh yeah - plus BTRFS on the horizon?
BTRFS is to be discussed in 6 years, after BTRFS might have become mature.

J?rg

-- 
 EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
       js at cs.tu-berlin.de                (uni)  
       joerg.schilling at fokus.fraunhofer.de (work) Blog:
http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

On 3/21/2011 5:44 PM, Garrett D''Amore wrote:
> We do have support for running your own code using our API. Its just
>  that we can''t reasonably be expected to support people who want
do
> things like... oh, "zpool import -f"  (note the -f).  Or editing
> local configuration files that are also managed by the management
> software.
You wouldn''t have to worry about the latter in my case, as I''d
turn off
the management software and manage the configuration files automatically
along with the rest of my infrastructure ;).

The main additional components we would need to run would be apache with
mod_authz_fsacl, and a separate instance of apache with mod_perl that
provides the API our identity management infrastructure hooks into to
manage zfs file systems. We also replace syslog with syslog-ng, run
openntpd instead of xntpd, and run a variety of management tools such as
tenshi, aide, munin... Another project that''s coming up is going to
require the creation of a captive service account for a CMS to sftp
files into user and group web directories.

I dunno that your support guys would be happy with the relatively
extensive changes we would make to the default state of your appliance
:). And I wouldn''t particularly want to worry about having to get
approval to make changes when things come up that don''t fit into the
out-of-the-box experience. So unfortunately NexentaStor most likely
won''t fit our requirements; I kind of prefer a general-purpose
operating
system over an appliance anyway.
> NCP 4 will have the same fixes that OpenSolaris has.  I''d be
> interested to know which bugs are most annoying for this person --
> we have a variety of them fixed in NS 3.1, but have not yet
> resync''ed NCP 3 (something we will do when 3.1 ships).
He mentioned in passingan "NFSv4 OpenOwner lock problem" that
I''m
unfamiliar with, and a TCP/IP related kernel panic. He''s on the list,
so
I guess he could pipe in with more details if he chooses.

One thing that''s really biting me right now is the interaction between
NFS exclusive open, ACL''s, and mode bits. Due to a limitation in the
protocol, the initial open has a mode of 0, and then the intended
creation mode is separately set later with a setattr. So the object
inherits the correct ACL on the open, and then the equivalent of a chmod
is performed destroying it :(. Oracle most likely isn''t going to fix
it,
it''s been a known issue since January 2005 (CR6215088, initially with
UFS ACLs, also breaks ZFS ACLs),
and the ticket I opened about it was closed. Unfortunately they didn''t
take the opportunity to fix it in NFSv4, although it looks like it was
addressed in NFSv4.1. It seems like it should be possible to work around
it in NFSv4, if we end up going with an open source distribution
hopefully we can fix it ourselves. Or it would be solved as a side 
effect of aclmode=ignore.
> On a general basis, its hard to allocate engineers for ad-hoc
> projects like this mostly because I already have more work than I
> have engineers to perform the work.
>
> Oh, did I mention, we''re hiring? :-)
I wish you the best of luck in hiring sufficient engineers to be able to
offer support for NCP or OpenIndiana :)...


-- 
Paul B. Henson  |  (909) 979-6361  |  http://www.csupomona.edu/~henson/
Operating Systems and Network Analyst  |  henson at csupomona.edu
California State Polytechnic University  |  Pomona CA 91768

David Magda <dmagda at ee.ryerson.ca> wrote:
> On Mar 20, 2011, at 09:26, Joerg Schilling wrote:
>
> > The long term acceptance for ZFS depends on how Oracle will behave
past the
> > announced Solaris 11 is released. If they don''t Opensource
the related ZFS,
> > they will harm the future of ZFS. If they Opensource it again, there
is still a
> > problem with syncing the ZFS ve3rsions from the OSS OpenSolaris
continuation
> > projects.
>
> For a while Apple was considering it, and if Ellison and Jobs can come to
an agreement, it would certainly become very popular very quickly.
I am sure that there is still a chance to see ZFS in Mac OS X and Linux.

To make it really open, I would like to see "vendor" tagged feature
descriptors
as mentioned before.


J?rg

-- 
 EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
       js at cs.tu-berlin.de                (uni)  
       joerg.schilling at fokus.fraunhofer.de (work) Blog:
http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily