zfs discuss - Apr 2008 - Diverse, Dispersed, Distributed, Unscheduled RAID volumes

Hallo!

[Asked by a technical managerial new person assembling a  
recommendation (sorry for my relative ignorance) and apologies for  
the long English sentence]:

Is ZFS ready today to link a set of dispersed desktop computers  
(diverse operating systems) into a distributed RAID volume that  
supports desktops being turned on and off (with a power-down, "close- 
up" protocol but without a schedule), and assimilates new desktops  
very easily when they come online in such a manner that It would  
distribute the network load across all desktops for both reads and  
writes?

And could, say, such a set-up email the administrator if the array is  
in danger of losing integrity (due to too many desktops being taken  
offline), with the performance proviso that the number of machines  
needed to maintain integrity should scale as more data is added to  
the volume, and the volume(s) size should grow automatically as more  
machines are added?

Many thanks






-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080425/2fc12840/attachment.html>

No.  ZFS is not a distributed file system.
 -- richard

kilamanjaro wrote:
> Hallo!
>
> [Asked by a technical managerial new person assembling a 
> recommendation (sorry for my relative ignorance) and apologies for the 
> long English sentence]: 
>
> Is ZFS ready today to link a set of dispersed desktop computers 
> (diverse operating systems) into a distributed RAID volume that 
> supports desktops being turned on and off (with a power-down, 
> "close-up" protocol but without a schedule), and assimilates new 
> desktops very easily when they come online in such a manner that It 
> would distribute the network load across all desktops for both reads 
> and writes?  
>
> And could, say, such a set-up email the administrator if the array is 
> in danger of losing integrity (due to too many desktops being taken 
> offline), with the performance proviso that the number of machines 
> needed to maintain integrity should scale as more data is added to the 
> volume, and the volume(s) size should grow automatically as more 
> machines are added?
>
> Many thanks
>
>
>
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

On Fri, 25 Apr 2008, Richard Elling wrote:
> No.  ZFS is not a distributed file system.
While the results might not be pretty, if each PC exports a drive via 
iSCSI and mirroring is used with plenty of PCs in each mirror, it 
seems like it would "work" but with likely dismal performance if a PC 
was turned off (retries and 3+ minute iSCSI failure recovery logic). 
There would be additional dismal performance when the PC is turned 
back on due to cumulative resilvering.

Bob
=====================================Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

No :-)

The slightly more longer answer is that I don''t think ZFS is what
you''re looking for, I believe there are other projects better suited
(although I don''t recall the names off the top of my head).

For starters, ZFS only offers at best dual parity raid, so if more than two
computers went offline at once you could have problems.  I guess
there''s potential to just use multiple mirrors or copies of data, but
I''ve no idea what performance would be like for that.

Then there''s the question of how you connect the computers up.  You
could use iSCSI, but my experience is that ZFS freezes for a good while when a
device is disconnected.  I''m also not aware of any way to automatically
reconnect or manage devices.

Then you have to consider the network overhead from ZFS re-syncing data if a
computer goes offline and a ''hot spare'' computer replaces it. 
ZFS I believe will just go flat out, potentially slowing your network.

You may be able to bodge ZFS into a solution for this, but I don''t
think it''s even ready for that yet, and there are in all likelyhood far
better alternatives.
 
 
This message posted from opensolaris.org

Hi,
this is true, but it might still be possible to use zfs in a  
distributed setup, as you can build pools from plain files that may be  
located anywhere on a network.
Automatic status notification would require some custom scripting and  
it is obviously not recommended by sun ;-)
The other drawback is that you can only use it as a single file- 
server, i.e. you have a performance bottleneck.
I have tested thi setup as a secondary backup pool and it worked well  
(albeit slow) for me.

ralf
Am 25.04.2008 um 16:42 schrieb zfs-discuss-request at opensolaris.org:
> Subject: Re: [zfs-discuss] Diverse, Dispersed, Distributed,
> 	Unscheduled RAID volumes
> To: kilamanjaro <kilamanjaro at web.de>
> Cc: zfs-discuss at opensolaris.org
> Message-ID: <4811EDA7.7080609 at sun.com>
> Content-Type: text/plain; format=flowed; charset=ISO-8859-1
>
> No.  ZFS is not a distributed file system.
> -- richard

On Fri, Apr 25, 2008 at 4:48 AM, kilamanjaro <kilamanjaro at web.de>
wrote:
> Is ZFS ready today to link a set of dispersed desktop computers (diverse
> operating systems) into a distributed RAID volume that supports desktops
It sounds like you''d want to use something like Lustre or Hadoop, both
of which are only supported on Linux.

I remember there being an application in the Windows 95/98 timeframe
that did what you want, but do idea on what it was called, how well it
worked, or if it still exists.

-B

-- 
Brandon High bhigh at freaks.com
"The good is the enemy of the best." - Nietzsche

Brandon High wrote:
> On Fri, Apr 25, 2008 at 4:48 AM, kilamanjaro <kilamanjaro at web.de>
wrote:
>   
>> Is ZFS ready today to link a set of dispersed desktop computers
(diverse
>> operating systems) into a distributed RAID volume that supports
desktops
>>     
>
> It sounds like you''d want to use something like Lustre or Hadoop,
both
> of which are only supported on Linux.
>
> I remember there being an application in the Windows 95/98 timeframe
> that did what you want, but do idea on what it was called, how well it
> worked, or if it still exists.
>
> -B
>
>   
I remember that Win95/98 tool as well.  It was advertised in the product 
catalogs of the day (~1996/1997/1998) like TigerDirect or PCConnection, 
etc.  It let you create a shared storage drive that used space on 
multiple computers on the network, without needing all the PCs to be on 
at the same time.  FWIW, the Google File System uses a policy of simply 
keeping 3 copies of everything.  The suggested scenario should 
definitely use multiple drive mirrors vdevs, and if they''re done as 
files on CIFS/SMB shares, that iscsi delay mentioned in this thread may 
not occur.

-hk.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080425/21e6a401/attachment.html>

Brandon High wrote:
> On Fri, Apr 25, 2008 at 4:48 AM, kilamanjaro <kilamanjaro at web.de>
wrote:
>   
>> Is ZFS ready today to link a set of dispersed desktop computers
(diverse
>> operating systems) into a distributed RAID volume that supports
desktops
>>     
>
> It sounds like you''d want to use something like Lustre or Hadoop,
both
> of which are only supported on Linux.
>
> I remember there being an application in the Windows 95/98 timeframe
> that did what you want, but do idea on what it was called, how well it
> worked, or if it still exists.
>
> -B
>
>   
I did some searching.and found the product you and I were thinking of.  
It was called Medley97 and versions definitely existed for Windows 95 
and NT.  It let you pool storage from many desktops, and attempted to 
always keep at least two copies of something available online.

Here''s info on it and some other similar software (see the Hadoop 
compatibility and Lustre w/ZFS notes too):

Medley97
"Medley97 is a virtually zero-administration, plug-and-play network 
operating system that creates a pooled network drive and disk cache from 
unused disk space and free memory on workstations. Available: Now (well 
if you live in 1997, that is). $695 per server MangoSoft Corp. (888) 
88-MANGO; fax (508) 898-9166 info at mango.com or www.mango.com"
http://ask.slashdot.org/comments.pl?sid=447752&threshold=1&commentsort=0&mode=thread&cid=22367030
Medley97 takes distributed computing a step further
http://www.mangosoft.com/news/pa/pa_0002_-_INFOWorld_-_distributed_computing.asp
Mango''s Medley97 Achieves Best of COMDEX Honors
http://www.mangosoft.com/news/pr/19971124.asp
Mango pooling is the biggest idea we''ve seen since network computers
http://www.mangosoft.com/news/pa/pa_0009_-_infoworld_-_mango_pooling.asp
http://www.networkcomputing.com/902/902ff.html
MojoNation ... Corporate Backup Tool?
http://developers.slashdot.org/article.pl?no_d2=1&sid=02/07/18/0244256
"Mango Medley 97 did this 5 years ago"
http://developers.slashdot.org/comments.pl?sid=36257&threshold=1&commentsort=0&mode=thread&no_d2=1&cid=3907959

FreeLoader : Scavenged Distributed Storage System
http://www.ece.ubc.ca/~samera/projects/freeloader/
http://www.csm.ornl.gov/~vazhkuda/Morsels/

vanDisk: An Exploration in Peer-To-Peer Collaborative Back-Up Storage
http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/4232658/4232659/04232719.pdf?tp=&isnumber=&arnumber=4232719
http://www.ece.ubc.ca/~matei/496/OldProjects/2007.04-vanDisk-ArminBahramshahry.pdf

Making Use of Terabytes of Unused Storage
http://ask.slashdot.org/article.pl?sid=08/02/09/1319258

Hadoop
http://hadoop.apache.org/core/
http://wiki.apache.org/hadoop/ProjectDescription
1. What is Hadoop? 
Hadoop is a distributed computing platform written in Java. It 
incorporates features similar to those of the  Google File System and 
of  MapReduce. For some details, see HadoopMapReduce.
2. What platform does Hadoop run on?
Java 1.5.x or higher, preferably from Sun
Linux and Windows are the supported operating systems, but BSD and Mac 
OS/X are known to work. (Windows requires the installation of  Cygwin).

Lustre
http://wiki.lustre.org/index.php?title=Main_Page
"Lustre is a scalable, secure, robust, highly-available cluster file 
system. It is designed, developed and maintained by Sun Microsystems, Inc.
"
"Even before the acquisition, Sun declared its intentions to marry 
Lustre to its own ZFS file system to produce a general-purpose, 
high-capacity parallel file system solution. ZFS is Sun''s Solaris-based
file system for applications that require very large storage capacity. 
For true scalability, the only element missing was a clustering 
capability, which they now have in Lustre."
http://www.hpcwire.com/blogs/17903424.html
"Lustre 1.8 will allow users to choose between ZFS and ldiskfs as 
back-end storage."
http://en.wikipedia.org/wiki/Lustre_(file_system)#ZFS_integration
Lustre to run on ZFS (3/26/2008)
"The development teams hopes to get a version of the ZFS-compatible 
Lustre released by the end of the year. "
http://www.gcn.com/online/vol1_no1/46011-1.html

-hk

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080425/57363aca/attachment.html>