Lets say I have two servers, both running opensolaris with ZFS. I basically want to be able to create a filesystem where the two servers have a common volume, that is mirrored between the two. Meaning, each server keeps an identical, real time backup of the other''s data directory. Set them both up as file servers, and load balance between the two for incoming requests. How would anyone suggest doing this? -- This message posted from opensolaris.org
On 05/ 9/10 10:07 AM, Tony wrote:> Lets say I have two servers, both running opensolaris with ZFS. I basically want to be able to create a filesystem where the two servers have a common volume, that is mirrored between the two. Meaning, each server keeps an identical, real time backup of the other''s data directory. Set them both up as file servers, and load balance between the two for incoming requests. > > How would anyone suggest doing this? >It sounds like you are looking for AVS. -- Ian.
On May 8, 2010, at 3:07 PM, Tony wrote:> Lets say I have two servers, both running opensolaris with ZFS. I basically want to be able to create a filesystem where the two servers have a common volume, that is mirrored between the two.In ZFS, file systems live in pools. Only one node can have a pool imported at any point in time.> Meaning, each server keeps an identical, real time backup of the other''s data directory.This can be done in several ways. Ian suggested AVS (Auto-CDP in NexentaStor terminology). Others offer the notion of using mirrored zvols exported through iSCSI. I prefer shared storage and DAS.> Set them both up as file servers, and load balance between the two for incoming requests.This is more difficult, but also rarely needed. Processors are much, much faster than disks, especially HDDs. So if you get to the point where you exceed the computational needs of a modern server (eg, more than 4x 10GbE) then you will also need a huge number of HDDs or even SSDs. OTOH, if you are just worried about the reliability of a server and need the ability to failover the service to another server automatically, then there are several HA cluster software solutions to consider: Oracle''s OpenStorage, Open HA Cluster, NexentaStor HA Cluster, RSF-1, etc.> How would anyone suggest doing this?Examine your requirements closely before you design the solution. -- richard -- ZFS storage and performance consulting at http://www.RichardElling.com
On 5/8/10 3:07 PM, Tony wrote:> Lets say I have two servers, both running opensolaris with ZFS. I basically want to be able to create a filesystem where the two servers have a common volume, that is mirrored between the two. Meaning, each server keeps an identical, real time backup of the other''s data directory. Set them both up as file servers, and load balance between the two for incoming requests. > > How would anyone suggest doing this? >I would carefully consider whether or not the _really_ need to be real time. Can you tolerate 5 minutes or even just 60 seconds of difference between them? If you can, then things are much easier and less complex. I''d personally use ZFS Snapshots to keep the two servers in sync every 60 seconds. As for load balancing, that depends on which protocal your using. FTP is easy. NFS/CIFS is a little harder. I''d simply use a load balancer (Zeus, NetScaler, Balance, HA-Proxy, etc.), but that is a little scary and bizarre in the case of NFS/CIFS, where you should instead use a single-server failover solution, such as Sun Cluster. benr.
Ok, this is definitely the kind of feedback I was looking for..... I''ll have to check out the docs on these technologies it looks like. Appreciate it. I figured I would load balance the hosts with a Cisco device, since I can get around the IOS ok. I want to offer a online backup service that provides high availability. Check out the following concept : http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ I like their basic idea, but they are in the market of $5/month unlimited. I want to take this to the next level. While this solution provides fault tolerance for drive failures, it does not seem to have a safeguard for if terrorists bomb one of your servers. So I figured ZFS would take care of the soft RAID, encryption (for compliance), dedup, and that kind of neat stuff. I''m still not 100% on how we''re going to give access to the storage. I had thought about using RSync or DFS/RDC for Win hosts. VPN for encrypted transfer if needed. Would avoid CIFS probably. I guess a couple minute delay in the replication is ok, if the backup management software is smart enough to say.... wtf.... get over it, and retransmit the file again without user intervention. -- This message posted from opensolaris.org
On May 8, 2010, at 7:01 PM, Tony wrote:> Ok, this is definitely the kind of feedback I was looking for..... I''ll have to check out the docs on these technologies it looks like. Appreciate it. > > I figured I would load balance the hosts with a Cisco device, since I can get around the IOS ok. > > I want to offer a online backup service that provides high availability.Yep, for this sort of business, a replication scheme works well. Note that the replication is better when it is done closer to the application. For a cloud storage company, it is relatively easy to modify the app to perform the redundancy and keep the storage simple. By contrast, if you are running a legacy app that you have no control over, you are stuck with their architecture and adding redundancy lower in the software stack is a necessary evil.> Check out the following concept : > http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/Syncing a box of this size could take a week or two. Consider triple redundancy.> I like their basic idea, but they are in the market of $5/month unlimited. I want to take this to the next level. While this solution provides fault tolerance for drive failures, it does not seem to have a safeguard for if terrorists bomb one of your servers. So I figured ZFS would take care of the soft RAID, encryption (for compliance), dedup, and that kind of neat stuff.Yep.> I''m still not 100% on how we''re going to give access to the storage. I had thought about using RSync or DFS/RDC for Win hosts. VPN for encrypted transfer if needed. Would avoid CIFS probably. > > I guess a couple minute delay in the replication is ok, if the backup management software is smart enough to say.... wtf.... get over it, and retransmit the file again without user intervention.If you put redundancy in the infeed, then you won''t have a delay like this. -- richard -- ZFS storage and performance consulting at http://www.RichardElling.com
>It sounds like you are looking for AVS.Consider a replication scenario where A is primary and B, secondary and A fails. Say you get A up again on Monday AM, but you are unable to summarily shut down B to bring A back online until Friday evening. During that whole time, you will not have a current mirror because AVS copies only in one direction from A to B.>If you can, then things are much easier and less complex. I''d >personally use ZFS Snapshots to keep the two servers in sync every 60 >seconds.I''ve never tested this myself, but if you are depending on the server to perform NFS, it has been said here, http://opensolaris.org/jive/thread.jspa?messageID=174846𪫾, that this will fail because the secondary''s filesystem will have a different FSID, which the NFS client won''t recognize. -- Maurice Volaski, maurice.volaski at einstein.yu.edu Computing Support, Rose F. Kennedy Center Albert Einstein College of Medicine of Yeshiva University