Has there been any discussion here about the idea integrating a virtual IP into ZFS. It makes sense to me because of the integration of NFS and iSCSI with the sharenfs and shareiscsi properties. Since these are both dependent on an IP it would be pretty cool if there was also a virtual IP that would automatically move with the pool. Maybe something like "zfs set ip.nge0=x.x.x.x mypool" Or since we may have different interfaces on the nodes where we want to move the zpool... zfs set ip.server1.nge0=x.x.x.x mypool zfs set ip.server2.bge0=x.x.x.x mypool I know this could be handled with Sun Cluster but if I am only building a simple storage appliance to serve NFS and iSCSI along with CIFS via SAMBA then I don''t want or need the overhead and complexity of Sun Cluster. Anyone have comments about whether this is needed and worthwhile? Any good simple alternative ways to move a virtual IP with a zpool from one node to another? Regards, Vic This message posted from opensolaris.org
Vic Engle wrote:> Has there been any discussion here about the idea integrating a virtual IP into ZFS. It makes sense to me because of the integration of NFS and iSCSI with the sharenfs and shareiscsi properties. Since these are both dependent on an IP it would be pretty cool if there was also a virtual IP that would automatically move with the pool. > > Maybe something like "zfs set ip.nge0=x.x.x.x mypool" > > Or since we may have different interfaces on the nodes where we want to move the zpool... > > zfs set ip.server1.nge0=x.x.x.x mypool > zfs set ip.server2.bge0=x.x.x.x mypool > > I know this could be handled with Sun Cluster but if I am only building a simple storage appliance to serve NFS and iSCSI along with CIFS via SAMBA then I don''t want or need the overhead and complexity of Sun Cluster.Overhead? The complexity of a simple HA storage service is quite small. The complexity arises when you have multiple dependencies where various applications depend on local storage and other applications. (think SMF, but spread across multiple OSes). For a simple relationship such as storage--ZFS--share, there isn''t much complexity. Reinventing the infrastructure needed to manage access in the face of failures is a distinctly non-trivial task. You can even begin with a single node cluster, though a virtual IP on a single node cluster isn''t very interesting. -- richard
Well I suppose complexity is relative. Still, to use Sun Cluster at all I have to install the cluster framework on each node, correct? And even before that I have to install an interconnect with 2 switches unless I direct connect a simple 2 node cluster. My thinking was that ZFS seems to try and bundle all storage related tasks into 1 simple interface including making vfstab and dfstab entries unnecessary and considered legacy wrt ZFS. If I am using ZFS only to serve storage via IP then the only component I''m forced to manage outside of ZFS is the IP and if that''s really all I want then it does seem like overkill to install, configure and administer sun cluster framework on even 2 nodes. I''m not really thinking about an application where I really need sun cluster like availability. Just the convenience factor of being able to export a pool to another system if I need to do maintenance or patching or whatever without having to go configure the other system. As it is now, the only thing I might need to do is go bring the virtual IP on the system I import the pool to. A good example would be maybe a system where I keep jumpstart images. I really don''t need HA for it but simple administration is always a plus. It''s an easy enough task to script I suppose but it occurred to me that it would be very convenient to have this task builtin to ZFS. Regards, Vic On 6/15/07, Richard Elling <Richard.Elling at sun.com> wrote:> Vic Engle wrote: > > Has there been any discussion here about the idea integrating a virtual IP into ZFS. It makes sense to me because of the integration of NFS and iSCSI with the sharenfs and shareiscsi properties. Since these are both dependent on an IP it would be pretty cool if there was also a virtual IP that would automatically move with the pool. > > > > Maybe something like "zfs set ip.nge0=x.x.x.x mypool" > > > > Or since we may have different interfaces on the nodes where we want to move the zpool... > > > > zfs set ip.server1.nge0=x.x.x.x mypool > > zfs set ip.server2.bge0=x.x.x.x mypool > > > > I know this could be handled with Sun Cluster but if I am only building a simple storage appliance to serve NFS and iSCSI along with CIFS via SAMBA then I don''t want or need the overhead and complexity of Sun Cluster. > > Overhead? > > The complexity of a simple HA storage service is quite small. > The complexity arises when you have multiple dependencies where > various applications depend on local storage and other applications. > (think SMF, but spread across multiple OSes). For a simple > relationship such as storage--ZFS--share, there isn''t much complexity. > > Reinventing the infrastructure needed to manage access in the > face of failures is a distinctly non-trivial task. You can > even begin with a single node cluster, though a virtual IP on a > single node cluster isn''t very interesting. > -- richard >
Victor Engle wrote:> Well I suppose complexity is relative. Still, to use Sun Cluster at > all I have to install the cluster framework on each node, correct? And > even before that I have to install an interconnect with 2 switches > unless I direct connect a simple 2 node cluster.Yes, rolling your own cluster software will not release you from these requirements. The only way to release these requirements is to increase the risk of data corruption.> My thinking was that ZFS seems to try and bundle all storage related > tasks into 1 simple interface including making vfstab and dfstab > entries unnecessary and considered legacy wrt ZFS. If I am using ZFS > only to serve storage via IP then the only component I''m forced to > manage outside of ZFS is the IP and if that''s really all I want then > it does seem like overkill to install, configure and administer sun > cluster framework on even 2 nodes.If you are considering manual failover, then this isn''t very difficult. For automated failover, you will need automated management of the services, which is what Solaris Cluster provides.> I''m not really thinking about an application where I really need sun > cluster like availability. Just the convenience factor of being able > to export a pool to another system if I need to do maintenance or > patching or whatever without having to go configure the other system. > As it is now, the only thing I might need to do is go bring the > virtual IP on the system I import the pool to. > > A good example would be maybe a system where I keep jumpstart images. > I really don''t need HA for it but simple administration is always a > plus. > > It''s an easy enough task to script I suppose but it occurred to me > that it would be very convenient to have this task builtin to ZFS.I don''t see where this is a problem in a manual case. You can have more than one IP address per NIC. So you could use a virtual IP address that "moves" to the active server without affecing the "fixed" IP addresses. But you can''t have both servers attempting to use the same IP address at the same time -- Solaris Cluster manages this task automatically. There are other nuances as well. Clients tend to react poorly if an IP address is active, but the service is not. For NFS, the behaviour is widely understood and Solaris Cluster will ensure that the server side tasks occur in proper order (eg. import storage before it is shared, resolve locks, etc.) For iSCSI, I''m not sure what the client behaviours are. In any case, you do not want your clients to restart/reboot/hang when you migrate the service between nodes. Bottom line: manual tasks can work around the nuances, for those who are interested in manual tasks. -- richard