Hello, Does anyone have any suggestion on how to manage migrations, live or otherwise, with domains installed on iSCSI targets between different physical hosts? The problem I see is that you cannot be guaranteed that the device name that the iSCSI target gets mapped to on each system will be the same. In my test rig, for example, I see: on host1: /dev/rdsk/c2t010000144F40979800002A0048D97F70d0s2 on host2: /dev/rdsk/c4t010000144F40979800002A0048D97F70d0s2 (Notice the controller number differs.) I was hoping that devfsadm might have some (easy) way to force enforce uniformity, but so far I don''t see anything there. For now, I have worked around this with the use of symlinks to the devices which are then specified in the domain config rather than these dev links above. It isn''t horrible, but its another step in the process. Anyone doing migration with domains iSCSI? How are you doing it?
On Wed, Sep 24, 2008 at 12:45:31PM -0700, Joseph Mocker wrote:> The problem I see is that you cannot be guaranteed that the device name > that the iSCSI target gets mapped to on each system will be the same. In > my test rig, for example, I see: > > > on host1: /dev/rdsk/c2t010000144F40979800002A0048D97F70d0s2 > on host2: /dev/rdsk/c4t010000144F40979800002A0048D97F70d0s2 > > (Notice the controller number differs.) > > For now, I have worked around this with the use of symlinks to the > devices which are then specified in the domain config rather than these > dev links above.Seems like a reasonable work around. It would be nice to be able to open by a meaningful Device ID such as the Alias value, but iSCSI on Solaris falls far short here.> Anyone doing migration with domains iSCSI? How are you doing it?We are regularly. Sadly it''s by having the same disk setup across the hosts, so the controller number is matched. regards john
John Levon wrote:> On Wed, Sep 24, 2008 at 12:45:31PM -0700, Joseph Mocker wrote: > >> The problem I see is that you cannot be guaranteed that the device name >> that the iSCSI target gets mapped to on each system will be the same. In >> my test rig, for example, I see: >> >> >> on host1: /dev/rdsk/c2t010000144F40979800002A0048D97F70d0s2 >> on host2: /dev/rdsk/c4t010000144F40979800002A0048D97F70d0s2 >> >> (Notice the controller number differs.) >> >> For now, I have worked around this with the use of symlinks to the >> devices which are then specified in the domain config rather than these >> dev links above. > > Seems like a reasonable work around. > > It would be nice to be able to open by a meaningful Device ID such as > the Alias value, but iSCSI on Solaris falls far short here.Hi John, surely the ldi_open_by_devid function would be useful here? The devid really shouldn''t be changing, not even with iSCSI. James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog
Joseph Mocker wrote:> Hello, > > Does anyone have any suggestion on how to manage migrations, live or > otherwise, with domains installed on iSCSI targets between different > physical hosts? > > The problem I see is that you cannot be guaranteed that the device name > that the iSCSI target gets mapped to on each system will be the same. In > my test rig, for example, I see: > > > on host1: /dev/rdsk/c2t010000144F40979800002A0048D97F70d0s2 > on host2: /dev/rdsk/c4t010000144F40979800002A0048D97F70d0s2 > > (Notice the controller number differs.)I have some changes in a stale local gate where you can do something like the following.. disk = [ phy:iscsi:[static]/<targetid>/<lun>/[serverIpAddr] ] e.g. disk = [''phy:iscsi:static/iqn.1986-03.com.sun:02:17f34578-00a9-ef69-f3e9-b8a2896a4915/0/192.168.0.70,0,w''] if the mapping is static, the hotplug scripts will dynamically add the target to dom0 on guest boot and remove it on shutdown. I need to refresh the code and clean it up.. If anyone has any options or further enhancements in mind, I''d be interested in hearing them. MRJ> I was hoping that devfsadm might have some (easy) way to force enforce > uniformity, but so far I don''t see anything there. > > For now, I have worked around this with the use of symlinks to the > devices which are then specified in the domain config rather than these > dev links above. > > It isn''t horrible, but its another step in the process. > > Anyone doing migration with domains iSCSI? How are you doing it? > > > _______________________________________________ > xen-discuss mailing list > xen-discuss@opensolaris.org
Wow, that sounds slick! Check it in! Check it in! :-) I assume similar syntax would work for domains defined with virsh/virt-install ? --joe Mark Johnson wrote:> > > Joseph Mocker wrote: >> Hello, >> >> Does anyone have any suggestion on how to manage migrations, live or >> otherwise, with domains installed on iSCSI targets between different >> physical hosts? >> >> The problem I see is that you cannot be guaranteed that the device >> name that the iSCSI target gets mapped to on each system will be the >> same. In my test rig, for example, I see: >> >> on host1: /dev/rdsk/c2t010000144F40979800002A0048D97F70d0s2 >> on host2: /dev/rdsk/c4t010000144F40979800002A0048D97F70d0s2 >> >> (Notice the controller number differs.) > > I have some changes in a stale local gate where you can do something > like the following.. > disk = [ phy:iscsi:[static]/<targetid>/<lun>/[serverIpAddr] ] > > e.g. > disk = > [''phy:iscsi:static/iqn.1986-03.com.sun:02:17f34578-00a9-ef69-f3e9-b8a2896a4915/0/192.168.0.70,0,w''] > > > if the mapping is static, the hotplug scripts > will dynamically add the target to dom0 on guest > boot and remove it on shutdown. > > I need to refresh the code and clean it up.. If anyone > has any options or further enhancements in mind, I''d be > interested in hearing them. > > > > MRJ > > > >> I was hoping that devfsadm might have some (easy) way to force >> enforce uniformity, but so far I don''t see anything there. >> >> For now, I have worked around this with the use of symlinks to the >> devices which are then specified in the domain config rather than >> these dev links above. >> >> It isn''t horrible, but its another step in the process. >> >> Anyone doing migration with domains iSCSI? How are you doing it? >> >> >> _______________________________________________ >> xen-discuss mailing list >> xen-discuss@opensolaris.org
On Thu, Sep 25, 2008 at 11:24:15AM -0400, Mark Johnson wrote:> disk = [''phy:iscsi:static/iqn.1986-03.com.sun:02:17f34578-00a9-ef69-f3e9-b8a2896a4915/0/192.168.0.70,0,w'']Ah yes, I forgot about this discussion.> I need to refresh the code and clean it up.. If anyone > has any options or further enhancements in mind, I''d be > interested in hearing them.Only one: # iscsiadm list target Target: iqn.1986-03.com.sun:02:e78fb84f-89f2-c80d-df8f-8f7101d6a9f4 Alias: dskpool/dom/johnlev/nevada-pv/root TPGT: 1 ISID: 4000002a0000 Connections: 1 ... It''d be mega-cool to be able to use that ''Alias'' value directly. regards john
John Levon wrote:> Only one: > > # iscsiadm list target > Target: iqn.1986-03.com.sun:02:e78fb84f-89f2-c80d-df8f-8f7101d6a9f4 > Alias: dskpool/dom/johnlev/nevada-pv/root > TPGT: 1 > ISID: 4000002a0000 > Connections: 1 > ... > > It''d be mega-cool to be able to use that ''Alias'' value directly. >Yes, that would be excellent. With this change does one even need to run devfsadm to create the disk device ? --joe
Joseph Mocker wrote:
> Wow, that sounds slick! Check it in! Check it in!
>
> :-)
Ha. If it was only that easy :-)
> I assume similar syntax would work for domains defined with
> virsh/virt-install?
Yes. It should be something like...
virsh attach-disk nevada
"static/iqn.1986-03.com.sun:02:17f34578-00a9-ef69-f3e9-b8a2896a4915/0/192.168.0.70,0"
xvdb --driver phy --subdriver iscsi
Joseph Mocker wrote:> John Levon wrote:
>> Only one:
>>
>> # iscsiadm list target
>> Target: iqn.1986-03.com.sun:02:e78fb84f-89f2-c80d-df8f-8f7101d6a9f4
>> Alias: dskpool/dom/johnlev/nevada-pv/root
>> TPGT: 1
>> ISID: 4000002a0000
>> Connections: 1
>> ...
>>
>> It''d be mega-cool to be able to use that
''Alias'' value directly.
>>
> Yes, that would be excellent.
>
> With this change does one even need to run devfsadm to create the disk
> device ?
No, the hotplug scripts do it all for you...
MRJ