zfs discuss - Oct 2010 - Ubuntu iSCSI install to COMSTAR zfs volume Howto

I apologize if this has been covered before.  I have not seen a blow-by-blow
installation guide for Ubuntu onto an iSCSI target.

The install guides I have seen assume that you can make a target visible to all,
which is a problem if you want multiple iSCSI installations on the same COMSTAR
target.  During install Ubuntu generates three random initiators and you have to
deal with them to get things working correctly.

I did this for a few reasons:
1. I have some PCs which already have another OS installed on them and want
Ubuntu available without any changes to the local drive
2. I want each PC to netboot Ubuntu with no interaction from the user and some
assurance that each machine will boot the correct image
3. It''s cool
4. Because I can

I am confident that there are things here which can be done better.  Any and all
feedback is appreciated.

Server is OpenSolaris build 128a at 192.168.223.147.
Client is Acer laptop with pxe boot enabled.
DHCP server is dd-wrt router with DHCP modifications

I have the following modifications made to the DHCP server.
===============================================================dhcp-match=gpxe,175
dhcp-option=175,8:1:1
dhcp-boot=net:#gpxe,gpxe-1.0.1-undionly.kpxe,v40z,192.168.223.147
dhcp-boot=net:gpxe,menu.gpxe,v40z,192.168.223.147
===============================================================
I have added the following files to /tftpboot.

* /tftp/gpxe-1.0.1-undionly.kpxe
This is available from www.etherboot.org

* /tftp/menu.pxe
This file is needed to get gpxe to do an iSCSI boot to a target using an
initiator based on the client uuid.  The contents of my file follow.

===============================================================#!gpxe

# initialize
dhcp net0

# keep our iSCSI mappings around even if the drive does not resolve
set keep-san 1

# set the initiator using our uuid
set initiator-iqn iqn.1993-08.org.debian:${uuid}

# set the target
set root-path
iscsi:192.168.223.147::::iqn.1986-03.com.sun:02:41fb1720-66ce-c72a-81fb-bbf396db7849

# try to boot from the iSCSI device
echo "Attempting to boot from san ${root-path}"
sanboot ${root-path}

# if we made it here, then boot failed, probably a new disk, chainload
# ubuntu installer

chain pxelinux.0

# for some reason, the silly system stalls and doesn''t bother to
chainload
===============================================================
* The Ubuntu Lucid netboot files, found at
http://archive.ubuntu.com/ubuntu/dists/lucid/main/installer-amd64/current/images/netboot/netboot.tar.gz

Just follow the 8 steps below, and you have a fully installed Ubuntu client on
iSCSI

STEP 1 -- Create sparse zfs volume on OpenSolaris

===============================================================bash-4.0$ pfexec
zfs create -s -V 320G tank/export/iscsi/acer-ubuntu
bash-4.0$ zfs get all tank/export/iscsi/acer-ubuntu
NAME                           PROPERTY               VALUE                 
SOURCE
tank/export/iscsi/acer-ubuntu  type                   volume                 -
tank/export/iscsi/acer-ubuntu  creation               Mon Oct 11 13:30 2010  -
tank/export/iscsi/acer-ubuntu  used                   54.5K                  -
tank/export/iscsi/acer-ubuntu  available              709G                   -
tank/export/iscsi/acer-ubuntu  referenced             54.5K                  -
tank/export/iscsi/acer-ubuntu  compressratio          1.00x                  -
tank/export/iscsi/acer-ubuntu  reservation            none                  
default
tank/export/iscsi/acer-ubuntu  volsize                320G                   -
tank/export/iscsi/acer-ubuntu  checksum               on                    
default
tank/export/iscsi/acer-ubuntu  compression            on                    
inherited from tank
tank/export/iscsi/acer-ubuntu  readonly               off                   
default
tank/export/iscsi/acer-ubuntu  shareiscsi             off                   
inherited from tank/export/iscsi
tank/export/iscsi/acer-ubuntu  copies                 1                     
default
tank/export/iscsi/acer-ubuntu  refreservation         none                  
default
tank/export/iscsi/acer-ubuntu  primarycache           all                   
default
tank/export/iscsi/acer-ubuntu  secondarycache         all                   
default
tank/export/iscsi/acer-ubuntu  usedbysnapshots        0                      -
tank/export/iscsi/acer-ubuntu  usedbydataset          54.5K                  -
tank/export/iscsi/acer-ubuntu  usedbychildren         0                      -
tank/export/iscsi/acer-ubuntu  usedbyrefreservation   0                      -
tank/export/iscsi/acer-ubuntu  logbias                latency               
default
tank/export/iscsi/acer-ubuntu  dedup                  off                   
default
tank/export/iscsi/acer-ubuntu  mlslabel               none                  
default
tank/export/iscsi/acer-ubuntu  com.sun:auto-snapshot  true                  
inherited from tank/export/iscsi
===============================================================
STEP 2 -- Create the logical unit

===============================================================bash-4.0$ pfexec
stmfadm create-lu /dev/zvol/rdsk/tank/export/iscsi/acer-ubuntu
Logical unit created: 600144F00800278C06EE4CB389310005
===============================================================
STEP 3 -- Create the host group, add initiator and add a view to lun

===============================================================bash-4.0$ pfexec
stmfadm create-hg acer-laptop-guid
bash-4.0$ pfexec stmfadm add-hg-member -g acer-laptop-guid
iqn.1993-08.org.debian:20a2531e-e791-d811-b971-001636bfa884
bash-4.0$ pfexec stmfadm add-view -n 0 -h acer-laptop-guid
600144F00800278C06EE4CB389310005
bash-4.0$ pfexec stmfadm list-view -l 600144F00800278C06EE4CB389310005
View Entry: 0
    Host group   : acer-laptop-guid
    Target group : All
    LUN          : 0
bash-4.0$ pfexec stmfadm list-hg -v acer-laptop-guid
Host Group: acer-laptop-guid
        Member: iqn.1993-08.org.debian:20a2531e-e791-d811-b971-001636bfa884
===============================================================
STEP 4 -- Start Ubuntu install
* Start network boot -- at *second* prompt to press Ctrl-B, do so and get a
prompt
* At the prompt, type the following command

===============================================================gPXE> chain
pxelinux.0
===============================================================
* wait for it to find the Ubuntu image and begin loading the default
configuration, which will present a menu screen
* Select "Install" by simply pressing enter
* Answer all of the prompts for location, keyboard, hostname, etc.
* Keep going until the installer prompts about partitioning disks

STEP 5 -- configure Ubuntu install iscsi initiator
* when prompted for disk partition information, select "Manual"
* Select "Configure iSCSI volumes"
* Select "Log into iSCSI targets"
* Enter the IP address of COMSTAR server, in my case, 192.168.223.147
* Leaver user name blank
* Select our target from the list.  My setup shows only one target to choose
from
* At this point, Ubunutu will have made up an initiator name which is logged in
but has no LUN available

===============================================================bash-4.0$ pfexec
stmfadm list-target -v
Target: iqn.1986-03.com.sun:02:41fb1720-66ce-c72a-81fb-bbf396db7849
    Operational Status: Online
    Provider Name     : iscsit
    Alias             : -
    Protocol          : iSCSI
    Sessions          : 1
        Initiator: iqn.1993-08.org.debian:01:bf55119a596a
            Alias: -
            Logged in since: Mon Oct 11 16:08:08 2010
===============================================================
* For now, we need to allow it to add it to our host group

===============================================================bash-4.0$ pfexec
stmfadm add-hg-member -g acer-laptop-guid iqn.1993-08.org.debian:01:bf55119a596a
bash-4.0$ pfexec stmfadm list-hg -v acer-laptop-guid
Host Group: acer-laptop-guid
        Member: iqn.1993-08.org.debian:20a2531e-e791-d811-b971-001636bfa884
        Member: iqn.1993-08.org.debian:01:bf55119a596a
===============================================================
* We need to log out Ubuntu and let it try again
* Press Alt-F2
* Press Enter to get a shell
* Type in the following into Ubuntu

===============================================================~ # iscsiadm -m
session -u
===============================================================
* Press Alt-F1 to go back to installer
* Select "Log into iSCSI targets" again fill out the information again
* Select "Finish" and you will see your drive show up
* Success!  Ubuntu shows drive in the list and COMSTAR shows the target attached

===============================================================bash-4.0$ pfexec
stmfadm list-target -v
Target: iqn.1986-03.com.sun:02:41fb1720-66ce-c72a-81fb-bbf396db7849
    Operational Status: Online
    Provider Name     : iscsit
    Alias             : -
    Protocol          : iSCSI
    Sessions          : 1
        Initiator: iqn.1993-08.org.debian:01:bf55119a596a
            Alias: -
            Logged in since: Mon Oct 11 16:13:25 2010
===============================================================
* Select the COMSTAR disk, making note of the disk name.  For me, it is always
"/dev/sdb"
* Allow Ubuntu to put a partition table on the drive
* Select the free space of the drive
* Select "Automatically partition the free space"
* Select "Finish partitioning and write changes to disk"
* Select "<Yes>"
* At this point, the installer has the drive and can install to it
* Continue the install by selecting all of the packages you want.  I just choose
"Ubuntu Desktop"

STEP 6 -- Finish the installation
* Continue installing until you get to the menu titled, "Configuring
grub-pc"
* This menu step will try to install grub onto the local drive and *NOT* the
iSCSI drive, so select "<No>"
* The next menu screen will ask for the device name to install grub to.  Enter
the disk name you noted in Step 5, "/dev/sdb"
* After some work, the installation will ask permission to restart.  Select
"<Continue>"
* After the installer shuts down, the initiator used by the installer is no
longer needed, so remove it

===============================================================bash-4.0$ pfexec
stmfadm remove-hg-member -g acer-laptop-guid
iqn.1993-08.org.debian:01:bf55119a596a
bash-4.0$ pfexec stmfadm list-hg -v acer-laptop-guid
Host Group: acer-laptop-guid
        Member: iqn.1993-08.org.debian:20a2531e-e791-d811-b971-001636bfa884
===============================================================
STEP 7 -- Reboot with failure
* The installer has picked yet another initiator name to use for booting and
will fail, dropping to an initramfs prompt
* For now, add the initiator name to the host group
* Get the new initiator name.  You can see it by querying COMSTAR

===============================================================bash-4.0$ pfexec
stmfadm list-target -v
Target: iqn.1986-03.com.sun:02:41fb1720-66ce-c72a-81fb-bbf396db7849
    Operational Status: Online
    Provider Name     : iscsit
    Alias             : -
    Protocol          : iSCSI
    Sessions          : 1
        Initiator: iqn.1993-08.org.debian:01:12b2b974a538
            Alias: -
            Logged in since: Mon Oct 11 17:00:41 2010
===============================================================
* add it to the group

===============================================================bash-4.0$ pfexec
stmfadm add-hg-member -g acer-laptop-guid iqn.1993-08.org.debian:01:12b2b974a538
bash-4.0$ pfexec stmfadm list-hg -v acer-laptop-guid
Host Group: acer-laptop-guid
        Member: iqn.1993-08.org.debian:20a2531e-e791-d811-b971-001636bfa884
        Member: iqn.1993-08.org.debian:01:12b2b974a538
===============================================================
* Type in "reboot" into Ubuntu machine and press Enter.

<<< If you are happy with two Ubuntu initiators on the target, you can
stop here.  For cleanliness sake, I prefer to fix the second name >>>

STEP 8 -- Fix the second initiator name
* Wait for reboot to finish and log in
* Open a terminal and edit the file /etc/iscsi/initiatorname.iscsi.  Ubuntu made
a third initiator name.  Mine reads as follows

===============================================================## DO NOT EDIT OR
REMOVE THIS FILE!
## If you remove this file, the iSCSI daemon will not start.
## If you change the InitiatorName, existing access control lists
## may reject this initiator.  The InitiatorName must be unique
## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.
InitiatorName=iqn.1993-08.org.debian:01:bf55119a596a
===============================================================
* Change the file to have the initiator name with the uuid

===============================================================## DO NOT EDIT OR
REMOVE THIS FILE!
## If you remove this file, the iSCSI daemon will not start.
## If you change the InitiatorName, existing access control lists
## may reject this initiator.  The InitiatorName must be unique
## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.
InitiatorName=iqn.1993-08.org.debian:20a2531e-e791-d811-b971-001636bfa884
===============================================================
* make a new initramfs on Ubuntu

===============================================================marty at
toni-pc:~$ sudo update-initramfs -u
update-initramfs: Generating /boot/initrd.img-2.6.32-25-generic-pae
===============================================================
* reboot Ubuntu
* remove old initiator from group

===============================================================bash-4.0$ pfexec
stmfadm remove-hg-member -g acer-laptop-guid
iqn.1993-08.org.debian:01:12b2b974a538
bash-4.0$ pfexec stmfadm list-hg -v acer-laptop-guid
Host Group: acer-laptop-guid
        Member: iqn.1993-08.org.debian:20a2531e-e791-d811-b971-001636bfa884
===============================================================
All should be well at this point.  All feedback appreciated.

Cheers,
Marty
-- 
This message posted from opensolaris.org