I have a Sun machine running Solaris 10, and a Vbox instance running Solaris 11 11/11. The vbox machine has a virtual disk pointing to /dev/disk1 (rawdisk), seen in sol11 as c0t2. If I create a zpool on the Sun s10 machine, on a USB stick, I can take that USB stick and access it through the vbox virtual disk. Just as expected. If I boot vbox from the s11 ISO, and install s11 onto USB stick (via the virtual device), I can boot the Sun machine from it, which puts up the grub menu but then fails to boot Solaris. There''s some kind of error which might not be making it to /SP/console, but after grub it seems to hang for a few seconds then reboot. The vbox happens to be running on Mac OS 10.6.x. This *should* work, yes? Any thoughts as to why it doesn''t? Not that this should matter, but on the vbox machine, sol11 sees the USB stick as a normal SATA hard drive, e.g. when I run ''format'' it is in the list of drives. On the Sun machine, it is seen as a removable drive by s10, e.g. I have to run ''format -e'' to see the drive. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20111121/c90bc5b6/attachment.html>
On Tue, Nov 22, 2011 at 11:26 AM, Frank Cusack <frank at linetwo.net> wrote:> I have a Sun machine running Solaris 10, and a Vbox instance running Solaris > 11 11/11.? The vbox machine has a virtual disk pointing to /dev/disk1 > (rawdisk), seen in sol11 as c0t2. > > If I create a zpool on the Sun s10 machine, on a USB stick, I can take that > USB stick and access it through the vbox virtual disk.? Just as expected. > > If I boot vbox from the s11 ISO, and install s11 onto USB stick (via the > virtual device), I can boot the Sun machine from it, which puts up the grub > menu but then fails to boot Solaris.? There''s some kind of error which might > not be making it to /SP/console, but after grub it seems to hang for a few > seconds then reboot. > > The vbox happens to be running on Mac OS 10.6.x. > > This *should* work, yes?? Any thoughts as to why it doesn''t? > > Not that this should matter, but on the vbox machine, sol11 sees the USB > stick as a normal SATA hard drive, e.g. when I run ''format'' it is in the > list of drives.? On the Sun machine, it is seen as a removable drive by s10, > e.g. I have to run ''format -e'' to see the drive.So basically the question is if you install solaris on one machine, can you move the disk (in this case the usb stick) to another machine and boot it there, right? The answer, as far as I know, is NO, you can''t. Of course, I could be wrong though (and in this case I''ll be happy if I''m wrong :D ). IIRC the only supported way to move (or clone) solaris installation is by using flash archive (flar), which (now) should also work on zfs. -- Fajar
On Mon, Nov 21, 2011 at 9:04 PM, Fajar A. Nugraha <work at fajar.net> wrote:> So basically the question is if you install solaris on one machine, > can you move the disk (in this case the usb stick) to another machine > and boot it there, right? >Yes, but one of the machines is a virtual machine. The answer, as far as I know, is NO, you can''t. Of course, I could be> wrong though (and in this case I''ll be happy if I''m wrong :D ). IIRC > the only supported way to move (or clone) solaris installation is by > using flash archive (flar), which (now) should also work on zfs. >If we ignore the vbox aspect of it, and assume real hardware with real devices, of course you can install on one x86 hardware and move the drive to boot on another x86 hardware. This is harder on SPARC (b/c hostid and zfs mount issues) but still possible. The weird thing here is that the install hardware is a virtual machine. One thing I know is odd is that the USB drive is seen to the virtual machine as a SATA drive but when moved to the real hardware it''s seen as a USB drive. There may be something else going on here that someone more familiar with vbox may know more about. Since this works seamlessly when the zpool in question is just a data pool, I''m wondering why it doesn''t work when it''s a boot drive. One thing I noticed is that when mounting it as a data drive, the real hardware sees the type of disk (between <...> in ''format'' output) as ATA-VBOX. Clearly that info must have been written when the pool was created on vbox, and maybe some hardware info was encoded that doesn''t match up when it''s booted as a real USB stick. This doesn''t to matter when it''s a data pool but maybe this is tripping it up during boot. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20111121/1d9a29a7/attachment.html>
On Tue, Nov 22, 2011 at 12:19 PM, Frank Cusack <frank at linetwo.net> wrote:> On Mon, Nov 21, 2011 at 9:04 PM, Fajar A. Nugraha <work at fajar.net> wrote: >> >> So basically the question is if you install solaris on one machine, >> can you move the disk (in this case the usb stick) to another machine >> and boot it there, right? > > Yes, but one of the machines is a virtual machine.It shouldn''t matter, really. As far as solaris (or any other is) is concerned, it''s just a different machine.> >> The answer, as far as I know, is NO, you can''t. Of course, I could be >> wrong though (and in this case I''ll be happy if I''m wrong :D ). IIRC >> the only supported way to move (or clone) solaris installation is by >> using flash archive (flar), which (now) should also work on zfs. > > > If we ignore the vbox aspect of it, and assume real hardware with real > devices, of course you can install on one x86 hardware and move the drive to > boot on another x86 hardware.? This is harder on SPARC (b/c hostid and zfs > mount issues) but still possible.Have you tried? :D IIRC there was a discussion about it (several years ago, I think), and the issue back then was that there might be some necessary device nodes not available when you simply move the disk around.> > The weird thing here is that the install hardware is a virtual machine.? One > thing I know is odd is that the USB drive is seen to the virtual machine as > a SATA driveThat''s how it works when you use rawdisk passthrough. Virtualbox does not have the necessary USB-boot support (yet). Think of it like you have a SATA disk with usb enclosure, but now you remove the enclosure and plug it directly to the onboard SATA controller.> but when moved to the real hardware it''s seen as a USB drive.... and that''s how it should be> There may be something else going on here that someone more familiar with > vbox may know more about. > > Since this works seamlessly when the zpool in question is just a data pool, > I''m wondering why it doesn''t work when it''s a boot drive.I have a hunch that it might be something related to grub. Trying something ... -- Fajar
On Mon, Nov 21, 2011 at 9:31 PM, Fajar A. Nugraha <work at fajar.net> wrote:> > On Tue, Nov 22, 2011 at 12:19 PM, Frank Cusack <frank at linetwo.net> wrote: > > > > If we ignore the vbox aspect of it, and assume real hardware with real > > devices, of course you can install on one x86 hardware and move the > drive to > > boot on another x86 hardware. This is harder on SPARC (b/c hostid and > zfs > > mount issues) but still possible. > > Have you tried? :D >Yes, I do this all the time. Between identical hardware, though. It used to be tricky when you had to know actual device paths and/or /dev/dsk/* names but with zfs that issue has gone away and it doesn''t matter if drives show up at different locations when moving the boot drive around. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20111121/1a1abef1/attachment-0001.html>
On Tue, Nov 22, 2011 at 12:53 PM, Frank Cusack <frank at linetwo.net> wrote:> On Mon, Nov 21, 2011 at 9:31 PM, Fajar A. Nugraha <work at fajar.net> wrote: >> >> On Tue, Nov 22, 2011 at 12:19 PM, Frank Cusack <frank at linetwo.net> wrote: >> > >> > If we ignore the vbox aspect of it, and assume real hardware with real >> > devices, of course you can install on one x86 hardware and move the >> > drive to >> > boot on another x86 hardware.? This is harder on SPARC (b/c hostid and >> > zfs >> > mount issues) but still possible. >> >> Have you tried? :D > > Yes, I do this all the time.? Between identical hardware, though.? It used > to be tricky when you had to know actual device paths and/or /dev/dsk/* > names but with zfs that issue has gone away and it doesn''t matter if drives > show up at different locations when moving the boot drive around. >Ah, you''re more experienced that I am then. In that case you might want to try: - boot with live CD on your sun box - plug your usb drive there - force-import then export your usb root pool (to eliminate any disk path or ID problem) - try boot from usb drive - if the above still doesn''t work, try running installgrub: http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide#ZFS_Root_Pool_and_Boot_Issues I''m still trying to install sol11 on USB, but it''s dreadfully slow on my system (not sure why) -- Fajar
On Mon, Nov 21, 2011 at 9:59 PM, Fajar A. Nugraha <work at fajar.net> wrote:> On Tue, Nov 22, 2011 at 12:53 PM, Frank Cusack <frank at linetwo.net> wrote: > > On Mon, Nov 21, 2011 at 9:31 PM, Fajar A. Nugraha <work at fajar.net> > wrote: > >> > >> On Tue, Nov 22, 2011 at 12:19 PM, Frank Cusack <frank at linetwo.net> > wrote: > >> > > >> > If we ignore the vbox aspect of it, and assume real hardware with real > >> > devices, of course you can install on one x86 hardware and move the > >> > drive to > >> > boot on another x86 hardware. This is harder on SPARC (b/c hostid and > >> > zfs > >> > mount issues) but still possible. > >> > >> Have you tried? :D > > > > Yes, I do this all the time. Between identical hardware, though. It > used > > to be tricky when you had to know actual device paths and/or /dev/dsk/* > > names but with zfs that issue has gone away and it doesn''t matter if > drives > > show up at different locations when moving the boot drive around. > > > > Ah, you''re more experienced that I am then. In that case you might want to > try: > - boot with live CD on your sun box > - plug your usb drive there > - force-import then export your usb root pool (to eliminate any disk > path or ID problem) >ah, good idea> - try boot from usb drive > - if the above still doesn''t work, try running installgrub: > > http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide#ZFS_Root_Pool_and_Boot_Issues >grub does need to have an idea of the device path, maybe in vbox it''s seen as the 3rd disk (c0t2), so the boot device name written to grub.conf is "disk3" (whatever the terminology for that is in grub-speak), but when I boot on the Sun hardware it is seen as "disk0" and this just doesn''t work. If it''s that easy that''d be awesome, all I need is an alternate grub entry. I''m still trying to install sol11 on USB, but it''s dreadfully slow on> my system (not sure why) >Same here, sustained write to a USB stick is painfully slow. Normal operation is fine though. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20111121/899ec65b/attachment.html>
Moving boot disks from one machine to another used to work as long as the machines were of the same architecture. I don''t recall if it was *supported* (and wouldn''t want to pretend to speak for Oracle now), but it was meant to work (unless you minimized the install and removed drivers not needed on the first system that are needed on the other system). You did have to do a reconfigure boot though! Nico --
On Mon, Nov 21, 2011 at 10:06 PM, Frank Cusack <frank at linetwo.net> wrote:> grub does need to have an idea of the device path, maybe in vbox it''s seen > as the 3rd disk (c0t2), so the boot device name written to grub.conf is > "disk3" (whatever the terminology for that is in grub-speak), but when I > boot on the Sun hardware it is seen as "disk0" and this just doesn''t work. > If it''s that easy that''d be awesome, all I need is an alternate grub entry. >Or maybe not. I guess this was findroot() in sol10 but in sol11 this seems to have gone away. Also, I was wrong about the disk target. When I do the install I configure the USB stick at disk0, seen by Solaris as c3t0, and no findroot() line gets written to menu.lst. Maybe it needs that line when it boots as a USB still on real hardware? I''ll try import/export and a reconfigure boot when I get a chance. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20111121/a4c1bef9/attachment.html>
2011-11-22 10:24, Frank Cusack ?????:> On Mon, Nov 21, 2011 at 10:06 PM, Frank Cusack <frank at linetwo.net > <mailto:frank at linetwo.net>> wrote: > > grub does need to have an idea of the device path, maybe in vbox > it''s seen as the 3rd disk (c0t2), so the boot device name written to > grub.conf is "disk3" (whatever the terminology for that is in > grub-speak), but when I boot on the Sun hardware it is seen as > "disk0" and this just doesn''t work. If it''s that easy that''d be > awesome, all I need is an alternate grub entry. >Regarding the disk numbering, GRUB and other x86 loaders assume that the current "BIOS boot disk" (one passed from BIOS as the boot device) is always number zero. Numbering of secondary drives may vary from boot to boot (i.e. if you boot from one or another disk of a mirrored root).> > Or maybe not. I guess this was findroot() in sol10 but in sol11 this > seems to have gone away.I haven''t used sol11 yet, so I can''t say for certain. But it is possible that the default boot (without findroot) would use the bootfs property of your root pool. At least that''s the way it worked in intermediate SXCE releases, where you had to set the whole bootfs path (zfs dataset name) in grub menu, or use the bootfs property.> > Also, I was wrong about the disk target. When I do the install I > configure the USB stick at disk0, seen by Solaris as c3t0, and no > findroot() line gets written to menu.lst. Maybe it needs that line when > it boots as a USB still on real hardware? > > I''ll try import/export and a reconfigure boot when I get a chance. >
On Tue, Nov 22, 2011 at 7:32 PM, Jim Klimov <jimklimov at cos.ru> wrote:>> Or maybe not. ?I guess this was findroot() in sol10 but in sol11 this >> seems to have gone away. > > I haven''t used sol11 yet, so I can''t say for certain. > But it is possible that the default boot (without findroot) > would use the bootfs property of your root pool.Nope. S11''s grub specifies bootfs for every stanza in menu.lst. bootfs pool property is no longer used. Anyway, after some testing, I found out you CAN use vbox-installed s11 usb stick on real notebook (enough hardware difference there). The trick is you have to import-export the pool on the system you''re going to boot the stick on. Meaning, you need to have S11 live cd/usb handy and boot with that first before booting using your disk. -- Fajar
I haven''t been able to get this working. To keep it simpler, next I am going to try usbcopy of the live USB image in the VM, and see if I can boot real hardware from the resultant live USB stick. On Tue, Nov 22, 2011 at 5:25 AM, Fajar A. Nugraha <list at fajar.net> wrote:> On Tue, Nov 22, 2011 at 7:32 PM, Jim Klimov <jimklimov at cos.ru> wrote: > >> Or maybe not. I guess this was findroot() in sol10 but in sol11 this > >> seems to have gone away. > > > > I haven''t used sol11 yet, so I can''t say for certain. > > But it is possible that the default boot (without findroot) > > would use the bootfs property of your root pool. > > Nope. > > S11''s grub specifies bootfs for every stanza in menu.lst. bootfs pool > property is no longer used. > > Anyway, after some testing, I found out you CAN use vbox-installed s11 > usb stick on real notebook (enough hardware difference there). The > trick is you have to import-export the pool on the system you''re going > to boot the stick on. Meaning, you need to have S11 live cd/usb handy > and boot with that first before booting using your disk. > > -- > Fajar > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20111129/e443fc22/attachment-0001.html>
On Wed, Nov 30, 2011 at 1:25 PM, Frank Cusack <frank at linetwo.net> wrote:> I haven''t been able to get this working.? To keep it simpler, next I am > going to try usbcopy of the live USB image in the VM, and see if I can boot > real hardware from the resultant live USB stick.To be clear, I''m talking about two things: (1) live USB, created from Live CD (2) solaris installed on USB This first one works on real hardware, but not on a VM. The cause is simple: seems like a boot code somewhere searches ONLY removable media for live solaris image. Since you need to map the USB disk as regular disk (SATA/IDE/SCSI) in a VM to be able to boot from it, you won''t be able to boot live usb on a VM. The second one works on both real hardare and VM, BUT with a prequisite that you have to export-import rpool first on that particular system. Unless you already have solaris installed, this usually means you need to boot with a live cd/usb first. I''m not sure what you mean by "usbcopy of the live USB image in the VM, and see if I can boot real hardware from the resultant live USB stick.". If you''re trying to create (1), it''d be simpler to just use live cd on real hardware, and if necessary create live usb there (MUCH faster than on a VM). If you mean (2), then it won''t work unless you boot with live cd/usb first. Oh and for reference, instead of usbcopy, I prefer using this method: http://blogs.oracle.com/jim/entry/how_to_create_a_usb -- Fajar> > On Tue, Nov 22, 2011 at 5:25 AM, Fajar A. Nugraha <list at fajar.net> wrote: >> >> On Tue, Nov 22, 2011 at 7:32 PM, Jim Klimov <jimklimov at cos.ru> wrote: >> >> Or maybe not. ?I guess this was findroot() in sol10 but in sol11 this >> >> seems to have gone away. >> > >> > I haven''t used sol11 yet, so I can''t say for certain. >> > But it is possible that the default boot (without findroot) >> > would use the bootfs property of your root pool. >> >> Nope. >> >> S11''s grub specifies bootfs for every stanza in menu.lst. bootfs pool >> property is no longer used. >> >> Anyway, after some testing, I found out you CAN use vbox-installed s11 >> usb stick on real notebook (enough hardware difference there). The >> trick is you have to import-export the pool on the system you''re going >> to boot the stick on. Meaning, you need to have S11 live cd/usb handy >> and boot with that first before booting using your disk. >> >> -- >> Fajar >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > >
On Tue, Nov 29, 2011 at 10:39 PM, Fajar A. Nugraha <list at fajar.net> wrote:> On Wed, Nov 30, 2011 at 1:25 PM, Frank Cusack <frank at linetwo.net> wrote: > > I haven''t been able to get this working. To keep it simpler, next I am > > going to try usbcopy of the live USB image in the VM, and see if I can > boot > > real hardware from the resultant live USB stick. > > To be clear, I''m talking about two things: > (1) live USB, created from Live CD > (2) solaris installed on USB >yup> > This first one works on real hardware, but not on a VM. The cause is > simple: seems like a boot code somewhere searches ONLY removable media > for live solaris image. Since you need to map the USB disk as regular > disk (SATA/IDE/SCSI) in a VM to be able to boot from it, you won''t be > able to boot live usb on a VM. >yup> > The second one works on both real hardare and VM, BUT with a > prequisite that you have to export-import rpool first on that > particular system. Unless you already have solaris installed, this > usually means you need to boot with a live cd/usb first. >yup. I didn''t quite do that, what I did is exit to shell after installing (from install CD) onto the USB. Then in the shell from the install CD I did the zpool export. The resultant USB is still unbootable for me on real hardware. During this install, the USB is seen as a SATA disk. I tried to install onto it as a pass through USB device, but a python script in the installer that tries to label the disk fails. This is likely because it has to invoke ''format -e'' instead of ''format'' in order to see the USB disk in the first place. When you invoke the ''label'' command, if you have invoked ''format'' as ''format -e'' you get prompted whether you want an SMI or EFI label. The python script doesn''t know about this and wants to just do ''y'' or ''n''. In S10, I have no problem installing on real hardware onto a USB stick (seen as USB), so I imagine this is just a deficiency of the new S11 installer. Anyway, the point of that story is that I tried to install onto it as as USB device, instead of as a SATA device, in case something special happens to make USB bootable that doesn''t happen when the S11 installer thinks it''s a SATA device. But I was unable to complete that test.> I''m not sure what you mean by "usbcopy of the live USB image in the > VM, and see if I can boot real hardware from the resultant live USB > stick.". If you''re trying to create (1), it''d be simpler to just use > live cd on real hardware, and if necessary create live usb there (MUCH > faster than on a VM). If you mean (2), then it won''t work unless you > boot with live cd/usb first. >I meant (1), because I think this is an easier case to try out than (2). (1) should DEFINITELY work, IMHO. I don''t use live cd on real hardware because that doesn''t meet my objective of being able to create a removable boot drive, created in a VM, that I can boot on real hardware if I wanted to. I mean, I *could* do it that way, but I want to be able to do this in a 100% VM environment.> > Oh and for reference, instead of usbcopy, I prefer using this method: > http://blogs.oracle.com/jim/entry/how_to_create_a_usb >Thanks, I''ll check it out.> > -- > Fajar > > > > > On Tue, Nov 22, 2011 at 5:25 AM, Fajar A. Nugraha <list at fajar.net> > wrote: > >> > >> On Tue, Nov 22, 2011 at 7:32 PM, Jim Klimov <jimklimov at cos.ru> wrote: > >> >> Or maybe not. I guess this was findroot() in sol10 but in sol11 this > >> >> seems to have gone away. > >> > > >> > I haven''t used sol11 yet, so I can''t say for certain. > >> > But it is possible that the default boot (without findroot) > >> > would use the bootfs property of your root pool. > >> > >> Nope. > >> > >> S11''s grub specifies bootfs for every stanza in menu.lst. bootfs pool > >> property is no longer used. > >> > >> Anyway, after some testing, I found out you CAN use vbox-installed s11 > >> usb stick on real notebook (enough hardware difference there). The > >> trick is you have to import-export the pool on the system you''re going > >> to boot the stick on. Meaning, you need to have S11 live cd/usb handy > >> and boot with that first before booting using your disk. > >> > >> -- > >> Fajar > >> _______________________________________________ > >> zfs-discuss mailing list > >> zfs-discuss at opensolaris.org > >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20111129/a33dbac8/attachment.html>
On Wed, Nov 30, 2011 at 2:35 PM, Frank Cusack <frank at linetwo.net> wrote:>> The second one works on both real hardare and VM, BUT with a >> prequisite that you have to export-import rpool first on that >> particular system. Unless you already have solaris installed, this >> usually means you need to boot with a live cd/usb first. > > > yup.? I didn''t quite do that, what I did is exit to shell after installing > (from install CD) onto the USB.? Then in the shell from the install CD I did > the zpool export.? The resultant USB is still unbootable for me on real > hardware.It won''t work unless you did export-import on the real hardware. Blame oracle for that. Even zfsonlinux can work without this hassle. ... then again your kind of use case is probably not the supported configuration anyway, and there''s no incentive for Oracle to "fix" it :)> > Anyway, the point of that story is that I tried to install onto it as as USB > device, instead of as a SATA device, in case something special happens to > make USB bootable that doesn''t happen when the S11 installer thinks it''s a > SATA device.? But I was unable to complete that test.Not sure about solaris, but in linux grub1 installation would fail in the BIOS does not list the disk as bootable. Virtualbox definitely does not support booting from passthru-usb, so that may be the source of your problem. Mapping it as SATA disk should work as expected.> I don''t use live cd on real hardware because that doesn''t meet my objective > of being able to create a removable boot drive, created in a VM, that I can > boot on real hardware if I wanted to.? I mean, I could do it that way, but I > want to be able to do this in a 100% VM environment.I use ubuntu for that, which works fine :D It also supports zfs (via zfsonlinux), albeit limited to pool version 28 (same as openindiana) -- Fajar