Hi folks; I've been running 4.x quite happily for some time, and am trying to set up a box with 5.3 that hopefully will end up being a viable platform to move over to. Tried on both 5.3-RELEASE and 5.3-STABLE (built this evening) Having some problems here... First, I discovered that vinum isn't "quite right" with 5.3. Ok, so I moved to "gvinum", but ran into some really bizarre behavior there too, where it was detaching subdisks apparently at random and on a rebuild would occasionally crash with some really wild results. It also doesn't load-balance reads (barf). All this was discovered after I spent an afternoon getting it set up properly with the "shadow" root and all. Bummer.... The instability is a real deal-breaker - this is supposed to (eventually) be a production server, so it can't be "squirrelly". So, my next attempt (with a new, fresh load) was to give gmirror a shot instead, since I only need Raid 1, and this looks like a "better" option from that perspective given gvinum's squirrely and not-all-there status at the moment. Weeeeellll.. the manual page (and the archives on the list here) implies that I can do the following: 1. Set up the system on one of the two mirror disks. 2. In single user mode, use "gmirror label -v -b round-robin disk ad4" to set up the pre-existing disk (with data on it), which I booted from, as the "primary". 3. Then use "gmirror insert disk ad6" to add the secondary (mirrored) disk, which will automatically sync it and bring them into a consistent state. Only one small problem - the first command (in #2) fails, with: "Can't store metadata on ad4: Operation not permitted" This is true if the system is mounted read-only (in single user mode) or read-write (in multiuser, with all filesystems mounted) Attempting to label the SECOND (bare) disk works, but of course that's backwards and an "insert" would immediately destroy the running system (I assume it would also fail with the same permission error, but I've not tried it) - so that's not an option. I also tried it from the fixit disk, but the KLD cannot be loaded from the fixit disk, and none of the commands work - so that appears to not be an option either. How do you get this thing set up? The archives here (and man pages) strongly imply that this works for a root filesystem and boot drive - what am I missing? Need a Raid 1 solution that works to move over to the 5.x world out here... -- -- Karl Denninger (karl@denninger.net) Internet Consultant & Kids Rights Activist http://www.denninger.net My home on the net - links to everything I do! http://scubaforum.org Your UNCENSORED place to talk about DIVING! http://www.spamcuda.net SPAM FREE mailboxes - FREE FOR A LIMITED TIME! http://genesis3.blogspot.com Musings Of A Sentient Mind
On Tuesday 18 January 2005 09:46 pm, Karl Denninger wrote:> Hi folks; > > I've been running 4.x quite happily for some time, and am trying to > set up a box with 5.3 that hopefully will end up being a viable > platform to move over to. > > Tried on both 5.3-RELEASE and 5.3-STABLE (built this evening) > > Having some problems here... > > First, I discovered that vinum isn't "quite right" with 5.3. Ok, so > I moved to "gvinum", but ran into some really bizarre behavior there > too, where it was detaching subdisks apparently at random and on a > rebuild would occasionally crash with some really wild results. It > also doesn't load-balance reads (barf). All this was discovered > after I spent an afternoon getting it set up properly with the > "shadow" root and all. Bummer.... > > The instability is a real deal-breaker - this is supposed to > (eventually) be a production server, so it can't be "squirrelly". > > So, my next attempt (with a new, fresh load) was to give gmirror a > shot instead, since I only need Raid 1, and this looks like a > "better" option from that perspective given gvinum's squirrely and > not-all-there status at the moment. > > Weeeeellll.. the manual page (and the archives on the list here) > implies that I can do the following: > > 1. Set up the system on one of the two mirror disks. > > 2. In single user mode, use "gmirror label -v -b round-robin disk > ad4" to set up the pre-existing disk (with data on it), which I > booted from, as the "primary". > > 3. Then use "gmirror insert disk ad6" to add the secondary (mirrored) > disk, which will automatically sync it and bring them into a > consistent state. > > Only one small problem - the first command (in #2) fails, with: > > "Can't store metadata on ad4: Operation not permitted" > > This is true if the system is mounted read-only (in single user mode) > or read-write (in multiuser, with all filesystems mounted) > > Attempting to label the SECOND (bare) disk works, but of course > that's backwards and an "insert" would immediately destroy the > running system (I assume it would also fail with the same permission > error, but I've not tried it) - so that's not an option. > > I also tried it from the fixit disk, but the KLD cannot be loaded > from the fixit disk, and none of the commands work - so that appears > to not be an option either. > > How do you get this thing set up? The archives here (and man pages) > strongly imply that this works for a root filesystem and boot drive - > what am I missing? > > Need a Raid 1 solution that works to move over to the 5.x world out > here...I had a pentium 166 with four matched ide drives set up as Striped plexes that had better disk access than my current 1ghz system. I sure miss that! The only time FreeBSD let me down when they let stopped supporting vinum. Guess no OS is perfect <sigh> -Mike
On 01/18/05 23:46, Karl Denninger wrote:> <snip backstory and gvinum stuff> > > So, my next attempt (with a new, fresh load) was to give gmirror a shot > instead, since I only need Raid 1, and this looks like a "better" option > from that perspective given gvinum's squirrely and not-all-there status at > the moment. > > Weeeeellll.. the manual page (and the archives on the list here) implies that > I can do the following: > > 1. Set up the system on one of the two mirror disks. > > 2. In single user mode, use "gmirror label -v -b round-robin disk ad4" > to set up the pre-existing disk (with data on it), which I booted from, > as the "primary". > > 3. Then use "gmirror insert disk ad6" to add the secondary (mirrored) disk, > which will automatically sync it and bring them into a consistent state. > > Only one small problem - the first command (in #2) fails, with: > > "Can't store metadata on ad4: Operation not permitted" > > This is true if the system is mounted read-only (in single user mode) or > read-write (in multiuser, with all filesystems mounted) > > Attempting to label the SECOND (bare) disk works, but of course that's > backwards and an "insert" would immediately destroy the running system (I > assume it would also fail with the same permission error, but I've not tried > it) - so that's not an option. > > I also tried it from the fixit disk, but the KLD cannot be loaded from the > fixit disk, and none of the commands work - so that appears to not be an > option either. > > How do you get this thing set up? The archives here (and man pages) > strongly imply that this works for a root filesystem and boot drive - > what am I missing? > > Need a Raid 1 solution that works to move over to the 5.x world out here...If any filesystem is mounted then GEOM will not allow metadata updates for the drive. Here's a solution given over on the geom@ list: http://lists.freebsd.org/pipermail/freebsd-geom/2004-November/000379.html I converted to gmirror from ataraid, so I already had 2 disks with identical contents. I worked around this problem as outlined here (the instructions have errors that are corrected later in the thread): http://lists.freebsd.org/pipermail/freebsd-current/2004-October/039907.html I wish we handled this in the installer, but as I can't produce the code to do it I'll shut up now... Jon