I am having a real hard time with gmirror. I recently bought two new 400G SATA disks and I want to mirror them. I think I am following the directions, but I am not sure. I generally do the following steps: edit /boot/loader.conf to add geom_mirror_load. reboot into single-user gmirror label -v -b round-robin gm0 ad4s1 ad6s1 bsdlabel -w mirror/gm0 bsdlabel -e mirror/gm0 newfs mirror/gm0a When I attempt to perform the newfs, it generally goes through all of the backup super blocks and hangs the system at the end of the newfs. At that point, my only recourse is pushing the reset button. When the system comes up, GEOM_MIRROR tries to rebuild one of the providers (usually ad4s1), but never seems to succeed. Anyway, the first problem I had was not having geom_mirror loaded. The only solution seems to be adding the load to /boot/loader.conf. None of the manual pages or handbook pages says anything about this except when discusssing making your system disk mirrored. I am not doing that. So, after loading the mirror stuff, I regularly lock up the system by trying to perform simple activities on the mirror. What do I need to do differently? Here are the relevant dmesg lines: atapci0: <SiI 3512 SATA150 controller> port 0xa000-0xa007,0x9800-0x9803,0x9400-0x9407,0x9000-0x9003,0x8800-0x880f mem 0xfba00000-0xfba001ff irq 18 at device 13.0 on pci0 ata2: <ATA channel 0> on atapci0 ata3: <ATA channel 1> on atapci0 ad4: 381554MB <Seagate ST3400620AS 3.AAC> at ata2-master SATA150 ad6: 381554MB <Seagate ST3400620AS 3.AAC> at ata3-master SATA150 FreeBSD zircon.zircon.seattle.wa.us 6.2-STABLE FreeBSD 6.2-STABLE #0: Sat Mar 24 16:32:06 PDT 2007 root@zircon.zircon.seattle.wa.us:/usr/obj/usr/src/sys/ZIRCON amd64 amd64 amd64 FreeBSD /Joe
Joe Kelsey wrote:> So, after loading the mirror stuff, I regularly lock up the system by > trying to perform simple activities on the mirror. What do I need to do > differently? > > Here are the relevant dmesg lines: > atapci0: <SiI 3512 SATA150 controller> port > 0xa000-0xa007,0x9800-0x9803,0x9400-0x9407,0x9000-0x9003,0x8800-0x880f > mem 0xfba00000-0xfba001ff irq 18 at device 13.0 on pci0I did almost the same thing you did with gmirror on 6.2-release on amd64 the other day and it worked. There were several complaints about "SiI" hardware in the past, though - you might want to search the lists. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 250 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20070325/86191339/signature.pgp
Ivan Voras wrote:> Joe Kelsey wrote: > > >> So, after loading the mirror stuff, I regularly lock up the system by >> trying to perform simple activities on the mirror. What do I need to do >> differently? >> >> Here are the relevant dmesg lines: >> atapci0: <SiI 3512 SATA150 controller> port >> 0xa000-0xa007,0x9800-0x9803,0x9400-0x9407,0x9000-0x9003,0x8800-0x880f >> mem 0xfba00000-0xfba001ff irq 18 at device 13.0 on pci0 >> > > I did almost the same thing you did with gmirror on 6.2-release on amd64 > the other day and it worked. There were several complaints about "SiI" > hardware in the past, though - you might want to search the lists. > >Thank you for the suggestion, but it does not help. There is some traffic on the list about the 3112, but I have a 3512, which does not have any list traffic about bugs. The major thing that needs doing is a detailed explanation of how to take two brand new disk drives and mirror them. Nothing in the documentation discusses this. Do you have to create file systems on the drives first? Do you have to use fdisk to slice them up? Is there a size limit on drives? I am trying to mirror two 400G drives, is this supported? There is no information anywhere that I can find about these topics. /Joe
On Mar 25, 2007, at 5:12 PM, Joe Kelsey wrote:> Ivan Voras wrote: >> Joe Kelsey wrote: >> >> >>> So, after loading the mirror stuff, I regularly lock up the >>> system by >>> trying to perform simple activities on the mirror. What do I >>> need to do >>> differently? >>> >>> Here are the relevant dmesg lines: >>> atapci0: <SiI 3512 SATA150 controller> port >>> 0xa000-0xa007,0x9800-0x9803,0x9400-0x9407,0x9000-0x9003,0x8800-0x880 >>> f >>> mem 0xfba00000-0xfba001ff irq 18 at device 13.0 on pci0 >>> >> >> I did almost the same thing you did with gmirror on 6.2-release on >> amd64 >> the other day and it worked. There were several complaints about >> "SiI" >> hardware in the past, though - you might want to search the lists. >> >> > Thank you for the suggestion, but it does not help. There is some > traffic on the list about the 3112, but I have a 3512, which does > not have any list traffic about bugs. > > The major thing that needs doing is a detailed explanation of how > to take two brand new disk drives and mirror them. Nothing in the > documentation discusses this. Do you have to create file systems > on the drives first? Do you have to use fdisk to slice them up? > Is there a size limit on drives? I am trying to mirror two 400G > drives, is this supported? There is no information anywhere that I > can find about these topics.Have you seen this: http://people.freebsd.org/~rse/mirror/ g.
Regarding the gmirror talk: I found this helpful, as well: http://www.onlamp.com/pub/a/bsd/2005/11/10/FreeBSD_Basics.html Has there been any talk of a "gmirror aware" sysinstall that would adjust the size of the disk layout by one sector to ensure that the metadata is not overwritten? (and perhaps make loader.conf and fstab changes) As I understand it now, the user has to manually account for this at OS install, and adjust the disk layout accordingly... yes?
On 12/23/-58 20:59, Joe Kelsey wrote:> I am having a real hard time with gmirror. > I recently bought two new 400G SATA disks and I want to mirror them. I > think I am following the directions, but I am not sure. > > I generally do the following steps: > > edit /boot/loader.conf to add geom_mirror_load. > reboot into single-user > gmirror label -v -b round-robin gm0 ad4s1 ad6s1 > bsdlabel -w mirror/gm0 > bsdlabel -e mirror/gm0 > newfs mirror/gm0aJoe, check what `bsdlabel mirror/gm0' gives right after editing the partitioning information and re-check that info after a reboot. There might be an issue... Check if bsdlabel complains about partition c not being sized well. Could you please try to `gmirror unload' before doing anything and a `gmirror load' right after labeling the mirror? Or just try a reboot after labeling the mirror. If that does make a difference, it's a sign I've been right with my last week's experience. If you're still getting into trouble, you may want to zero out the first sectors of the disk using `dd'. When creating a 400 gig mirror, it will take a long time for the mirror to complete and you're mirroring waste in the first step. I would recommend to create the mirror with just one consumer, create the partitions, newfs and then insert the 2nd consumer into the mirror. Just to make sure, I would check the status of the mirror using `gmirror status' and wait until its state is COMPLETE before doing anything else. BTW, do you really want to use a dangerously dedicated disk (by using gm0X) without any slices? HTH, Volker
On Sun, 2007-03-25 at 13:53 -0700, Joe Kelsey wrote:> I am having a real hard time with gmirror. I recently bought two new > 400G SATA disks and I want to mirror them. I think I am following the > directions, but I am not sure. >I have seen lockups, spontaneous reboots, corrupt filsystems on 6.0 and 6.2 trigger: 1.a disk io with rsync / scp > say coping > 100Gb 1.b making backup with tar 2. full filesystems setup: -ata controller card -two ata disks (250Gb) system is now in via itxboard. but was the same in inter p2 board I 've tested mem with memtest I 've replaced ata controller card I 've replaced disk If I can provide more debug info pls let me know. kind regards, Marten dmesg %cat /var/run/dmesg.boot Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 6.2-STABLE #5: Sat Mar 3 10:47:32 CET 2007 root@martenvijn.nl:/usr/obj/usr/src/sys/henk Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: VIA C3 Samuel 2 (597.07-MHz 686-class CPU) Origin = "CentaurHauls" Id = 0x673 Stepping = 3 Features=0x803035<FPU,DE,TSC,MSR,MTRR,PGE,MMX> real memory = 201261056 (191 MB) avail memory = 187387904 (178 MB) acpi0: <VT9174 AWRDACPI> on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 cpu0: <ACPI CPU> on acpi0 acpi_throttle0: <ACPI CPU Throttling> on cpu0 acpi_button0: <Power Button> on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 agp0: <VIA 862x (CLE266) host to PCI bridge> mem 0xc0000000-0xcfffffff at device 0.0 on pci0 pcib1: <PCI-PCI bridge> at device 1.0 on pci0 pci1: <PCI bus> on pcib1 pci1: <display, VGA> at device 0.0 (no driver attached) pci0: <bridge, PCI-CardBus> at device 10.0 (no driver attached) pci0: <bridge, PCI-CardBus> at device 10.1 (no driver attached) uhci0: <VIA 83C572 USB controller> port 0xc000-0xc01f irq 5 at device 16.0 on pci0 uhci0: [GIANT-LOCKED] usb0: <VIA 83C572 USB controller> on uhci0 usb0: USB revision 1.0 uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: <VIA 83C572 USB controller> port 0xc400-0xc41f irq 12 at device 16.1 on pci0 uhci1: [GIANT-LOCKED] usb1: <VIA 83C572 USB controller> on uhci1 usb1: USB revision 1.0 uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: <VIA 83C572 USB controller> port 0xc800-0xc81f irq 9 at device 16.2 on pci0 uhci2: [GIANT-LOCKED] usb2: <VIA 83C572 USB controller> on uhci2 usb2: USB revision 1.0 uhub2: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered ehci0: <VIA VT6202 USB 2.0 controller> mem 0xd700c000-0xd700c0ff irq 12 at device 16.3 on pci0 ehci0: [GIANT-LOCKED] usb3: EHCI version 1.0 usb3: companion controllers, 2 ports each: usb0 usb1 usb2 usb3: <VIA VT6202 USB 2.0 controller> on ehci0 usb3: USB revision 2.0 uhub3: VIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub3: 6 ports with 6 removable, self powered isab0: <PCI-ISA bridge> at device 17.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <VIA 8235 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xcc00-0xcc0f at device 17.1 on pci0 ata0: <ATA channel 0> on atapci0 ata1: <ATA channel 1> on atapci0 pci0: <multimedia, audio> at device 17.5 (no driver attached) vr0: <VIA VT6102 Rhine II 10/100BaseTX> port 0xd800-0xd8ff mem 0xd700d000-0xd700d0ff irq 5 at device 18.0 on pci0 miibus0: <MII bus> on vr0 ukphy0: <Generic IEEE 802.3u media interface> on miibus0 ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto vr0: Ethernet address: 00:40:63:d5:e5:f5 atapci1: <Promise PDC20269 UDMA133 controller> port 0xdc00-0xdc07,0xe000-0xe003,0xe400-0xe407,0xe800-0xe803,0xec00-0xec0f mem 0xd7000000-0xd7003fff irq 12 at device 20.0 on pci0 ata2: <ATA channel 0> on atapci1 ata3: <ATA channel 1> on atapci1 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] pmtimer0 on isa0 orm0: <ISA Option ROMs> at iomem 0xc0000-0xcdfff,0xd8000-0xda7ff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 597071185 Hz quality 800 Timecounters tick every 1.000 msec ad4: 238475MB <Seagate ST3250823A 3.02> at ata2-master UDMA100 GEOM_MIRROR: Device gm0 created (id=222343067). GEOM_MIRROR: Device gm0: provider ad4 detected. ad6: 238475MB <WDC WD2500JB-00GVA0 08.02D08> at ata3-master UDMA100 GEOM_MIRROR: Device gm0: provider ad6 detected. GEOM_MIRROR: Device gm0: provider ad6 activated. GEOM_MIRROR: Device gm0: provider ad4 activated. GEOM_MIRROR: Device gm0: provider mirror/gm0 launched. Trying to mount root from ufs:/dev/mirror/gm0s1a %pciconf -lvv agp0@pci0:0:0: class=0x060000 card=0xaa011106 chip=0x31231106 rev=0x00 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT8623 Apollo CLE266 CastleRock CPU to PCI Bridge' class = bridge subclass = HOST-PCI pcib1@pci0:1:0: class=0x060400 card=0x00000000 chip=0xb0911106 rev=0x00 hdr=0x01 vendor = 'VIA Technologies Inc' device = 'VT8633 Apollo Pro 266 CPU to AGP Controller' class = bridge subclass = PCI-PCI none0@pci0:10:0: class=0x060700 card=0xaa011106 chip=0x04761180 rev=0x80 hdr=0x02 vendor = 'Ricoh Co Ltd' device = 'RL5c476 CardBus Controller' class = bridge subclass = PCI-CardBus none1@pci0:10:1: class=0x060700 card=0xaa011106 chip=0x04761180 rev=0x80 hdr=0x02 vendor = 'Ricoh Co Ltd' device = 'RL5c476 CardBus Controller' class = bridge subclass = PCI-CardBus uhci0@pci0:16:0: class=0x0c0300 card=0xaa011106 chip=0x30381106 rev=0x80 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT82xxxxx UHCI USB 1.1 Controller (All VIA Chipsets)' class = serial bus subclass = USB uhci1@pci0:16:1: class=0x0c0300 card=0xaa011106 chip=0x30381106 rev=0x80 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT82xxxxx UHCI USB 1.1 Controller (All VIA Chipsets)' class = serial bus subclass = USB uhci2@pci0:16:2: class=0x0c0300 card=0xaa011106 chip=0x30381106 rev=0x80 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT82xxxxx UHCI USB 1.1 Controller (All VIA Chipsets)' class = serial bus subclass = USB ehci0@pci0:16:3: class=0x0c0320 card=0x31041106 chip=0x31041106 rev=0x82 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT6202 USB 2.0 Enhanced Host Controller' class = serial bus subclass = USB isab0@pci0:17:0: class=0x060100 card=0xaa011106 chip=0x31771106 rev=0x00 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT8235 PCI to ISA Bridge' class = bridge subclass = PCI-ISA atapci0@pci0:17:1: class=0x01018a card=0xaa011106 chip=0x05711106 rev=0x06 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT82xxxx EIDE Controller (All VIA Chipsets)' class = mass storage subclass = ATA none2@pci0:17:5: class=0x040100 card=0xaa011106 chip=0x30591106 rev=0x50 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT8233/33A/8235/8237 AC97 Enhanced Audio Controller' class = multimedia subclass = audio vr0@pci0:18:0: class=0x020000 card=0x01021106 chip=0x30651106 rev=0x74 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT6102 Rhine II PCI Fast Ethernet Controller' class = network subclass = ethernet atapci1@pci0:20:0: class=0x018085 card=0x4d68105a chip=0x4d69105a rev=0x02 hdr=0x00 vendor = 'Promise Technology Inc' device = 'PDC20269 Ultra133 TX2 EIDE Controller' class = mass storage none3@pci1:0:0: class=0x030000 card=0x31221106 chip=0x31221106 rev=0x03 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT8623 Apollo CLE266 CastleRock AGP 8X Controller' class = display subclass = VGA
On Mar 26, 2007, at 11:19 AM, Marten wrote:> On Sun, 2007-03-25 at 13:53 -0700, Joe Kelsey wrote: >> I am having a real hard time with gmirror. I recently bought two new >> 400G SATA disks and I want to mirror them. I think I am following >> the >> directions, but I am not sure. >> > I have seen lockups, spontaneous reboots, corrupt filsystems on > 6.0 and > 6.2 > > trigger: > > 1.a disk io with rsync / scp > say coping > 100Gb > 1.b making backup with tar > 2. full filesystems > > setup: > -ata controller card > -two ata disks (250Gb)Try hunting around for the latest VIA test BIOS for the ITX M-series miniboards (check the viaarea.com web-forums, IIRC)...before updating to it, I tended to see fairly reproducible data corruption whenever accessing two or more IDE devices at the same time on a 600MHz VIA C3 ITX system. -- -Chuck
Joe Kelsey wrote:> I am having a real hard time with gmirror. I recently bought two new > 400G SATA disks and I want to mirror them. I think I am following the > directions, but I am not sure. > > I generally do the following steps: > > edit /boot/loader.conf to add geom_mirror_load. > reboot into single-user > gmirror label -v -b round-robin gm0 ad4s1 ad6s1 > bsdlabel -w mirror/gm0 > bsdlabel -e mirror/gm0 > newfs mirror/gm0a > > When I attempt to perform the newfs, it generally goes through all of > the backup super blocks and hangs the system at the end of the newfs. > At that point, my only recourse is pushing the reset button. When the > system comes up, GEOM_MIRROR tries to rebuild one of the providers > (usually ad4s1), but never seems to succeed. > > Anyway, the first problem I had was not having geom_mirror loaded. > The only solution seems to be adding the load to /boot/loader.conf. > None of the manual pages or handbook pages says anything about this > except when discusssing making your system disk mirrored. I am not > doing that. > > So, after loading the mirror stuff, I regularly lock up the system by > trying to perform simple activities on the mirror. What do I need to > do differently? > > Here are the relevant dmesg lines: > atapci0: <SiI 3512 SATA150 controller> port > 0xa000-0xa007,0x9800-0x9803,0x9400-0x9407,0x9000-0x9003,0x8800-0x880f > mem 0xfba00000-0xfba001ff irq 18 at device 13.0 on pci0 > ata2: <ATA channel 0> on atapci0 > ata3: <ATA channel 1> on atapci0 > ad4: 381554MB <Seagate ST3400620AS 3.AAC> at ata2-master SATA150 > ad6: 381554MB <Seagate ST3400620AS 3.AAC> at ata3-master SATA150OK. I did some more testing and found a bug in the SII3512 controller. If I mount just one of the drives just as a single disk and attempt to move a 250G drive over using tar, sometime during the copying the system simply hangs. No errors, no messages, the system stops and my only recourse is jto press the reset button. For that reason, I have decided that the SII3512 is unreliable and will replace it with a Promise controller, basically the cheapest one (TX2). I hope it works better. Thank you everyone for the links to other pages. /Joe