So, I''ve been stuck in kind of an ugly pattern. I zpool create and nothing goes wrong for a while, and then eventually I''ll zpool status, which doesn''t respond to ^C or kill -9s or anything. Also, setting NOINUSE_CHECK=1 doesn''t appear to make a difference. I''ll try and truss it next time I get a chance if that helps. Anywho, other problem is I get a huge storm of these around the same time zpool hangs. Jun 4 23:17:59 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/storage at 6/disk at 0,0 (sd5): Jun 4 23:17:59 cake offline or reservation conflict Jun 4 23:18:00 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/storage at 6/disk at 0,0 (sd5): Jun 4 23:18:00 cake offline or reservation conflict Jun 4 23:18:01 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/hub at 3/storage at 2/disk at 0,0 (sd6): Jun 4 23:18:01 cake offline or reservation conflict Jun 4 23:18:02 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/hub at 3/storage at 2/disk at 0,0 (sd6): Jun 4 23:18:02 cake offline or reservation conflict Jun 4 23:18:03 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/hub at 3/storage at 2/disk at 0,0 (sd6): Jun 4 23:18:03 cake offline or reservation conflict Jun 4 23:18:04 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/hub at 3/storage at 2/disk at 0,0 (sd6): Jun 4 23:18:04 cake offline or reservation conflict Jun 4 23:18:04 cake zfs: [ID 664491 kern.warning] WARNING: Pool ''tank'' has encountered an uncorrectable I/O error. Manual intervention is required. Sorry if this isn''t enough information, but if there''s anything else I can provide that''ll help please let me know. This message posted from opensolaris.org
For what it''s worth, I started playing with USB + flash + ZFS and was most unhappy for quite a while. I was suffering with things hanging, going slow or just going away and breaking, and thought I was witnessing something zfs was doing as I was trying to do mirror recovery and all that sort of stuff. On a hunch, I tried doing UFS and RAW instead and saw the same issues. It''s starting to look like my USB hubs. Once they are under any reasonable read/write load, they just make bunches of things go offline. Yep - They are powered and plugged in. So, at this stage, I''ll be grabbing a couple of ''better'' USB hubs (Mine are pretty much the cheapest I could buy) and see how that goes. For gags, take ZFS out of the equation and validate that your hardware is actually providing a stable platform for ZFS... Mine wasn''t... Nathan. Evan Geller wrote:> So, I''ve been stuck in kind of an ugly pattern. I zpool create and nothing goes wrong for a while, and then eventually I''ll zpool status, which doesn''t respond to ^C or kill -9s or anything. Also, setting NOINUSE_CHECK=1 doesn''t appear to make a difference. I''ll try and truss it next time I get a chance if that helps. > > Anywho, other problem is I get a huge storm of these around the same time zpool hangs. > > Jun 4 23:17:59 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/storage at 6/disk at 0,0 (sd5): > Jun 4 23:17:59 cake offline or reservation conflict > Jun 4 23:18:00 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/storage at 6/disk at 0,0 (sd5): > Jun 4 23:18:00 cake offline or reservation conflict > Jun 4 23:18:01 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/hub at 3/storage at 2/disk at 0,0 (sd6): > Jun 4 23:18:01 cake offline or reservation conflict > Jun 4 23:18:02 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/hub at 3/storage at 2/disk at 0,0 (sd6): > Jun 4 23:18:02 cake offline or reservation conflict > Jun 4 23:18:03 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/hub at 3/storage at 2/disk at 0,0 (sd6): > Jun 4 23:18:03 cake offline or reservation conflict > Jun 4 23:18:04 cake scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci8086,7270 at 1d,7/hub at 3/storage at 2/disk at 0,0 (sd6): > Jun 4 23:18:04 cake offline or reservation conflict > Jun 4 23:18:04 cake zfs: [ID 664491 kern.warning] WARNING: Pool ''tank'' has encountered an uncorrectable I/O error. Manual intervention is required. > > Sorry if this isn''t enough information, but if there''s anything else I can provide that''ll help please let me know. > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-- ////////////////////////////////////////////////////////////////// // Nathan Kroenert nathan.kroenert at aus.sun.com // // Technical Support Engineer Phone: +61 3 9869-6255 // // Sun Services Fax: +61 3 9869-6288 // // Level 3, 476 St. Kilda Road // // Melbourne 3004 Victoria Australia // //////////////////////////////////////////////////////////////////
Nathan Kroenert wrote:> For what it''s worth, I started playing with USB + flash + ZFS and was > most unhappy for quite a while. > > I was suffering with things hanging, going slow or just going away and > breaking, and thought I was witnessing something zfs was doing as I was > trying to do mirror recovery and all that sort of stuff. > > On a hunch, I tried doing UFS and RAW instead and saw the same issues. > > It''s starting to look like my USB hubs. Once they are under any > reasonable read/write load, they just make bunches of things go offline. > > Yep - They are powered and plugged in. > > So, at this stage, I''ll be grabbing a couple of ''better'' USB hubs (Mine > are pretty much the cheapest I could buy) and see how that goes. > > For gags, take ZFS out of the equation and validate that your hardware > is actually providing a stable platform for ZFS... Mine wasn''t...That''s my experience too. USB HUBs are cheap shit mostly...
> Nathan Kroenert wrote: > > For what it''s worth, I started playing with USB + > flash + ZFS and was > > most unhappy for quite a while. > > > > I was suffering with things hanging, going slow or > just going away and > > breaking, and thought I was witnessing something > zfs was doing as I was > > trying to do mirror recovery and all that sort of > stuff. > > > > On a hunch, I tried doing UFS and RAW instead and > saw the same issues. > > > > It''s starting to look like my USB hubs. Once they > are under any > > reasonable read/write load, they just make bunches > of things go offline. > > > > Yep - They are powered and plugged in. > > > > So, at this stage, I''ll be grabbing a couple of > ''better'' USB hubs (Mine > > are pretty much the cheapest I could buy) and see > how that goes. > > > > For gags, take ZFS out of the equation and validate > that your hardware > > is actually providing a stable platform for ZFS... > Mine wasn''t... > > That''s my experience too. USB HUBs are cheap [ expletive deleted ] > mostly...What do you expect? They''re mostly consumer-grade, which is to say garbage, rather than datacenter-grade. And it''s not just USB hubs - I''ve got a consumer-grade external modem, and I swear it must have little or no ECC and/or watchdog, because I have to power-cycle it every so often. Wish I had a lead box to put it in to shield it from the cosmic rays, maybe that would help... This message posted from opensolaris.org
Wow that fixed everything! I connected the hard drives directly to my computer and there were no problems. Go figure. I have some other computers I could use for my server that could take a few more USB cards, but they''re all < 1 GHz and < 1 gig of ram. What''s the minimum you''d run a 2.2 tb raid z on? Also, are there any companies that make good USB hubs? This message posted from opensolaris.org