zfs discuss - Mar 2009 - Growing a zpool mirror breaks on Adaptec 1205sa PCI

Setup: Osol.11 build 109
Athlon64 3400+ Aopen AK-86L mobo
adeptec 1250sa Sata PCI controller card

[re-posted from accidental post to osol `general'' group]

I''m having trouble with an adaptec 1205sa (non-raid) SATA PCI card.

It was all working fine when I plugged 2 used sata 200gb disks of a
windows xp machine into it.  Booted my osol server and added a zpool
mirror using those 2 drives and the adaptec 1205sa SATA PCI card.

So I know it works.

On every boot I see a message like this

   Press F3 to enter configuration utility
   Primary channel:  WDC WD200-blah-blah           190782 MB
 Primary secondary:  WDC WD200-blah-blah           190782 MB

There would be a bief pause which I ignored then it would continue on
up.  And work just fine.

I did try pressing F3 to see what the utility did but it had no effect
and booted on up.

Now my 200gb mirror is too small, I''m replacing it with 2 750s.

I thought I would shutdown, pull one 200gb and replace it with a 750
to be attached and resilvered.  Then the other 750.

However I hit a snag right away.  Pulled a 200gb added a 750 but on
boot when I get to that screen I see:

   Press F3 to enter configuration utility
   Primary channel:  WDC WD200-blah-blah           190782 MB
 Primary secondary:  WDC WD750-blah-blah

Note the size is missing.  And there the boot hangs.. forever.
Pressing F3 has no effect whatever.

So it appears I''m stuck.  I''m thinking the problem is that the
adaptec
for some reason needs to be able to report a size and that is whats
hanging things up.

So I wondered if it might get me past this hang if I were to plug that
new 750 into a windows/or linux machine and  format it, either as ntfs
on windows or just fdisk a partition of the entire drive on linux.

I hate to do all that if it isn''t necessary and wondered if some of
you experienced people might have a suggestion or two

Harry Putnam wrote:
> Setup: Osol.11 build 109
> Athlon64 3400+ Aopen AK-86L mobo
> adeptec 1250sa Sata PCI controller card
>
> [re-posted from accidental post to osol `general'' group]
>
> I''m having trouble with an adaptec 1205sa (non-raid) SATA PCI
card.
>
> It was all working fine when I plugged 2 used sata 200gb disks of a
> windows xp machine into it.  Booted my osol server and added a zpool
> mirror using those 2 drives and the adaptec 1205sa SATA PCI card.
>   
<snip>
> Now my 200gb mirror is too small, I''m replacing it with 2 750s.
>
> I thought I would shutdown, pull one 200gb and replace it with a 750
> to be attached and resilvered.  Then the other 750.
>
> However I hit a snag right away.  Pulled a 200gb added a 750 but on
> boot when I get to that screen I see:
>
>    Press F3 to enter configuration utility
>    Primary channel:  WDC WD200-blah-blah           190782 MB
>  Primary secondary:  WDC WD750-blah-blah
>
> Note the size is missing.  And there the boot hangs.. forever.
> Pressing F3 has no effect whatever.
>
>   
It looks like the card BOIS has written to the dives and expects the 
same drive to be there.
> So it appears I''m stuck.  I''m thinking the problem is
that the adaptec
> for some reason needs to be able to report a size and that is whats
> hanging things up.
>
> So I wondered if it might get me past this hang if I were to plug that
> new 750 into a windows/or linux machine and  format it, either as ntfs
> on windows or just fdisk a partition of the entire drive on linux.
>
>   
Unlikely, ZFS will re-label the drive an the card should be able to work 
with new drives.

Check the BIOS documentation for the card.

-- 
Ian.

Harry wrote:
>> However I hit a snag right away.  Pulled a 200gb added a 750 but on
>> boot when I get to that screen I see:
>>
>>    Press F3 to enter configuration utility
>>    Primary channel:  WDC WD200-blah-blah           190782 MB
>>  Primary secondary:  WDC WD750-blah-blah
>>
>> Note the size is missing.  And there the boot hangs.. forever.
>> Pressing F3 has no effect whatever.
>>
Ian Collins <ian at ianshome.com> replied:
   
> It looks like the card BOIS has written to the dives and expects the
> same drive to be there.
Harry wrote:
>> So it appears I''m stuck.  I''m thinking the problem is
that the adaptec
>> for some reason needs to be able to report a size and that is whats
>> hanging things up.
>>
>> So I wondered if it might get me past this hang if I were to plug that
>> new 750 into a windows/or linux machine and  format it, either as ntfs
>> on windows or just fdisk a partition of the entire drive on linux.
>>
Ian replied:
   
> Unlikely, ZFS will re-label the drive an the card should be able to
> work with new drives.
How would this relabeling occur.  Somehow I need to get past that
screen right?
> Check the BIOS documentation for the card.
I mentioned that pressing F3 doesn''t do anything.  That is, I have
now way to enter the configuration tool.

I''m not sure what you mean then, except to maybe learn if this card
will not work for these drives.

>I mentioned that pressing F3 doesn''t do anything.  That is, I have
>now way to enter the configuration tool.

Does it work when you first remove the 200GB drive, reboot, press F3
and see what you can do in that configuration tool.

It is possible that it first needs to forget the 200GB drive.

Casper

Casper.Dik at Sun.COM writes:
> Does it work when you first remove the 200GB drive, reboot, press F3
> and see what you can do in that configuration tool.

> It is possible that it first needs to forget the 200GB drive.
Might be the way.  Seems likely given the way you''ve shown how zfs can
remember the real size of a mirror when dealing with different size
drives in a mirror.

I''ll try that next.

I''ve also been told there are sometimes a jumper necessary.  The
adaptec specs show this controller at 1.5 transfer rate and the drives
are sata II 3.gb.  I found some info about it suggesting jumpering
pins 5-6 would set it to 1.5gb so tried that, but got the same hang
at PCI sata card screen as before.  And as before pressing F3 does
nothing.

Casper.Dik at Sun.COM writes:
>>I mentioned that pressing F3 doesn''t do anything.  That is, I
have
>>now way to enter the configuration tool.
>
>
> Does it work when you first remove the 200GB drive, reboot, press F3
> and see what you can do in that configuration tool.
>
> It is possible that it first needs to forget the 200GB drive.
OK, booted with no sata drives attached.  At the press F3 screen,
again pressing F3 appears to have no effect.  But after a moment two
messages saying `no drives found'' print out, and then boot proceeds.

Once booted up I see the recurring message where I should see a login
prompt (I''m setup to boot into console mode).

  ata_id_common Busy Status 0xfe error 0x0  

Repeated 4 times, then after maybe a 2-3 minute wait the regular login
prompt appears.

There is something more going on that is dragging the machine down so
that typing at the prompt is delayed by huge pauses.  

I''m not really able to find out what it is yet but will post back
later after a reboot.

Have you checked the specs of the 1205 to see what maximum drive size
it supports?  That''s an older card, IIRC, so it might top out at 500gb
or something.

On Sat, Mar 28, 2009 at 10:33 AM, Harry Putnam <reader at newsguy.com>
wrote:
> Casper.Dik at Sun.COM writes:
>
>>>I mentioned that pressing F3 doesn''t do anything. ?That is,
I have
>>>now way to enter the configuration tool.
>>
>>
>> Does it work when you first remove the 200GB drive, reboot, press F3
>> and see what you can do in that configuration tool.
>>
>> It is possible that it first needs to forget the 200GB drive.
>
> OK, booted with no sata drives attached. ?At the press F3 screen,
> again pressing F3 appears to have no effect. ?But after a moment two
> messages saying `no drives found'' print out, and then boot
proceeds.
>
> Once booted up I see the recurring message where I should see a login
> prompt (I''m setup to boot into console mode).
>
> ?ata_id_common Busy Status 0xfe error 0x0
>
> Repeated 4 times, then after maybe a 2-3 minute wait the regular login
> prompt appears.
>
> There is something more going on that is dragging the machine down so
> that typing at the prompt is delayed by huge pauses.
>
> I''m not really able to find out what it is yet but will post back
> later after a reboot.
>
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

Blake <blake.irvin at gmail.com> writes:
> Have you checked the specs of the 1205 to see what maximum drive size
> it supports?  That''s an older card, IIRC, so it might top out at
500gb
> or something.
I did yes, all it really says is that is supports drives larger than
137gb.

   http://www.adaptec.com/en-US/support/sata/sata_host/ASH-1205SA/

But it also says its top end transfer rate is 1.5 gb.
The new drives are 3.0gb.  

I''m told there is a possibility of setting a jumper to make sata II
work as sata I.  I''ve tried one jumper position I found some info on
here:
http://www.google.com/search?hl=en&rlz=1G1GGLQ_ENUS273&q=WD+Sata+II+jumpers&btnG=Search

The direct URL is too long to post here but the one above will get you
there.  Its the top one on the page there.  And then Figure 4 in the
pdf.

I tried jumping 5-6 but still got the same behavior when the Sata
controller screen (Press F3 to configure) showed up.  It sees the
drive by name but not the size and there it hangs.

Casper has suggested I pull both old sata drives and reboot, possibly
allowing the hardware to `forget'' those drives.

I''m doing that now as reported earlier in the thread.

Harry Putnam <reader at newsguy.com> writes:
> Once booted up I see the recurring message where I should see a login
> prompt (I''m setup to boot into console mode).
>
>   ata_id_common Busy Status 0xfe error 0x0  
>
> Repeated 4 times, then after maybe a 2-3 minute wait the regular login
> prompt appears.
>
> There is something more going on that is dragging the machine down so
> that typing at the prompt is delayed by huge pauses.  
>
> I''m not really able to find out what it is yet but will post back
> later after a reboot.
Those 4 lines continue to appear on ever boot now.. (still with old
200gb sata drives out.

Seems when the boot prompt does appear there is something dragging
down the os making logging in really sluggish.

It appears to be the fmd daemon running periodically and driving
resource drain on cpu up to 99 percent.

That seems to continue to occur for at least several cycles watching
it in `top''.  They last maybe close to 1 minute then disappear for a
while.  Several minutes at least.  The cylces seem to keep coming
after being booted up for 10 minutes now.

I''m not sure which logs to look in to see whats happing.  But earlier
today these showed up in /var/adm/messages:

Mar 28 09:38:25 zfs fmd: [ID 441519 daemon.error] SUNW-MSG-ID:
ZFS-8000-D3, TYPE: Fault, VER: 1, SEVERITY: Major
Mar 28 09:38:25 zfs IMPACT: Fault tolerance of the pool may be
compromised.

The console login continues to be nearly useless with the delays and
pauses while typing.

However ssh in and the terminal I get seem to be less effected, or
even un-effected so I can do things in some kind or reasonable way.

I''m really not sure what to do though.  I did 
  `zpool destroy thatpool'' on the pool that was on the sata drives.

That appears to have worked, but didn''t help with resource drain
coming from `fmd''

The tail of the log pointed to by svcs -l system/fmd shows:
(/var/svc/log/system-fmd:default.log)
[...]
[ Mar 28 09:20:31 Enabled. ]
[ Mar 28 09:25:11 Executing start method ("/usr/lib/fm/fmd/fmd"). ]
[ Mar 28 09:28:24 Method "start" exited with status 0. ]
[ Mar 28 09:53:50 Enabled. ]
[ Mar 28 09:54:50 Executing start method ("/usr/lib/fm/fmd/fmd"). ]
[ Mar 28 09:55:44 Method "start" exited with status 0. ]
[ Mar 28 09:59:21 Stopping because service disabled. ]
[ Mar 28 09:59:21 Executing stop method (:kill). ]
[ Mar 28 10:01:09 Enabled. ]
[ Mar 28 10:02:17 Executing start method ("/usr/lib/fm/fmd/fmd"). ]
[ Mar 28 10:02:49 Method "start" exited with status 0. ]

I have no idea is that is normal or what.

what''s the output of ''fmadm faulty''?

On Sat, Mar 28, 2009 at 12:22 PM, Harry Putnam <reader at newsguy.com>
wrote:
> Harry Putnam <reader at newsguy.com> writes:
>
>> Once booted up I see the recurring message where I should see a login
>> prompt (I''m setup to boot into console mode).
>>
>> ? ata_id_common Busy Status 0xfe error 0x0
>>
>> Repeated 4 times, then after maybe a 2-3 minute wait the regular login
>> prompt appears.
>>
>> There is something more going on that is dragging the machine down so
>> that typing at the prompt is delayed by huge pauses.
>>
>> I''m not really able to find out what it is yet but will post
back
>> later after a reboot.
>
> Those 4 lines continue to appear on ever boot now.. (still with old
> 200gb sata drives out.
>
> Seems when the boot prompt does appear there is something dragging
> down the os making logging in really sluggish.
>
> It appears to be the fmd daemon running periodically and driving
> resource drain on cpu up to 99 percent.
>
> That seems to continue to occur for at least several cycles watching
> it in `top''. ?They last maybe close to 1 minute then disappear for
a
> while. ?Several minutes at least. ?The cylces seem to keep coming
> after being booted up for 10 minutes now.
>
> I''m not sure which logs to look in to see whats happing. ?But
earlier
> today these showed up in /var/adm/messages:
>
> Mar 28 09:38:25 zfs fmd: [ID 441519 daemon.error] SUNW-MSG-ID:
> ZFS-8000-D3, TYPE: Fault, VER: 1, SEVERITY: Major
> Mar 28 09:38:25 zfs IMPACT: Fault tolerance of the pool may be
> compromised.
>
> The console login continues to be nearly useless with the delays and
> pauses while typing.
>
> However ssh in and the terminal I get seem to be less effected, or
> even un-effected so I can do things in some kind or reasonable way.
>
> I''m really not sure what to do though. ?I did
> ?`zpool destroy thatpool'' on the pool that was on the sata drives.
>
> That appears to have worked, but didn''t help with resource drain
> coming from `fmd''
>
> The tail of the log pointed to by svcs -l system/fmd shows:
> (/var/svc/log/system-fmd:default.log)
> [...]
> [ Mar 28 09:20:31 Enabled. ]
> [ Mar 28 09:25:11 Executing start method ("/usr/lib/fm/fmd/fmd").
]
> [ Mar 28 09:28:24 Method "start" exited with status 0. ]
> [ Mar 28 09:53:50 Enabled. ]
> [ Mar 28 09:54:50 Executing start method ("/usr/lib/fm/fmd/fmd").
]
> [ Mar 28 09:55:44 Method "start" exited with status 0. ]
> [ Mar 28 09:59:21 Stopping because service disabled. ]
> [ Mar 28 09:59:21 Executing stop method (:kill). ]
> [ Mar 28 10:01:09 Enabled. ]
> [ Mar 28 10:02:17 Executing start method ("/usr/lib/fm/fmd/fmd").
]
> [ Mar 28 10:02:49 Method "start" exited with status 0. ]
>
> I have no idea is that is normal or what.
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

Blake <blake.irvin at gmail.com> writes:
> what''s the output of ''fmadm faulty''?
It hangs for maybe 30 seconds, then returns the prompt.
And again now on this boot, before the console prompt appears I see:
   ata_id_common: Busy Status 0xfe error 0x0

Repeated 4 times.

And again, the console prompt is barely usable.

 login <type username <return>>

A full minute or so later, the passwd part comes up

Enter passwd and another minute or so to login.

I don''t see fmd making huge cpu resource drain so not sure what is
dragging down the login prompt.

`top'' shows 99.6 % idle even 99.8 but still the login prompt is very
sluggish. 

I can''t really even work with it, even after finally getting logged in
its just too aggravating to wait that minute or so between typing
something and seeing it appear on the cmd line.

Fortunately ssh works and that terminal seems un-affected. 

But it appears I have done major damage somehow by pulling out those 2
drives.  Or something else I''ve unwittingly done.

Casper.Dik at Sun.COM writes:
>>I mentioned that pressing F3 doesn''t do anything.  That is, I
have
>>now way to enter the configuration tool.
>
>
> Does it work when you first remove the 200GB drive, reboot, press F3
> and see what you can do in that configuration tool.
>
> It is possible that it first needs to forget the 200GB drive.
It was worth a try... but as posted further along in the thread I ran
into a bunch of mess going that route.  Booting without the drives
causes several kinds of things to happen that I don''t understand.

bootup passes the `Press F3 screen'' Ok after a brief pause.

But when I get to the consol login I see a line like:

  ata_id_common Busy Status 0xfe error 0x0

Repeated 4 times then after a 30 second or so pause comes the login
prompt.  But the prompt is very unresponsive.  I mean if I type into
it takes nearly a full minute or more to show up

So after taking a couple minutes getting logged in, any activity at
the command line remains very sluggish.

Fortunately though, I can ssh into the box and that terminal seems to
be un-effected.

Watching with `top'' I see the fmd daemon running in a recurring cycle
and each time it comes on it drives the resource usage up into the
high 90s for a minute or two.  It appears to continue to cycle for at
least 15 minutes at which point I rebooted again.

It appears that if I boot without the drives installed I will continue
to have this happen.  But re-insert the 2 drives in the original order
and it all goes away. 

Some of this may be repititious since I''ve been dicussing the sata
card on another (non-osol [gentoo.user]) list too and have kind of
lost track of what all got posted where.

But first, both of you have really been patient and taken a fair bit
of time to explain things... and I really appreciate it... so thanks. 


Blake <blake.irvin at gmail.com> writes:
> what''s the output of ''fmadm faulty''?
You may have already seen my answer to that, but wanted to let you
both know that I''ve learned that at least one other computer user has
posted information to the effect that my card the Adaptec 1205sa which
apparently has an sil3112a chip is incompatable with Seagate 750 sata
drives ... mine are WD but its beginning to look like this card may
not support the newer sata drives.

I''m starting to think I may have just got the wrong card and should
just throw in the towel and get something known to work with my
hardware.

I did attempt to learn that before buying and did post at least
twice on the opensolaris groups trying to find out what would work for
sure on my hardware.  Either I didn''t get replies or no one knew, I
don''t remember which now.

Readers may have thought I was just too lazy to use the HCL but the
truth is there is precious little on it for my hardware.

But on the off chance either of you or maybe someone following along
will know of a sata card known to work with my hardware... I''m listing
the hardware here :

Athlon64 2.2ghz 3400+ - Aopen AK86-L mobo - Topped out at 3gb CORSAIR
ram

 (The onboard sata controller is not recognized by osol - hence
   the need for sata PCI card.).

Incidentally, I did find that Adaptec 1205sa on the HCL, listed by one
user who was able to boot with it installed and have drives
recognized.  The hardware was NOT a real match for mine but I took a
chance on it anyway... and it does work, at least with older/smaller
sata drives.

The specs on the 1205sa say that it will handle drives larger than
137gb and that its top transfer rate is 1.5gb.

So its not really clear what its'' upper limit, if any, is far as size.
But it is an older card and may predate sata II.

Blake wrote:
> Have you checked the specs of the 1205 to see what maximum drive size
> it supports?  That''s an older card, IIRC, so it might top out at
500gb
> or something.
>
>   
Please stop top-posting to threads where everyone else is 
normal-posting, it mucks up the flow of the thread.

Thanks,

-- 
Ian.

Harry Putnam wrote:
>
> I''m starting to think I may have just got the wrong card and
should
> just throw in the towel and get something known to work with my
> hardware.
>
>   
That sounds like a good plan!

Using older SATA controllers with SATA II drives may work, but there 
aren''t any guarantees.

-- 
Ian.

Ian Collins <ian at ianshome.com> writes:
> Harry Putnam wrote:
>>
>> I''m starting to think I may have just got the wrong card and
should
>> just throw in the towel and get something known to work with my
>> hardware.
>>
>>   
> That sounds like a good plan!
>
> Using older SATA controllers with SATA II drives may work, but there
> aren''t any guarantees.
Finding a sataII card is proving to be very difficult.  The reason is
that I only have PCI no PCI express.  I haven''t see a single one
listed as SATAII compatible and have spent a bit time googling.

I''d settle for a PCI card that someone can vouch for.  I did find a
message on an Australian overclockers list that indicated that fellow
had used sil3114 (mine is 3112a) card to run SATAII at 640gb.  But of
course those guys are all talking about `Windows'' of one strain or
another so may or may not mean much.

I don''t supposed you''d have a tip about a PCI sata card
capable of
handling newer SATAII would you?

On 03/28/09 20:01, Harry Putnam wrote:
> Finding a sataII card is proving to be very difficult.  The reason is
> that I only have PCI no PCI express.  I haven''t see a single one
> listed as SATAII compatible and have spent a bit time googling.
It''s even worse if you have an old SPARC system. We''ve had
great results
with some LSI LOGIC SAS3041XL-S cards we got on E-Bay in conjunction
with 3x1.5TB Seagate drives, for 2.7TiB of raidz. The combination proved
faster than mirrored 10,000 RPM SCSI disks using UFS, in an unscientific
benchmark (bonnie). I don''t think this LSI controller is SATA II, but
it has no problems with the 1.5TB Seagates; they are Sun branded cards
and worked right of of the box in 3.3V 66MHz PCI slots. 2.7TiB of raidz
for around $400. Amazing... And ZFS is just plain incredible - makes
every other file system look so antiquated :-)

Hope this helps -- Frank

Frank Middleton <f.middleton at apogeect.com> writes:
> It''s even worse if you have an old SPARC system. We''ve
had great results
> with some LSI LOGIC SAS3041XL-S cards we got on E-Bay in conjunction
> with 3x1.5TB Seagate drives, for 2.7TiB of raidz. The combination proved
> faster than mirrored 10,000 RPM SCSI disks using UFS, in an unscientific
> benchmark (bonnie). I don''t think this LSI controller is SATA II,
but
> it has no problems with the 1.5TB Seagates; they are Sun branded cards
> and worked right of of the box in 3.3V 66MHz PCI slots. 2.7TiB of raidz
> for around $400. Amazing... And ZFS is just plain incredible - makes
> every other file system look so antiquated :-)
>
> Hope this helps -- Frank
This looks really promising, scouting around on ebay I see one has
sold for $62 us.  Not too bad.  I''ll keep that item name on file.

I''ve already bought a sil3114 chip card off ebay ($20 us) yesterday so
I''ll see how I fare on that first.  I''m told it will recognize
sata II
if you flash it to latest bios update.  I''m not sure but that may also
be possible with the card I''m having the trouble with. (Adaptec
1205sa) which is Sil3112a chip.

I''ll have to install it on a windows machine... since my Osol server
has no floppy drive.   Hehe, I plan to open up my wifes machine to do
it. (Shes gone for the day or the fight would be on)

Of course, soon as I bought it I saw another site selling the same card
for $18 us...

Harry Putnam <reader at newsguy.com> writes:
> I''ve already bought a sil3114 chip card off ebay ($20 us)
yesterday so
> I''ll see how I fare on that first.  I''m told it will
recognize sata II
> if you flash it to latest bios update.  I''m not sure but that may
also
> be possible with the card I''m having the trouble with. (Adaptec
> 1205sa) which is Sil3112a chip.
>
> I''ll have to install it on a windows machine... since my Osol
server
> has no floppy drive.   Hehe, I plan to open up my wifes machine to do
> it. (Shes gone for the day or the fight would be on)
I quite being a lazy puke, braved up and installed the Adaptec 1205sa
on a windows machine and updated the bios.  Apparently its either in
two places on the chip or each on its own chip, but it took both
`base BIOS'' and `sata_raid BIOS'', so a two part flash.

I didn''t really think it would help, mostly why I didn''t do
that
sooner. Also didn''t really know how.

But cutting to the chase here, it worked just fine after putting it
back in the OSOL box, resilvering of one 200gb from the old mirror is
in progress as I write.

No jumpers needed, it just works.  Some had said I might need to use a
jumper to make the disk 1.5 gb instead of 3.0gb but apparently the
Adaptec 1205sa can do that with out a jumper.

I did jump the gun and order a Sil3114 from ebay for $20 + $10
shipping but it won''t hurt to have a backup, and I may need the extra
ports at some point.

Anyway, I''m a happy camper and didn''t even get another beating
from my
wife. . . well, at least not yet...

On Sat, Mar 28, 2009 at 6:57 PM, Ian Collins <ian at ianshome.com>
wrote:
> Please stop top-posting to threads where everyone else is normal-posting,
it
> mucks up the flow of the thread.
>
> Thanks,
>
> --
> Ian.
Apologies - top-posting seems to be the Gmail default (or I set it so
long ago that I forgot it was there).