Julian Stacey
2008-Sep-25 16:17 UTC
rl0: watchdog timeout + 40, 000 ms ping with 7.1-BETA-i386-disc1.iso
Hi stable@, I just imported an old tower from a friend. Used to run Linux OK. Reset BIOS to defaults, turned off power saving etc, installed 7.1-BETA-i386-disc1.iso I now sees rl0: watchdog timeout + 40,000 ms ping outgoing. ping incoming fails, it's not my net switch, I've moved to different segments etc & all else fine I'm remaking binaries, & will look around for netstat r whatever commands later, meanwhile here's dmesg (via a floppy) Of course it could be somehow a hardaware bad config, its a new box to me. Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.1-BETA #0: Sun Sep 7 13:49:18 UTC 2008 root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel Pentium III (651.48-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x681 Stepping = 1 Features=0x383f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE> real memory = 134152192 (127 MB) avail memory = 117157888 (111 MB) kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: <AWARD AWRDACPI> on motherboard acpi0: [ITHREAD] ACPI Error (psargs-0459): [INX_] Namespace lookup failure, AE_NOT_FOUND ACPI Error (psparse-0626): Method parse/execution failed [\\_SB_.PCI0._PRW] (Node 0xc1bd6700), AE_NOT_FOUND acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 7ef0000 (3) failed Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff,0x4000-0x407f,0x4080-0x40ff,0x5000-0x500f on acpi0 pci0: <ACPI PCI bus> on pcib0 agp0: <VIA 82C691 (Apollo Pro) host to PCI bridge> on hostb0 agp0: aperture size is 256M pcib1: <PCI-PCI bridge> at device 1.0 on pci0 pci1: <PCI bus> on pcib1 vgapci0: <VGA-compatible display> port 0xc000-0xc0ff mem 0xe0000000-0xe7ffffff,0xed000000-0xed00ffff irq 11 at device 0.0 on pci1 isab0: <PCI-ISA bridge> at device 7.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <VIA 82C596B UDMA66 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xd000-0xd00f at device 7.1 on pci0 ata0: <ATA channel 0> on atapci0 ata0: [ITHREAD] ata1: <ATA channel 1> on atapci0 ata1: [ITHREAD] uhci0: <VIA 83C572 USB controller> port 0xd400-0xd41f irq 10 at device 7.2 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: <VIA 83C572 USB controller> on uhci0 usb0: USB revision 1.0 uhub0: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0 uhub0: 2 ports with 2 removable, self powered pci0: <bridge, HOST-PCI> at device 7.3 (no driver attached) rl0: <RealTek 8139 10/100BaseTX> port 0xd800-0xd8ff mem 0xee000000-0xee0000ff irq 12 at device 10.0 on pci0 miibus0: <MII bus> on rl0 rlphy0: <RealTek internal media interface> PHY 0 on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto rl0: Ethernet address: 00:08:a1:6d:65:07 rl0: [ITHREAD] pci0: <multimedia, audio> at device 11.0 (no driver attached) cpu0: <ACPI CPU> on acpi0 acpi_throttle0: <ACPI CPU Throttling> on cpu0 acpi_button0: <Power Button> on acpi0 acpi_tz0: <Thermal Zone> on acpi0 fdc0: <floppy drive controller> port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A sio1: [FILTER] atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] ACPI Error (psargs-0459): [INX_] Namespace lookup failure, AE_NOT_FOUND ACPI Error (psparse-0626): Method parse/execution failed [\\_SB_.PCI0._PRW] (Node 0xc1bd6700), AE_NOT_FOUND ACPI Error (psargs-0459): [INX_] Namespace lookup failure, AE_NOT_FOUND ACPI Error (psparse-0626): Method parse/execution failed [\\_SB_.PCI0._PRW] (Node 0xc1bd6700), AE_NOT_FOUND pmtimer0 on isa0 orm0: <ISA Option ROM> at iomem 0xc0000-0xccfff pnpid ORM0000 on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 651482522 Hz quality 800 Timecounters tick every 1.000 msec ad0: 4110MB <QUANTUM FIREBALL SE4.3A API.0C00> at ata0-master UDMA33 acd0: CDROM <ATAPI CDROM/V100Y> at ata1-master UDMA33 Trying to mount root from ufs:/dev/ad0s1a rl0: link state changed to UP rl0: watchdog timeout rl0: link state changed to DOWN rl0: link state changed to UP rl0: link state changed to DOWN rl0: link state changed to UP rl0: watchdog timeout rl0: watchdog timeout Cheers, Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org
Jeremy Chadwick
2008-Sep-25 16:45 UTC
rl0: watchdog timeout + 40, 000 ms ping with 7.1-BETA-i386-disc1.iso
On Thu, Sep 25, 2008 at 05:36:44PM +0200, Julian Stacey wrote:> Hi stable@, > I just imported an old tower from a friend. Used to run Linux OK. > Reset BIOS to defaults, turned off power saving etc, installed > 7.1-BETA-i386-disc1.iso > I now sees > rl0: watchdog timeout + 40,000 ms ping outgoing. > ping incoming fails, > it's not my net switch, I've moved to different segments etc & all else fine > > I'm remaking binaries, & will look around for netstat r whatever > commands later, meanwhile here's dmesg (via a floppy) > > Of course it could be somehow a hardaware bad config, its a new box to me.It's a "new box" with hardware from the late 90s? :-)> Copyright (c) 1992-2008 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 7.1-BETA #0: Sun Sep 7 13:49:18 UTC 2008 > root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel Pentium III (651.48-MHz 686-class CPU) > Origin = "GenuineIntel" Id = 0x681 Stepping = 1 > Features=0x383f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE> > real memory = 134152192 (127 MB) > avail memory = 117157888 (111 MB) > kbd1 at kbdmux0 > ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) > acpi0: <AWARD AWRDACPI> on motherboard > acpi0: [ITHREAD] > ACPI Error (psargs-0459): [INX_] Namespace lookup failure, AE_NOT_FOUND > ACPI Error (psparse-0626): Method parse/execution failed [\\_SB_.PCI0._PRW] (Node 0xc1bd6700), AE_NOT_FOUND > acpi0: Power Button (fixed) > acpi0: reservation of 0, a0000 (3) failed > acpi0: reservation of 100000, 7ef0000 (3) failed > Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 > pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff,0x4000-0x407f,0x4080-0x40ff,0x5000-0x500f on acpi0 > pci0: <ACPI PCI bus> on pcib0 > agp0: <VIA 82C691 (Apollo Pro) host to PCI bridge> on hostb0 > agp0: aperture size is 256M > pcib1: <PCI-PCI bridge> at device 1.0 on pci0 > pci1: <PCI bus> on pcib1 > vgapci0: <VGA-compatible display> port 0xc000-0xc0ff mem 0xe0000000-0xe7ffffff,0xed000000-0xed00ffff irq 11 at device 0.0 on pci1 > isab0: <PCI-ISA bridge> at device 7.0 on pci0 > isa0: <ISA bus> on isab0 > atapci0: <VIA 82C596B UDMA66 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xd000-0xd00f at device 7.1 on pci0 > ata0: <ATA channel 0> on atapci0 > ata0: [ITHREAD] > ata1: <ATA channel 1> on atapci0 > ata1: [ITHREAD] > uhci0: <VIA 83C572 USB controller> port 0xd400-0xd41f irq 10 at device 7.2 on pci0 > uhci0: [GIANT-LOCKED] > uhci0: [ITHREAD] > usb0: <VIA 83C572 USB controller> on uhci0 > usb0: USB revision 1.0 > uhub0: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0 > uhub0: 2 ports with 2 removable, self powered > pci0: <bridge, HOST-PCI> at device 7.3 (no driver attached) > rl0: <RealTek 8139 10/100BaseTX> port 0xd800-0xd8ff mem 0xee000000-0xee0000ff irq 12 at device 10.0 on pci0 > miibus0: <MII bus> on rl0 > rlphy0: <RealTek internal media interface> PHY 0 on miibus0 > rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto > rl0: Ethernet address: 00:08:a1:6d:65:07 > rl0: [ITHREAD] > pci0: <multimedia, audio> at device 11.0 (no driver attached) > cpu0: <ACPI CPU> on acpi0 > acpi_throttle0: <ACPI CPU Throttling> on cpu0 > acpi_button0: <Power Button> on acpi0 > acpi_tz0: <Thermal Zone> on acpi0 > fdc0: <floppy drive controller> port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0 > fdc0: [FILTER] > fd0: <1440-KB 3.5" drive> on fdc0 drive 0 > sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 > sio0: type 16550A > sio0: [FILTER] > sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 > sio1: type 16550A > sio1: [FILTER] > atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 > atkbd0: <AT Keyboard> irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > atkbd0: [ITHREAD] > ACPI Error (psargs-0459): [INX_] Namespace lookup failure, AE_NOT_FOUND > ACPI Error (psparse-0626): Method parse/execution failed [\\_SB_.PCI0._PRW] (Node 0xc1bd6700), AE_NOT_FOUND > ACPI Error (psargs-0459): [INX_] Namespace lookup failure, AE_NOT_FOUND > ACPI Error (psparse-0626): Method parse/execution failed [\\_SB_.PCI0._PRW] (Node 0xc1bd6700), AE_NOT_FOUND > pmtimer0 on isa0 > orm0: <ISA Option ROM> at iomem 0xc0000-0xccfff pnpid ORM0000 on isa0 > sc0: <System console> at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=0x300> > vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 > Timecounter "TSC" frequency 651482522 Hz quality 800 > Timecounters tick every 1.000 msec > ad0: 4110MB <QUANTUM FIREBALL SE4.3A API.0C00> at ata0-master UDMA33 > acd0: CDROM <ATAPI CDROM/V100Y> at ata1-master UDMA33 > Trying to mount root from ufs:/dev/ad0s1a > rl0: link state changed to UP > rl0: watchdog timeout > rl0: link state changed to DOWN > rl0: link state changed to UP > rl0: link state changed to DOWN > rl0: link state changed to UP > rl0: watchdog timeout > rl0: watchdog timeoutI've CC'd PYUN Yong-Hyeon (surname is Pyun), who helps maintain the rl(4) driver. He might have some ideas. For now, please provide the output from "vmstat -i" and "pciconf -lv". And some stuff to ponder, just ideas/comments in passing: * Board does not appear to have an APIC (not a typo), so IRQ sharing is likely. IRQ 12 is often used for PS/2 devices (mice, keyboards), and I see rl0 is sitting on that IRQ. * Read the BUGS section of the rl(4) manpage; these NICs are known to be questionable in many regards. The CVS commit log for this driver also has quite a list of other documented (major) bugs. "User beware": http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/pci/if_rl.c * It's a VIA chipset board. Old Tyan server boards I had using VIA chips (Apollo with 686B southbridge) would, even so often, stop handling/firing IRQs. This would happen with almost any device, even ones which were known to be relible (like fxp(4)). The only solution would be to hard reset it. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Pyun YongHyeon
2008-Sep-26 02:28 UTC
rl0: watchdog timeout + 40, 000 ms ping with 7.1-BETA-i386-disc1.iso
On Thu, Sep 25, 2008 at 05:36:44PM +0200, Julian Stacey wrote: > Hi stable@, > I just imported an old tower from a friend. Used to run Linux OK. > Reset BIOS to defaults, turned off power saving etc, installed > 7.1-BETA-i386-disc1.iso > I now sees > rl0: watchdog timeout + 40,000 ms ping outgoing. > ping incoming fails, > it's not my net switch, I've moved to different segments etc & all else fine > > I'm remaking binaries, & will look around for netstat r whatever > commands later, meanwhile here's dmesg (via a floppy) > > Of course it could be somehow a hardaware bad config, its a new box to me. > > Copyright (c) 1992-2008 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 7.1-BETA #0: Sun Sep 7 13:49:18 UTC 2008 > root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel Pentium III (651.48-MHz 686-class CPU) > Origin = "GenuineIntel" Id = 0x681 Stepping = 1 > Features=0x383f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE> > real memory = 134152192 (127 MB) > avail memory = 117157888 (111 MB) [...] > rl0: <RealTek 8139 10/100BaseTX> port 0xd800-0xd8ff mem 0xee000000-0xee0000ff irq 12 at device 10.0 on pci0 > miibus0: <MII bus> on rl0 > rlphy0: <RealTek internal media interface> PHY 0 on miibus0 > rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto > rl0: Ethernet address: 00:08:a1:6d:65:07 > rl0: [ITHREAD] [...] > rl0: link state changed to UP > rl0: watchdog timeout > rl0: link state changed to DOWN > rl0: link state changed to UP > rl0: link state changed to DOWN > rl0: link state changed to UP > rl0: watchdog timeout > rl0: watchdog timeout > Is there reliable way to reproduce the issue? Anyway, would you try attached patch and let me know result? -- Regards, Pyun YongHyeon -------------- next part -------------- Index: sys/pci/if_rl.c ==================================================================--- sys/pci/if_rl.c (revision 183366) +++ sys/pci/if_rl.c (working copy) @@ -650,6 +650,34 @@ static void rl_miibus_statchg(device_t dev) { + struct rl_softc *sc; + struct ifnet *ifp; + struct mii_data *mii; + + sc = device_get_softc(dev); + mii = device_get_softc(sc->rl_miibus); + ifp = sc->rl_ifp; + if (mii == NULL || ifp == NULL || + (ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) + return; + + sc->rl_flags &= ~RL_FLAG_LINK; + if ((mii->mii_media_status & (IFM_ACTIVE | IFM_AVALID)) =+ (IFM_ACTIVE | IFM_AVALID)) { + switch (IFM_SUBTYPE(mii->mii_media_active)) { + case IFM_10_T: + case IFM_100_TX: + sc->rl_flags |= RL_FLAG_LINK; + break; + default: + break; + } + } + /* + * RealTek controllers does not provide any interface to + * Tx/Rx MACs for resolved speed, duplex and flow-control + * parameters. + */ } /* @@ -1236,7 +1264,6 @@ CSR_WRITE_4(sc, RL_TXCFG, RL_TXCFG_CONFIG); oldthresh = sc->rl_txthresh; /* error recovery */ - rl_reset(sc); rl_init_locked(sc); /* restore original threshold */ sc->rl_txthresh = oldthresh; @@ -1305,10 +1332,8 @@ /* XXX We should check behaviour on receiver stalls. */ - if (status & RL_ISR_SYSTEM_ERR) { - rl_reset(sc); + if (status & RL_ISR_SYSTEM_ERR) rl_init_locked(sc); - } } } #endif /* DEVICE_POLLING */ @@ -1345,10 +1370,8 @@ rl_rxeof(sc); if ((status & RL_ISR_TX_OK) || (status & RL_ISR_TX_ERR)) rl_txeof(sc); - if (status & RL_ISR_SYSTEM_ERR) { - rl_reset(sc); + if (status & RL_ISR_SYSTEM_ERR) rl_init_locked(sc); - } } if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) @@ -1423,6 +1446,10 @@ RL_LOCK_ASSERT(sc); + if ((ifp->if_drv_flags & (IFF_DRV_RUNNING | IFF_DRV_OACTIVE)) !+ IFF_DRV_RUNNING || (sc->rl_flags & RL_FLAG_LINK) == 0) + return; + while (RL_CUR_TXMBUF(sc) == NULL) { IFQ_DRV_DEQUEUE(&ifp->if_snd, m_head); @@ -1489,6 +1516,8 @@ */ rl_stop(sc); + rl_reset(sc); + /* * Init our MAC address. Even though the chipset * documentation doesn't mention it, we need to enter "Config @@ -1564,6 +1593,7 @@ /* Enable receiver and transmitter. */ CSR_WRITE_1(sc, RL_COMMAND, RL_CMD_TX_ENB|RL_CMD_RX_ENB); + sc->rl_flags &= ~RL_FLAG_LINK; mii_mediachg(mii); CSR_WRITE_1(sc, RL_CFG1, RL_CFG1_DRVLOAD|RL_CFG1_FULLDUPLEX); @@ -1709,9 +1739,19 @@ sc->rl_watchdog_timer = 0; callout_stop(&sc->rl_stat_callout); ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE); + sc->rl_flags &= ~RL_FLAG_LINK; + CSR_WRITE_2(sc, RL_IMR, 0x0000); CSR_WRITE_1(sc, RL_COMMAND, 0x00); - CSR_WRITE_2(sc, RL_IMR, 0x0000); + for (i = 0; i < RL_TIMEOUT; i++) { + DELAY(1); + if ((CSR_READ_1(sc, RL_COMMAND) & + (RL_CMD_RX_ENB | RL_CMD_TX_ENB)) == 0) + break; + } + if (i == RL_TIMEOUT) + device_printf(sc->rl_dev, "stopping Tx/Rx MAC timed out!\n"); + bus_dmamap_unload(sc->rl_tag, sc->rl_cdata.rl_rx_dmamap); /*
Abdullah Ibn Hamad Al-Marri
2008-Sep-26 18:03 UTC
rl0: watchdog timeout + 40, 000 ms ping with 7.1-BETA-i386-disc1.iso
----- Original Message ----> From: Julian Stacey <jhs@berklix.org> > To: pyunyh@gmail.com > Cc: stable@freebsd.org > Sent: Friday, September 26, 2008 8:16:57 PM > Subject: Re: rl0: watchdog timeout + 40, 000 ms ping with 7.1-BETA-i386-disc1.iso > > > > I'm remaking binaries, > > New generic kernel built & installed, & install of all src/ done too. > No improvement. > > > Is there reliable way to reproduce the issue? > > Its continuous, the machine virtually never does a ping in less > than 10 seconds. > > > Anyway, would you try attached patch and let me know result? > > Thanks > Done, doesnt help. > Seeing a new message now too: > ping: sendto: No buffer space available. > > Output of vmstat -i and pciconf -lv look the same as before > > It's a small card. Weighs 46 gram. I was going to write > I could simply post it to you, & you could keep it if you > want. As I had quessed it might be some new kind of card > unexperienced before, > RTL8139D, card just says made in China > > But I just grabbed another card > card says Level One. > chip 8139B > & with both patched kernel & original no improvement. > So I tried a totaly different card xl0 fails too, > I think that 3com xl0 card was OK before in another box, > so I'd guess not an rl problem, Sorry. > > Probably not 7.1 either, but probably a BIOS config problem of some sort. > > IRQ 12 was listed in Award BIOS as Primary, options were also secondary or > disabled, so Ive set it disabled. > PNP OS Yes > Resources: Auto > "Reset config data" to Enabled (I forgot before after card changes) > > Did another restore BIOS factory defaults, no help. > Moved xl0 to another slot (all other 3 slots never use I guess, as > chassis plates not torn off on what I guess is original chassis. > No luck with xl0 > I'm out of ideas. > > > Cheers, > Julian > -- > Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com > Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.orgJust a shot in the darkness. Do you have poll enabled for rl0 ? Regards, -Abdullah Ibn Hamad Al-Marri Arab Portal http://www.WeArab.Net/