freebsd stable - May 2006 - Reproducable file corruption on 6-STABLE

Hello,

When I try to copy a large (7GB+) file from one filesystem to another the
copy is not equal to te source file.  This behaviour can be reproduced
repeatedly.

Details:

  - FreeBSD 6-STABLE, cvsupped yesterday
  - Intel L440GX+ main board, dual 700 MHz Pentium 3, 1 GB RAM
  - 2x Promise TX3 IDE controllers
  - 4x Hitachi HDS725050KLAT80 harddisks (500GB each)
    These disks are mirrored using gvinum as two 500GB volumes:

      drive disk0 device /dev/ad4s1e
      drive disk1 device /dev/ad6s1e
      drive disk2 device /dev/ad8s1e
      drive disk3 device /dev/ad10s1e
      volume vol0
       plex org concat
	sd length 0 drive disk0
       plex org concat
	sd length 0 drive disk2
      volume vol1
       plex org concat
	sd length 0 drive disk1
       plex org concat
	sd length 0 drive disk3

    Other filesystems are on SCSI disks.

  # sum /data/vol1/some_7GB_file
  19249 6990148 /data/vol1/some_7GB_file

  # cp -p /data/vol1/some_7GB_file /data/vol0/tmp.1
  # sum /data/vol0/tmp.1
  54291 6990148 /data/vol0/tmp.1

  # cp -p /data/vol1/some_7GB_file /data/vol0/tmp.2
  # sum /data/vol0/tmp.2
  46435 6990148 /data/vol0/tmp.2

  # cp -p /data/vol1/some_7GB_file /data/vol0/tmp.3
  # sum /data/vol0/tmp.3
  53090 6990148 /data/vol0/tmp.3

  # cp -p /data/vol1/some_7GB_file /data/vol0/tmp.4
  # sum /data/vol0/tmp.4
  63059 6990148 /data/vol0/tmp.4

  # cp -p /data/vol1/some_7GB_file /data/vol2/tmp.5
  # sum /data/vol2/tmp.5
  44188 6990148 /data/vol2/tmp.5

No errors in dmesg.

Checksums are the same when repeating the sum command later so the
problem occurs during copy, not during read-back.

When I first copy the file from /data/vol1 to a filesystem on a pair of
SCSI disks and then to /data/vol2 from the SCSI disks I get a correct
copy.

Relevant dmesg output:

  pcib1: <PCI-PCI bridge> at device 1.0 on pci0
  pci1: <PCI bus> on pcib1
  pcib2: <PCI-PCI bridge> at device 15.0 on pci1
  pci2: <PCI bus> on pcib2
  atapci0: <Promise PDC20269 UDMA133 controller> port
0x3030-0x3037,0x3024-0x3027,0x3028-0x302f,0x3020-0x3023,0x3000-0x300f mem
0xf4200000-0xf4203fff irq 20 at device 4.0 on pci2
  ata2: <ATA channel 0> on atapci0
  ata3: <ATA channel 1> on atapci0
  atapci1: <Promise PDC20269 UDMA133 controller> port
0x3048-0x304f,0x303c-0x303f,0x3040-0x3047,0x3038-0x303b,0x3010-0x301f mem
0xf4204000-0xf4207fff irq 23 at device 7.0 on pci2
  ata4: <ATA channel 0> on atapci1
  ata5: <ATA channel 1> on atapci1
  ad4: 476940MB <HDS725050KLAT80 K2AOA10C> at ata2-master UDMA133
  ad6: 476940MB <HDS725050KLAT80 K2AOA10C> at ata3-master UDMA133
  ad8: 476940MB <HDS725050KLAT80 K2AOA10C> at ata4-master UDMA133
  ad10: 476940MB <HDS725050KLAT80 K2AOA10C> at ata5-master UDMA133

Hope someone can shed a light on this.

Regards,

Paul Schenkeveld

Paul Schenkeveld wrote:
> Hello,
> 
> When I try to copy a large (7GB+) file from one filesystem to another the
> copy is not equal to te source file.  This behaviour can be reproduced
> repeatedly.
> 
> Hope someone can shed a light on this.
> 
I had a slot 1 board that exhibited a similar problem - changing the cpu 
heatsink so it didn't overhang the north bridge, cured the problem (i.e. 
letting the north bridge see some cool air).

While the *cpu* heatsink may not be your problem, you might want to 
either check your chipset temperature(s), or install heatsinks on them 
(two promise cards in the 66Mhz PCI slots can shift quite a lot of data, 
if your working both of 'em!).

Cheers

Mark

On 5/13/06, Paul Schenkeveld <fb-stable@psconsult.nl>
wrote:
> Hello,
>
> When I try to copy a large (7GB+) file from one filesystem to another the
> copy is not equal to te source file.  This behaviour can be reproduced
> repeatedly.
>
> Details:
>
>   - FreeBSD 6-STABLE, cvsupped yesterday
Hi!

I have very similar problem with my i386 6.1-STABLE installation. With
6.0 there were no problems. I have already eliminated bad hardware
(memory, hard drive) and also overheating. I'm able to reproduce the
problem with quite small files. 23 megabytes has been enough to get
corrupted. I have softupdates enabled and the disk is an IDE drive.

There however is one small difference compared to your situation. I
had a Java package in the root partition and after upgrading from 6.0
to 6.1 I reinstalled the package. The package was corrupted even
though I had used that same package before to install the Java
run-time to the 6.0 successfully. I haven't touched the file in any
way between the installations. I also has experienced multiple
corruptions while copying files. Usually I have to copy a single large
file three or four times to get it without problems. With smaller
files (less than 20 megs), there hasn't been any problems.

Next I'm going to downgrade my installation back to the 6.0 and see if
the problem will go away.

I'd be interested to hear if anyone else has experienced same sort of
problems with 6.1.


Sincerely,

        Matti