Mike Futerko
2008-Oct-16 11:05 UTC
[zfs-discuss] Strange result when syncing between SPARC and x86
Hello Today I''ve suddenly noticed that symlinks (at least) are corrupted when sync ZFS from SPARC to x86 (zfs send | ssh | zfs recv). Example is: root at sparc# ls -la /data/zones/testfs/root/etc/services lrwxrwxrwx 1 root root 15 Oct 13 14:35 /data/zones/testfs/root/etc/services -> ./inet/services root at x86# ls -la /data/zones/testfs/root/etc/services lrwxrwxrwx 1 root root 15 Oct 13 14:35 /data/zones/testfs/root/etc/services -> s/teni/.ervices Firstly I thought it''s because original FS on SPARC is compressed... so I''ve just synced it locally on same machine and all was OK just different FS size since destination was not compressed. Then I''ve synced that copy again to x86 but result was same - symlinks were corrupted... so it''s not compression. SPARC is snv_85 and x86 snv_82, I haven''t got a chance yet to test on latest OpenSolaris. Any suggestions? Thanks Mike
Casper.Dik at Sun.COM
2008-Oct-16 11:08 UTC
[zfs-discuss] Strange result when syncing between SPARC and x86
>Hello > > >Today I''ve suddenly noticed that symlinks (at least) are corrupted when >sync ZFS from SPARC to x86 (zfs send | ssh | zfs recv). > >Example is: > >root at sparc# ls -la /data/zones/testfs/root/etc/services >lrwxrwxrwx 1 root root 15 Oct 13 14:35 >/data/zones/testfs/root/etc/services -> ./inet/services > >root at x86# ls -la /data/zones/testfs/root/etc/services >lrwxrwxrwx 1 root root 15 Oct 13 14:35 >/data/zones/testfs/root/etc/services -> s/teni/.ervices > > >Firstly I thought it''s because original FS on SPARC is compressed... so >I''ve just synced it locally on same machine and all was OK just >different FS size since destination was not compressed. > >Then I''ve synced that copy again to x86 but result was same - symlinks >were corrupted... so it''s not compression. > >SPARC is snv_85 and x86 snv_82, I haven''t got a chance yet to test on >latest OpenSolaris. > > >Any suggestions?Looks like the first 8 bytes aren''t "reversed". Casper
Mike Futerko
2008-Oct-16 11:18 UTC
[zfs-discuss] Strange result when syncing between SPARC and x86
Hi Just checked with snv_99 on x86 (VMware install) - same result :( Regards Mike Casper.Dik at Sun.COM wrote:>> Hello >> >> >> Today I''ve suddenly noticed that symlinks (at least) are corrupted when >> sync ZFS from SPARC to x86 (zfs send | ssh | zfs recv). >> >> Example is: >> >> root at sparc# ls -la /data/zones/testfs/root/etc/services >> lrwxrwxrwx 1 root root 15 Oct 13 14:35 >> /data/zones/testfs/root/etc/services -> ./inet/services >> >> root at x86# ls -la /data/zones/testfs/root/etc/services >> lrwxrwxrwx 1 root root 15 Oct 13 14:35 >> /data/zones/testfs/root/etc/services -> s/teni/.ervices >> >> >> Firstly I thought it''s because original FS on SPARC is compressed... so >> I''ve just synced it locally on same machine and all was OK just >> different FS size since destination was not compressed. >> >> Then I''ve synced that copy again to x86 but result was same - symlinks >> were corrupted... so it''s not compression. >> >> SPARC is snv_85 and x86 snv_82, I haven''t got a chance yet to test on >> latest OpenSolaris. >> >> >> Any suggestions? > > Looks like the first 8 bytes aren''t "reversed". > > Casper >
Turanga Leela
2008-Oct-29 12:12 UTC
[zfs-discuss] Strange result when syncing between SPARC and x86
> Example is: > > root at sparc# ls -la > /data/zones/testfs/root/etc/services > lrwxrwxrwx 1 root root 15 Oct 13 14:35 > /data/zones/testfs/root/etc/services -> > ./inet/services > > root at x86# ls -la /data/zones/testfs/root/etc/services > lrwxrwxrwx 1 root root 15 Oct 13 14:35 > /data/zones/testfs/root/etc/services -> > s/teni/.ervicesOuch, thats a bad one. I downloaded and burnt b101 to dvd for x86 and solaris, i''m gonna install them tomorrow at work and try moving a pool between them to see what happens... -- This message posted from opensolaris.org
Mike Futerko
2008-Oct-29 14:13 UTC
[zfs-discuss] Strange result when syncing between SPARC and x86
Hi>> root at sparc# ls -la >> /data/zones/testfs/root/etc/services >> lrwxrwxrwx 1 root root 15 Oct 13 14:35 >> /data/zones/testfs/root/etc/services -> >> ./inet/services >> >> root at x86# ls -la /data/zones/testfs/root/etc/services >> lrwxrwxrwx 1 root root 15 Oct 13 14:35 >> /data/zones/testfs/root/etc/services -> >> s/teni/.ervices > > Ouch, thats a bad one. > > I downloaded and burnt b101 to dvd for x86 and solaris, i''m gonna install them tomorrow at work and try moving a pool between them to see what happens...Would be interesting to know how it''ll work if move whole zpool not just sync with send/recv. But I think all will be fine there as is seems the problem is in send/recv part on the file system itself on different architectures. Thanks Mike
Turanga Leela
2008-Nov-02 23:44 UTC
[zfs-discuss] Strange result when syncing between SPARC and x86
[b]Set up for test:[/b]
(I picked v8 for no particular reason.... this linux install can speak zfs v3
and zpool v13, as can our solaris b101 installs, but some of the older solaris
boxes we have can''t I guess... so... I picked 8? I want to use this my
little 500gb external ZFS disk. But first some testing!)
sdd is an external USB drive.
from dmesg:
usb-storage: device found at 11
usb-storage: waiting for device to settle before scanning
usb-storage: device scan complete
scsi 7:0:0:0: Direct-Access WDC WD50 00AAKB-00UKA0 PQ: 0 ANSI: 0
sd 7:0:0:0: [sdd] 976773168 512-byte hardware sectors (500108 MB)
sd 7:0:0:0: [sdd] Write Protect is off
sd 7:0:0:0: [sdd] Mode Sense: 33 00 00 00
sd 7:0:0:0: [sdd] Assuming drive cache: write through
sd 7:0:0:0: [sdd] 976773168 512-byte hardware sectors (500108 MB)
sd 7:0:0:0: [sdd] Write Protect is off
sd 7:0:0:0: [sdd] Mode Sense: 33 00 00 00
sd 7:0:0:0: [sdd] Assuming drive cache: write through
sdd: sdd1
sd 7:0:0:0: [sdd] Attached SCSI disk
sd 7:0:0:0: Attached scsi generic sg4 type 0
ganymede:~# uname -a
Linux ganymede 2.6.27.4-51.fc10.x86_64 #1 SMP Sun Oct 26 20:40:00 EDT 2008
x86_64 x86_64 x86_64 GNU/Linux
ganymede:~# zpool create -f external -o version=8 sdd
ganymede:~# zpool status
pool: external
state: ONLINE
status: The pool is formatted using an older on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using ''zpool upgrade''. Once this is
done, the
pool will no longer be accessible on older software versions.
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
external ONLINE 0 0 0
sdd ONLINE 0 0 0
errors: No known data errors
ganymede:~# zfs set compression=gzip-9 external
ganymede:~# cd /external/
ganymede:/external#
ganymede:/external# ls -al
total 6
drwxr-xr-x 2 root root 2 2008-10-31 12:20 .
drwxr-xr-x 26 root root 4096 2008-10-31 12:20 ..
ganymede:/external# touch testfile
ganymede:/external# ln -s testfile a_symlink
ganymede:/external# ln -s /to/some/random/location/to/see/what/happens
another_symlink
ganymede:/external# ls -al
total 7
drwxr-xr-x 2 root root 5 2008-10-31 12:25 .
drwxr-xr-x 26 root root 4096 2008-10-31 12:20 ..
lrwxrwxrwx 1 root root 44 2008-10-31 12:25 another_symlink ->
/to/some/random/location/to/see/what/happens
lrwxrwxrwx 1 root root 8 2008-10-31 12:24 a_symlink -> testfile
-rw-rw-rw- 1 root root 0 2008-10-31 12:24 testfile
ganymede:/external# zfs snapshot external at sendrecvtest
ganymede:/external# zfs list
NAME USED AVAIL REFER MOUNTPOINT
external 64K 457G 19K /external
external at sendrecvtest 0 - 19K -
ganymede:/external# zpool status
pool: external
state: ONLINE
status: The pool is formatted using an older on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using ''zpool upgrade''. Once this is
done, the
pool will no longer be accessible on older software versions.
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
external ONLINE 0 0 0
sdd ONLINE 0 0 0
errors: No known data errors
ganymede:/external#
ganymede:/external# cd /
ganymede:/# zpool export external
ganymede:/#
[b]Now we boot the b99 livecd import and look at it:[/b] (this is the same
machine as the linux box, ie. "ganymede")
jack at opensolaris:~$ pfexec zpool import
pool: external
id: 6611776765231431925
state: ONLINE
status: The pool is formatted using an older on-disk version.
action: The pool can be imported using its name or numeric identifier, though
some features will not be available without an explicit ''zpool
upgrade''.
config:
external ONLINE
c3t0d0p0 ONLINE
jack at opensolaris:~$ pfexec zpool import external
jack at opensolaris:~$ ls -al /external/
total 4
drwxr-xr-x 2 root root 5 2008-10-30 18:55 .
drwxr-xr-x 24 root root 512 2008-10-30 19:22 ..
lrwxrwxrwx 1 root root 44 2008-10-30 18:55 another_symlink ->
/to/some/random/location/to/see/what/happens
lrwxrwxrwx 1 root root 8 2008-10-30 18:54 a_symlink -> testfile
-rw-rw-rw- 1 root root 0 2008-10-30 18:54 testfile
jack at opensolaris:~$ pfexec zpool scrub external
jack at opensolaris:~$ pfexec zpool status
pool: external
state: ONLINE
status: The pool is formatted using an older on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using ''zpool upgrade''. Once this is
done, the
pool will no longer be accessible on older software versions.
scrub: scrub completed after 0h0m with 0 errors on Thu Oct 30 19:23:37 2008
config:
NAME STATE READ WRITE CKSUM
external ONLINE 0 0 0
c3t0d0p0 ONLINE 0 0 0
errors: No known data errors
jack at opensolaris:~$ pfexec zpool export external
jack at opensolaris:~$ uname -a
SunOS opensolaris 5.11 snv_99 i86pc i386 i86pc Solaris
jack at opensolaris:~$
[b]Now we let the sparc b101 look at it:[/b]
sol:~# uname -a
SunOS sol 5.11 snv_101 sun4u sparc SUNW,Sun-Blade-100
sol:~# zpool import
pool: external
id: 6611776765231431925
state: UNAVAIL
status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.
see: http://www.sun.com/msg/ZFS-8000-5E
config:
external UNAVAIL insufficient replicas
c0t0d0s0 UNAVAIL corrupted data
sol:~#
Hmm, thats weird? :(
Sol is actually a SunBlade 150 which prtdiag displays correctly, but for some
reason prtconf and uname display 100. *shrug*.
Any idea''s why it can''t import the pool on an external USB
drive when intel can?
[b]Lets try a send/recv from sparc b101 to x86 (linux):[/b]
Couldn''t do this since the sparc wouldn''t import it.
[b]Lets try a send/recv from sparc b101 to x86 b101:[/b]
Couldn''t do this since the sparc wouldn''t import it.
[b]Lets connect it back to the linux/x86 box and try to send/recv to sparc
b101:[/b]
ganymede:~# zpool import
pool: external
id: 6611776765231431925
state: ONLINE
status: The pool is formatted using an older on-disk version.
action: The pool can be imported using its name or numeric identifier, though
some features will not be available without an explicit ''zpool
upgrade''.
config:
external ONLINE
sdd ONLINE
ganymede:~# zpool import external
ganymede:~# zpool scrub external
ganymede:~# zpool status
pool: external
state: ONLINE
status: The pool is formatted using an older on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using ''zpool upgrade''. Once this is
done, the
pool will no longer be accessible on older software versions.
scrub: scrub completed after 0h0m with 0 errors on Mon Nov 3 09:42:49 2008
config:
NAME STATE READ WRITE CKSUM
external ONLINE 0 0 0
sdd ONLINE 0 0 0
errors: No known data errors
ganymede:~# uname -a
Linux ganymede 2.6.27.4-51.fc10.x86_64 #1 SMP Sun Oct 26 20:40:00 EDT 2008
x86_64 x86_64 x86_64 GNU/Linux
ganymede:~#
No problems.... ok, send/recv.
ganymede:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
external 106K 457G 19K /external
external at sendrecvtest 16K - 19K -
ganymede:~# zfs send external at sendrecvtest | ssh root at sol "zfs
receive -Fdv sol/external"
Password:
receiving full stream of external at sendrecvtest into sol/external at
sendrecvtest
received 16.3KB stream in 1 seconds (16.3KB/sec)
ganymede:~#
ganymede:~$ ssh sol
Last login: Mon Nov 3 09:34:50 2008 from ganymede.archim
Sun Microsystems Inc. SunOS 5.11 snv_101 November 2008
sol:~> s
sol:~# zfs create sol/external
sol:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
sol 19.0G 17.7G 155K /sol
sol/ROOT 12.0G 17.7G 18K legacy
sol/ROOT/snv_101 12.0G 17.7G 11.8G /
sol/ROOT/snv_101/var 111M 17.7G 99.8M /var
sol/dump 6.00G 17.7G 6.00G -
sol/external 28.5K 17.7G 28.5K /sol/external
sol/swap 1G 18.5G 172M -
sol:~# cd /sol/external/
sol:/sol/external# l
total 3
lrwxrwxrwx 1 root root 8 Oct 31 12:24 a_symlink -> testfile
lrwxrwxrwx 1 root root 44 Oct 31 12:25 another_symlink ->
/to/some/random/location/to/see/what/happens
-rw-rw-rw- 1 root root 0 Oct 31 12:24 testfile
sol:/sol/external#
Seemed to be okay? At least linux 64bit zfs-fuse 0.5.0 to sparc b101.
[b]Lets connect it to solaris b99 (live cd) and send/recv to sparc b101:[/b]
Prepare for next test:
sol:~# zfs list -r -t all
NAME USED AVAIL REFER MOUNTPOINT
sol 19.0G 17.7G 156K /sol
sol/ROOT 12.0G 17.7G 18K legacy
sol/ROOT/snv_101 12.0G 17.7G 11.8G /
sol/ROOT/snv_101 at installed 25.1M - 11.8G -
sol/ROOT/snv_101 at configured 3.45M - 11.8G -
sol/ROOT/snv_101/var 111M 17.7G 99.8M /var
sol/ROOT/snv_101/var at installed 11.2M - 99.6M -
sol/ROOT/snv_101/var at configured 406K - 99.8M -
sol/dump 6.00G 17.7G 6.00G -
sol/external 28.5K 17.7G 28.5K /sol/external
sol/external at sendrecvtest 0 - 28.5K -
sol/swap 1G 18.5G 172M -
sol:~# zfs destroy sol/external at sendrecvtest
sol:~#
ganymede:~# zpool export external
ganymede:~#
*reboot ganymede into livecd*
Okay, so I forgot my livecd, its at home. Nevertheless, I assume it would work
since the previous test worked fine. So perhaps that bug was fixed in a recent
build?
So now a different question, why did the sparc fail to import my pool?
--
This message posted from opensolaris.org