After a crash i cannot access one of my datasets anymore. ls -v cts brwxrwxrwx+ 2 root root 0, 0 ott 18 2009 cts zfs list sas/mail-cts NAME USED AVAIL REFER MOUNTPOINT sas/mail-cts 149G 250G 149G /sas/mail-cts as you can see, the space is referenced by this dataset, but i cannot access the directory /sas/mail-cts the system is: SunOS disk-01 5.11 snv_111b i86pc i386 i86pc Solaris whit zdb i can see the blocks referenced by this dataset. Any ideas?? Valerio Piancastelli piancastelli at iclos.com
On Mon, 20 Sep 2010, Valerio Piancastelli wrote:> After a crash i cannot access one of my datasets anymore. > > ls -v cts > brwxrwxrwx+ 2 root root 0, 0 ott 18 2009 cts > > zfs list sas/mail-cts > NAME USED AVAIL REFER MOUNTPOINT > sas/mail-cts 149G 250G 149G /sas/mail-cts > > as you can see, the space is referenced by this dataset, but i cannot access the directory /sas/mail-ctsIs the dataset mounted? i.e. what does ''zfs get mounted sas/mail-cts'' show?
Yes, it is mounted root at disk-00:/volumes/store# zfs get sas/mail-ccts NAME PROPERTY VALUE SOURCE sas/mail-cts mounted yes - Valerio Piancastelli +39 348 8072760 piancastelli at iclos.com ----- Messaggio originale ----- Da: "Mark J Musante" <Mark.Musante at oracle.com> A: "Valerio Piancastelli" <piancastelli at iclos.com> Cc: "zfs-discuss" <zfs-discuss at opensolaris.org> Inviato: Luned?, 20 settembre 2010 17:00:32 Oggetto: Re: [zfs-discuss] Cannot access dataset On Mon, 20 Sep 2010, Valerio Piancastelli wrote:> After a crash i cannot access one of my datasets anymore. > > ls -v cts > brwxrwxrwx+ 2 root root 0, 0 ott 18 2009 cts > > zfs list sas/mail-cts > NAME USED AVAIL REFER MOUNTPOINT > sas/mail-cts 149G 250G 149G /sas/mail-cts > > as you can see, the space is referenced by this dataset, but i cannot access the directory /sas/mail-ctsIs the dataset mounted? i.e. what does ''zfs get mounted sas/mail-cts'' show?
On Mon, 20 Sep 2010, Valerio Piancastelli wrote:> Yes, it is mounted > > root at disk-00:/volumes/store# zfs get sas/mail-ccts > NAME PROPERTY VALUE SOURCE > sas/mail-cts mounted yes -OK - so the next question would be where the data is. I assume when you say you "cannot access" the dataset, it means when you type ls -l /sas/mail-cts it shows up as an empty directory. Is that true? With luck, the data will still be in a snapshot. Given that the dataset has 149G referenced, it could be all there. Does ''zfs list -rt snapshot sas/mail-cts'' list any? If so, you can try using the most recent snapshot by looking in /sas/mail-cts/.zfs/snapshot/<snapshot name> and seeing if all your data are there. If it looks good, you can zfs rollback to that snapshot.
Unfortunately not. When i do # /usr/bin/ls -lv /sas/mail-cts brwxrwxrwx 2 777 root 0, 0 Oct 18 2009 /volumes/store/nfs/ICLOS/prod/mail-cts it seem a block device: # stat /sas/mail-cts File: `/sas/mail-cts'' Size: 3 Blocks: 3 IO Block: 512 block special file Device: 2d90062h/47775842d Inode: 3 Links: 2 Device type: 0,0 Access: (0777/brwxrwxrwx) Uid: ( 777/ UNKNOWN) Gid: ( 0/ root) Access: 2009-10-18 00:11:29.526578221 +0200 Modify: 2009-10-18 00:49:05.501732926 +0200 Change: 2010-09-17 19:32:10.113622993 +0200 if i do # /usr/bin/ls -lv /sas/mail-cts/ /usr/bin/ls: /volumes/store/nfs/ICLOS/prod/mail-cts/: Not a directory # stat /sas/mail-cts/ stat: cannot stat `/sas/mail-cts/'': Not a directory it seems that "something" turned the directory in block file Valerio Piancastelli +39 348 8072760 piancastelli at iclos.com ----- Messaggio originale ----- Da: "Mark J Musante" <Mark.Musante at oracle.com> A: "Valerio Piancastelli" <piancastelli at iclos.com> Cc: "zfs-discuss" <zfs-discuss at opensolaris.org> Inviato: Luned?, 20 settembre 2010 17:18:01 Oggetto: Re: [zfs-discuss] Cannot access dataset On Mon, 20 Sep 2010, Valerio Piancastelli wrote:> Yes, it is mounted > > root at disk-00:/volumes/store# zfs get sas/mail-ccts > NAME PROPERTY VALUE SOURCE > sas/mail-cts mounted yes -OK - so the next question would be where the data is. I assume when you say you "cannot access" the dataset, it means when you type ls -l /sas/mail-cts it shows up as an empty directory. Is that true? With luck, the data will still be in a snapshot. Given that the dataset has 149G referenced, it could be all there. Does ''zfs list -rt snapshot sas/mail-cts'' list any? If so, you can try using the most recent snapshot by looking in /sas/mail-cts/.zfs/snapshot/<snapshot name> and seeing if all your data are there. If it looks good, you can zfs rollback to that snapshot.
On Sep 20, 2010, at 7:23 PM, Valerio Piancastelli wrote:> Unfortunately not. > > When i do > > # /usr/bin/ls -lv /sas/mail-cts > brwxrwxrwx 2 777 root 0, 0 Oct 18 2009 /volumes/store/nfs/ICLOS/prod/mail-cts > > it seem a block device:Yes,it looks like we have bad mode field value in the znode for root directory object of this filesystem. Recently there was similar issue, but there was different sent of bits set in the mode field, so ''ls -l'' was displaying directory as question marks. Can you provide ssh or Shared Shell (http://sun.com/123) access to your system? Regards Victor> > # stat /sas/mail-cts > File: `/sas/mail-cts'' > Size: 3 Blocks: 3 IO Block: 512 block special file > Device: 2d90062h/47775842d Inode: 3 Links: 2 Device type: 0,0 > Access: (0777/brwxrwxrwx) Uid: ( 777/ UNKNOWN) Gid: ( 0/ root) > Access: 2009-10-18 00:11:29.526578221 +0200 > Modify: 2009-10-18 00:49:05.501732926 +0200 > Change: 2010-09-17 19:32:10.113622993 +0200 > > if i do > > # /usr/bin/ls -lv /sas/mail-cts/ > /usr/bin/ls: /volumes/store/nfs/ICLOS/prod/mail-cts/: Not a directory > > # stat /sas/mail-cts/ > stat: cannot stat `/sas/mail-cts/'': Not a directory > > it seems that "something" turned the directory in block file > > > Valerio Piancastelli > +39 348 8072760 > piancastelli at iclos.com > > ----- Messaggio originale ----- > Da: "Mark J Musante" <Mark.Musante at oracle.com> > A: "Valerio Piancastelli" <piancastelli at iclos.com> > Cc: "zfs-discuss" <zfs-discuss at opensolaris.org> > Inviato: Luned?, 20 settembre 2010 17:18:01 > Oggetto: Re: [zfs-discuss] Cannot access dataset > > On Mon, 20 Sep 2010, Valerio Piancastelli wrote: > >> Yes, it is mounted >> >> root at disk-00:/volumes/store# zfs get sas/mail-ccts >> NAME PROPERTY VALUE SOURCE >> sas/mail-cts mounted yes - > > OK - so the next question would be where the data is. I assume when you > say you "cannot access" the dataset, it means when you type ls -l > /sas/mail-cts it shows up as an empty directory. Is that true? > > With luck, the data will still be in a snapshot. Given that the dataset > has 149G referenced, it could be all there. Does ''zfs list -rt snapshot > sas/mail-cts'' list any? If so, you can try using the most recent snapshot > by looking in /sas/mail-cts/.zfs/snapshot/<snapshot name> and seeing if > all your data are there. If it looks good, you can zfs rollback to that > snapshot. > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
in the original server, i have also a dataset that shows this #ls -l ?--------- ? ? ? ? ? pgsql2 I''m sending you the IP and credentials to have ssh access. Please notify me when you don''t need anymore Valerio Piancastelli +39 348 8072760 piancastelli at iclos.com ----- Messaggio originale ----- Da: "Victor Latushkin" <victor.latushkin at oracle.com> A: "Valerio Piancastelli" <piancastelli at iclos.com> Cc: "mark musante" <mark.musante at oracle.com>, "zfs-discuss" <zfs-discuss at opensolaris.org> Inviato: Luned?, 20 settembre 2010 18:37:09 Oggetto: Re: [zfs-discuss] Cannot access dataset On Sep 20, 2010, at 7:23 PM, Valerio Piancastelli wrote:> Unfortunately not. > > When i do > > # /usr/bin/ls -lv /sas/mail-cts > brwxrwxrwx 2 777 root 0, 0 Oct 18 2009 /volumes/store/nfs/ICLOS/prod/mail-cts > > it seem a block device:Yes,it looks like we have bad mode field value in the znode for root directory object of this filesystem. Recently there was similar issue, but there was different sent of bits set in the mode field, so ''ls -l'' was displaying directory as question marks. Can you provide ssh or Shared Shell (http://sun.com/123) access to your system? Regards Victor> > # stat /sas/mail-cts > File: `/sas/mail-cts'' > Size: 3 Blocks: 3 IO Block: 512 block special file > Device: 2d90062h/47775842d Inode: 3 Links: 2 Device type: 0,0 > Access: (0777/brwxrwxrwx) Uid: ( 777/ UNKNOWN) Gid: ( 0/ root) > Access: 2009-10-18 00:11:29.526578221 +0200 > Modify: 2009-10-18 00:49:05.501732926 +0200 > Change: 2010-09-17 19:32:10.113622993 +0200 > > if i do > > # /usr/bin/ls -lv /sas/mail-cts/ > /usr/bin/ls: /volumes/store/nfs/ICLOS/prod/mail-cts/: Not a directory > > # stat /sas/mail-cts/ > stat: cannot stat `/sas/mail-cts/'': Not a directory > > it seems that "something" turned the directory in block file > > > Valerio Piancastelli > +39 348 8072760 > piancastelli at iclos.com > > ----- Messaggio originale ----- > Da: "Mark J Musante" <Mark.Musante at oracle.com> > A: "Valerio Piancastelli" <piancastelli at iclos.com> > Cc: "zfs-discuss" <zfs-discuss at opensolaris.org> > Inviato: Luned?, 20 settembre 2010 17:18:01 > Oggetto: Re: [zfs-discuss] Cannot access dataset > > On Mon, 20 Sep 2010, Valerio Piancastelli wrote: > >> Yes, it is mounted >> >> root at disk-00:/volumes/store# zfs get sas/mail-ccts >> NAME PROPERTY VALUE SOURCE >> sas/mail-cts mounted yes - > > OK - so the next question would be where the data is. I assume when you > say you "cannot access" the dataset, it means when you type ls -l > /sas/mail-cts it shows up as an empty directory. Is that true? > > With luck, the data will still be in a snapshot. Given that the dataset > has 149G referenced, it could be all there. Does ''zfs list -rt snapshot > sas/mail-cts'' list any? If so, you can try using the most recent snapshot > by looking in /sas/mail-cts/.zfs/snapshot/<snapshot name> and seeing if > all your data are there. If it looks good, you can zfs rollback to that > snapshot. > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
On Sep 20, 2010, at 9:03 PM, Valerio Piancastelli wrote:> in the original server, i have also a dataset that shows this > > #ls -l > ?--------- ? ? ? ? ? pgsql2could you please show output of ''zdb -ddddd <dataset> 3'' ? victor> > I''m sending you the IP and credentials to have ssh access. > Please notify me when you don''t need anymore > > Valerio Piancastelli > +39 348 8072760 > piancastelli at iclos.com > > ----- Messaggio originale ----- > Da: "Victor Latushkin" <victor.latushkin at oracle.com> > A: "Valerio Piancastelli" <piancastelli at iclos.com> > Cc: "mark musante" <mark.musante at oracle.com>, "zfs-discuss" <zfs-discuss at opensolaris.org> > Inviato: Luned?, 20 settembre 2010 18:37:09 > Oggetto: Re: [zfs-discuss] Cannot access dataset > > > On Sep 20, 2010, at 7:23 PM, Valerio Piancastelli wrote: > >> Unfortunately not. >> >> When i do >> >> # /usr/bin/ls -lv /sas/mail-cts >> brwxrwxrwx 2 777 root 0, 0 Oct 18 2009 /volumes/store/nfs/ICLOS/prod/mail-cts >> >> it seem a block device: > > Yes,it looks like we have bad mode field value in the znode for root directory object of this filesystem. Recently there was similar issue, but there was different sent of bits set in the mode field, so ''ls -l'' was displaying directory as question marks. > > Can you provide ssh or Shared Shell (http://sun.com/123) access to your system? > > Regards > Victor > > >> >> # stat /sas/mail-cts >> File: `/sas/mail-cts'' >> Size: 3 Blocks: 3 IO Block: 512 block special file >> Device: 2d90062h/47775842d Inode: 3 Links: 2 Device type: 0,0 >> Access: (0777/brwxrwxrwx) Uid: ( 777/ UNKNOWN) Gid: ( 0/ root) >> Access: 2009-10-18 00:11:29.526578221 +0200 >> Modify: 2009-10-18 00:49:05.501732926 +0200 >> Change: 2010-09-17 19:32:10.113622993 +0200 >> >> if i do >> >> # /usr/bin/ls -lv /sas/mail-cts/ >> /usr/bin/ls: /volumes/store/nfs/ICLOS/prod/mail-cts/: Not a directory >> >> # stat /sas/mail-cts/ >> stat: cannot stat `/sas/mail-cts/'': Not a directory >> >> it seems that "something" turned the directory in block file >> >> >> Valerio Piancastelli >> +39 348 8072760 >> piancastelli at iclos.com >> >> ----- Messaggio originale ----- >> Da: "Mark J Musante" <Mark.Musante at oracle.com> >> A: "Valerio Piancastelli" <piancastelli at iclos.com> >> Cc: "zfs-discuss" <zfs-discuss at opensolaris.org> >> Inviato: Luned?, 20 settembre 2010 17:18:01 >> Oggetto: Re: [zfs-discuss] Cannot access dataset >> >> On Mon, 20 Sep 2010, Valerio Piancastelli wrote: >> >>> Yes, it is mounted >>> >>> root at disk-00:/volumes/store# zfs get sas/mail-ccts >>> NAME PROPERTY VALUE SOURCE >>> sas/mail-cts mounted yes - >> >> OK - so the next question would be where the data is. I assume when you >> say you "cannot access" the dataset, it means when you type ls -l >> /sas/mail-cts it shows up as an empty directory. Is that true? >> >> With luck, the data will still be in a snapshot. Given that the dataset >> has 149G referenced, it could be all there. Does ''zfs list -rt snapshot >> sas/mail-cts'' list any? If so, you can try using the most recent snapshot >> by looking in /sas/mail-cts/.zfs/snapshot/<snapshot name> and seeing if >> all your data are there. If it looks good, you can zfs rollback to that >> snapshot. >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Here it is:
root at disk-01:~# zdb -ddddd sas/mail-cts 3
Dataset sas/mail-cts [ZPL], ID 30, cr_txg 8, 149G, 6 objects, rootbp [L0 DMU
objset] 400L/200P DVA[0]=<0:1e00076000:200> DVA[1]=<0:a00085400:200>
fletcher4 lzjb LE contiguous birth=1314 fill=6
cksum=abcdfba24:473e07d1142:ef9bd7e3b1d9:220b0c2e1d1c63
Deadlist: 8 entries, 16K
Item 0: 0:a53183000:200 400L/200P F=0 B=954
Item 1: 0:a53182c00:400 4000L/400P F=0 B=954
Item 2: 0:a53182800:400 4000L/400P F=0 B=954
Item 3: 0:a53182400:400 4000L/400P F=0 B=954
Item 4: 0:a4767fc00:400 4000L/400P F=0 B=954
Item 5: 0:a4767f800:400 4000L/400P F=0 B=954
Item 6: 0:a4767f400:400 4000L/400P F=0 B=954
Item 7: 0:a53181e00:600 4000L/600P F=0 B=954
Object lvl iblk dblk lsize asize type
3 1 16K 512 512 1K ZFS directory
264 bonus ZFS znode
path /
uid 777
gid 0
atime Mon Sep 20 21:17:34 2010
mtime Mon Sep 20 21:17:17 2010
ctime Mon Sep 20 21:17:17 2010
crtime Sun Oct 18 00:11:29 2009
gen 444361
mode 40777
size 3
parent 3
links 2
xattr 0
rdev 0x0000000000000000
microzap: 512 bytes, 1 entries
vdisk.raw = 7 (type: Regular File)
Indirect blocks:
0 L0 0:d600:200 200L/200P F=1 B=9
segment [0000000000000000, 0000000000000200) size 512
Valerio Piancastelli
piancastelli at iclos.com
----- Messaggio originale -----
Da: "Victor Latushkin" <victor.latushkin at oracle.com>
A: "Valerio Piancastelli" <piancastelli at iclos.com>
Cc: "zfs-discuss" <zfs-discuss at opensolaris.org>
Inviato: Luned?, 20 settembre 2010 22:37:29
Oggetto: Re: [zfs-discuss] Cannot access dataset
On Sep 20, 2010, at 9:03 PM, Valerio Piancastelli wrote:
> in the original server, i have also a dataset that shows this
>
> #ls -l
> ?--------- ? ? ? ? ? pgsql2
could you please show output of ''zdb -ddddd <dataset> 3''
?
victor
>
> I''m sending you the IP and credentials to have ssh access.
> Please notify me when you don''t need anymore
>
> Valerio Piancastelli
> +39 348 8072760
> piancastelli at iclos.com
>
> ----- Messaggio originale -----
> Da: "Victor Latushkin" <victor.latushkin at oracle.com>
> A: "Valerio Piancastelli" <piancastelli at iclos.com>
> Cc: "mark musante" <mark.musante at oracle.com>,
"zfs-discuss" <zfs-discuss at opensolaris.org>
> Inviato: Luned?, 20 settembre 2010 18:37:09
> Oggetto: Re: [zfs-discuss] Cannot access dataset
>
>
> On Sep 20, 2010, at 7:23 PM, Valerio Piancastelli wrote:
>
>> Unfortunately not.
>>
>> When i do
>>
>> # /usr/bin/ls -lv /sas/mail-cts
>> brwxrwxrwx 2 777 root 0, 0 Oct 18 2009
/volumes/store/nfs/ICLOS/prod/mail-cts
>>
>> it seem a block device:
>
> Yes,it looks like we have bad mode field value in the znode for root
directory object of this filesystem. Recently there was similar issue, but there
was different sent of bits set in the mode field, so ''ls -l''
was displaying directory as question marks.
>
> Can you provide ssh or Shared Shell (http://sun.com/123) access to your
system?
>
> Regards
> Victor
>
>
>>
>> # stat /sas/mail-cts
>> File: `/sas/mail-cts''
>> Size: 3 Blocks: 3 IO Block: 512 block special
file
>> Device: 2d90062h/47775842d Inode: 3 Links: 2 Device
type: 0,0
>> Access: (0777/brwxrwxrwx) Uid: ( 777/ UNKNOWN) Gid: ( 0/
root)
>> Access: 2009-10-18 00:11:29.526578221 +0200
>> Modify: 2009-10-18 00:49:05.501732926 +0200
>> Change: 2010-09-17 19:32:10.113622993 +0200
>>
>> if i do
>>
>> # /usr/bin/ls -lv /sas/mail-cts/
>> /usr/bin/ls: /volumes/store/nfs/ICLOS/prod/mail-cts/: Not a directory
>>
>> # stat /sas/mail-cts/
>> stat: cannot stat `/sas/mail-cts/'': Not a directory
>>
>> it seems that "something" turned the directory in block file
>>
>>
>> Valerio Piancastelli
>> +39 348 8072760
>> piancastelli at iclos.com
>>
>> ----- Messaggio originale -----
>> Da: "Mark J Musante" <Mark.Musante at oracle.com>
>> A: "Valerio Piancastelli" <piancastelli at iclos.com>
>> Cc: "zfs-discuss" <zfs-discuss at opensolaris.org>
>> Inviato: Luned?, 20 settembre 2010 17:18:01
>> Oggetto: Re: [zfs-discuss] Cannot access dataset
>>
>> On Mon, 20 Sep 2010, Valerio Piancastelli wrote:
>>
>>> Yes, it is mounted
>>>
>>> root at disk-00:/volumes/store# zfs get sas/mail-ccts
>>> NAME PROPERTY VALUE SOURCE
>>> sas/mail-cts mounted yes -
>>
>> OK - so the next question would be where the data is. I assume when
you
>> say you "cannot access" the dataset, it means when you type
ls -l
>> /sas/mail-cts it shows up as an empty directory. Is that true?
>>
>> With luck, the data will still be in a snapshot. Given that the
dataset
>> has 149G referenced, it could be all there. Does ''zfs list
-rt snapshot
>> sas/mail-cts'' list any? If so, you can try using the most
recent snapshot
>> by looking in /sas/mail-cts/.zfs/snapshot/<snapshot name> and
seeing if
>> all your data are there. If it looks good, you can zfs rollback to
that
>> snapshot.
>> _______________________________________________
>> zfs-discuss mailing list
>> zfs-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
with ls -li i have no inode number :
#ls -li
? ?--------- ? ? ? ? ? pgsql2
anyway :
# zdb -ddd store/nfs/APPS/prod/pgsql2
Dataset store/nfs/APPS/prod/pgsql2 [ZPL], ID 4595, cr_txg 96929, 18.8G, 6
objects
Deadlist: 0 (0/0 comp)
Object lvl iblk dblk dsize lsize %full type
0 7 16K 16K 15.0K 16K 18.75 DMU dnode
-1 1 16K 512 1K 512 100.00 ZFS user/group used
-2 1 16K 512 1K 512 100.00 ZFS user/group used
1 1 16K 512 1K 512 100.00 ZFS master node
2 1 16K 512 1K 512 100.00 ZFS delete queue
3 1 16K 512 1K 512 100.00 ZFS directory
4 1 16K 512 1K 512 100.00 ZFS directory
5 1 16K 512 1K 512 100.00 ZFS directory
6 4 16K 128K 18.8G 128G 14.70 ZFS plain file
and more verbose:
# zdb -dddd store/nfs/APPS/prod/pgsql2
Dataset store/nfs/APPS/prod/pgsql2 [ZPL], ID 4595, cr_txg 96929, 18.8G, 6
objects, rootbp DVA[0]=<1:322f45fe00:200> DVA[1]=<0:5514dba600:200>
[L0 DMU objset] fletcher4 lzjb LE contiguous unique double size=800L/200P
birth=177581L/177581P fill=6
cksum=170d0f10c3:78d4a37a631:1520d86b3b22d:29c8b0b6752f94
Deadlist: 0 (0/0 comp)
Object lvl iblk dblk dsize lsize %full type
0 7 16K 16K 15.0K 16K 18.75 DMU dnode
dnode flags: USED_BYTES
dnode maxblkid: 0
Object lvl iblk dblk dsize lsize %full type
-1 1 16K 512 1K 512 100.00 ZFS user/group used
dnode flags: USED_BYTES
dnode maxblkid: 0
microzap: 512 bytes, 2 entries
0 = 1536
309 = 20215323136
Object lvl iblk dblk dsize lsize %full type
-2 1 16K 512 1K 512 100.00 ZFS user/group used
dnode flags: USED_BYTES
dnode maxblkid: 0
microzap: 512 bytes, 2 entries
0 = 3072
ea61 = 20215321600
Object lvl iblk dblk dsize lsize %full type
1 1 16K 512 1K 512 100.00 ZFS master node
dnode flags: USED_BYTES USERUSED_ACCOUNTED
dnode maxblkid: 0
microzap: 512 bytes, 7 entries
SHARES = 4
normalization = 0
ROOT = 3
DELETE_QUEUE = 2
utf8only = 0
VERSION = 3
casesensitivity = 0
Object lvl iblk dblk dsize lsize %full type
2 1 16K 512 1K 512 100.00 ZFS delete queue
dnode flags: USED_BYTES USERUSED_ACCOUNTED
dnode maxblkid: 0
microzap: 512 bytes, 0 entries
Object lvl iblk dblk dsize lsize %full type
3 1 16K 512 1K 512 100.00 ZFS directory
264 bonus ZFS znode
dnode flags: USED_BYTES USERUSED_ACCOUNTED
dnode maxblkid: 0
path /
uid 777
gid 0
atime Fri Jun 26 13:13:46 2009
mtime Fri Jun 26 13:14:17 2009
ctime Wed Sep 1 15:02:24 2010
crtime Fri Jun 26 13:13:46 2009
gen 38459
mode 777
size 3
parent 3
links 3
pflags 40800000144
xattr 0
rdev 0x0000000000000000
microzap: 512 bytes, 1 entries
root = 5 (type: Directory)
Object lvl iblk dblk dsize lsize %full type
4 1 16K 512 1K 512 100.00 ZFS directory
264 bonus ZFS znode
dnode flags: USED_BYTES USERUSED_ACCOUNTED
dnode maxblkid: 0
path /
uid 0
gid 0
atime Fri Jun 26 13:13:46 2009
mtime Fri Jun 26 13:13:46 2009
ctime Fri Jun 26 13:13:46 2009
crtime Fri Jun 26 13:13:46 2009
gen 38459
mode 40555
size 2
parent 4
links 2
pflags 40800000044
xattr 0
rdev 0x0000000000000000
microzap: 512 bytes, 0 entries
Object lvl iblk dblk dsize lsize %full type
5 1 16K 512 1K 512 100.00 ZFS directory
264 bonus ZFS znode
dnode flags: USED_BYTES USERUSED_ACCOUNTED
dnode maxblkid: 0
path /root
uid 777
gid 60001
atime Fri Jun 26 13:14:17 2009
mtime Fri Jun 26 13:14:55 2009
ctime Wed Sep 1 15:02:24 2010
crtime Fri Jun 26 13:14:17 2009
gen 38462
mode 40777
size 3
parent 3
links 2
pflags 40800000144
xattr 0
rdev 0x0000000000000000
microzap: 512 bytes, 1 entries
vdisk.vmdk = 6 (type: Regular File)
Object lvl iblk dblk dsize lsize %full type
6 4 16K 128K 18.8G 128G 14.70 ZFS plain file
264 bonus ZFS znode
dnode flags: USED_BYTES USERUSED_ACCOUNTED
dnode maxblkid: 1048576
path /root/vdisk.vmdk
uid 777
gid 60001
atime Fri Jun 26 13:14:55 2009
mtime Thu Sep 9 16:36:40 2010
ctime Thu Sep 9 16:36:40 2010
crtime Fri Jun 26 13:14:55 2009
gen 38464
mode 100777
size 137438953473
parent 5
links 1
pflags 40800000104
xattr 0
rdev 0x0000000000000000
which object# i have to further investigate?
Valerio Piancastelli
piancastelli at iclos.com
----- Messaggio originale -----
Da: "Victor Latushkin" <victor.latushkin at oracle.com>
A: "Valerio Piancastelli" <piancastelli at iclos.com>
Inviato: Luned?, 20 settembre 2010 22:52:00
Oggetto: Re: [zfs-discuss] Cannot access dataset
On Sep 21, 2010, at 12:42 AM, Valerio Piancastelli wrote:
> Here it is:
This is not what I meant, sorry..
>> in the original server, i have also a dataset that shows this
>>
>> #ls -l
>> ?--------- ? ? ? ? ? pgsql2
Could you do ''ls -li'' in the directory with pgsql2 and then
use reported inode number for as an object number in the ??? command (instead of
3). You need to specify dataset name where pgsql2 lives.
victor
>
> could you please show output of ''zdb -ddddd <dataset>
3'' ?
>
> victor
>
>
>>
>> I''m sending you the IP and credentials to have ssh access.
>> Please notify me when you don''t need anymore
>>
>> Valerio Piancastelli
>> +39 348 8072760
>> piancastelli at iclos.com
>>
>> ----- Messaggio originale -----
>> Da: "Victor Latushkin" <victor.latushkin at oracle.com>
>> A: "Valerio Piancastelli" <piancastelli at iclos.com>
>> Cc: "mark musante" <mark.musante at oracle.com>,
"zfs-discuss" <zfs-discuss at opensolaris.org>
>> Inviato: Luned?, 20 settembre 2010 18:37:09
>> Oggetto: Re: [zfs-discuss] Cannot access dataset
>>
>>
>> On Sep 20, 2010, at 7:23 PM, Valerio Piancastelli wrote:
>>
>>> Unfortunately not.
>>>
>>> When i do
>>>
>>> # /usr/bin/ls -lv /sas/mail-cts
>>> brwxrwxrwx 2 777 root 0, 0 Oct 18 2009
/volumes/store/nfs/ICLOS/prod/mail-cts
>>>
>>> it seem a block device:
>>
>> Yes,it looks like we have bad mode field value in the znode for root
directory object of this filesystem. Recently there was similar issue, but there
was different sent of bits set in the mode field, so ''ls -l''
was displaying directory as question marks.
>>
>> Can you provide ssh or Shared Shell (http://sun.com/123) access to your
system?
>>
>> Regards
>> Victor
>>
>>
>>>
>>> # stat /sas/mail-cts
>>> File: `/sas/mail-cts''
>>> Size: 3 Blocks: 3 IO Block: 512 block
special file
>>> Device: 2d90062h/47775842d Inode: 3 Links: 2
Device type: 0,0
>>> Access: (0777/brwxrwxrwx) Uid: ( 777/ UNKNOWN) Gid: ( 0/
root)
>>> Access: 2009-10-18 00:11:29.526578221 +0200
>>> Modify: 2009-10-18 00:49:05.501732926 +0200
>>> Change: 2010-09-17 19:32:10.113622993 +0200
>>>
>>> if i do
>>>
>>> # /usr/bin/ls -lv /sas/mail-cts/
>>> /usr/bin/ls: /volumes/store/nfs/ICLOS/prod/mail-cts/: Not a
directory
>>>
>>> # stat /sas/mail-cts/
>>> stat: cannot stat `/sas/mail-cts/'': Not a directory
>>>
>>> it seems that "something" turned the directory in block
file
>>>
>>>
>>> Valerio Piancastelli
>>> +39 348 8072760
>>> piancastelli at iclos.com
>>>
>>> ----- Messaggio originale -----
>>> Da: "Mark J Musante" <Mark.Musante at oracle.com>
>>> A: "Valerio Piancastelli" <piancastelli at
iclos.com>
>>> Cc: "zfs-discuss" <zfs-discuss at opensolaris.org>
>>> Inviato: Luned?, 20 settembre 2010 17:18:01
>>> Oggetto: Re: [zfs-discuss] Cannot access dataset
>>>
>>> On Mon, 20 Sep 2010, Valerio Piancastelli wrote:
>>>
>>>> Yes, it is mounted
>>>>
>>>> root at disk-00:/volumes/store# zfs get sas/mail-ccts
>>>> NAME PROPERTY VALUE SOURCE
>>>> sas/mail-cts mounted yes -
>>>
>>> OK - so the next question would be where the data is. I assume
when you
>>> say you "cannot access" the dataset, it means when you
type ls -l
>>> /sas/mail-cts it shows up as an empty directory. Is that true?
>>>
>>> With luck, the data will still be in a snapshot. Given that the
dataset
>>> has 149G referenced, it could be all there. Does ''zfs
list -rt snapshot
>>> sas/mail-cts'' list any? If so, you can try using the most
recent snapshot
>>> by looking in /sas/mail-cts/.zfs/snapshot/<snapshot name> and
seeing if
>>> all your data are there. If it looks good, you can zfs rollback to
that
>>> snapshot.
>>> _______________________________________________
>>> zfs-discuss mailing list
>>> zfs-discuss at opensolaris.org
>>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>>
>> _______________________________________________
>> zfs-discuss mailing list
>> zfs-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Many thanks again. I''m leaving datasets there till u need. Let me know if i can help you in something. Bye Valerio Piancastelli piancastelli at iclos.com ----- Messaggio originale ----- Da: "Victor Latushkin" <victor.latushkin at oracle.com> A: "Valerio Piancastelli" <piancastelli at iclos.com> Inviato: Marted?, 21 settembre 2010 0:37:57 Oggetto: Re: [zfs-discuss] Cannot access dataset On Sep 21, 2010, at 2:28 AM, Valerio Piancastelli wrote:> Object lvl iblk dblk dsize lsize %full type > 3 1 16K 512 1K 512 100.00 ZFS directory > 264 bonus ZFS znode > dnode flags: USED_BYTES USERUSED_ACCOUNTED > dnode maxblkid: 0 > path / > uid 777 > gid 0 > atime Fri Jun 26 13:13:46 2009 > mtime Fri Jun 26 13:14:17 2009 > ctime Wed Sep 1 15:02:24 2010 > crtime Fri Jun 26 13:13:46 2009 > gen 38459 > mode 777Looks like mode on the root directory here is again wrong.. So far we have a pattern - in at least three cases the problem is related to mode value of the filesystem root directory Victor