zfs discuss - Apr 2010 - Mapping inode numbers to file names

Let''s suppose you rename a file or directory.

/tank/widgets/a/rel2049_773.13-4/somefile.txt

Becomes

/tank/widgets/b/foogoo_release_1.9/README

 

Let''s suppose you are now working on widget B, and you want to look at
the
past zfs snapshot of README, but you don''t remember where it came from.
That is, you don''t know the previous name or location where that file
used
to be.  One way you could do it would be:

 

Look up the inode number of README.  (for example, ls -i README)

                (suppose it''s inode 12345)

find /tank/.zfs/snapshot -inum 12345

 

Problem is, the find command will run for a long time.

 

Is there any faster way to find the file name(s) when all you know is the
inode number?  (Actually, all you know is all the info that''s in the
present
directory, which is not limited to inode number; but, inode number is the
only information that I personally know could be useful.)

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100427/915d1ae1/attachment.html>

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Edward Ned Harvey
> 
> Look up the inode number of README.? (for example, ls -i README)
> ??????????????? (suppose it?s inode 12345)
> find /tank/.zfs/snapshot -inum 12345
> 
> Problem is, the find command will run for a long time.
> 
> Is there any faster way to find the file name(s) when all you know is
> the inode number?? (Actually, all you know is all the info that?s in
> the present directory, which is not limited to inode number; but, inode
> number is the only information that I personally know could be useful.)
Due to lack of response, and based on my personal knowledge, and lack of any
useful response anywhere else I''ve asked this question, I''m
drawing the
conclusion it''s not possible to quickly lookup the name(s) of an inode.

Which means in ZFS, if you want to find the previous snapshots of some
file/directory that was renamed/moved, there are only two possibilities:

Assume the name and location didn''t change.  So you can quickly
"ls" or
"stat" specific already-known name(s) in the snapshot directories.
- or -
Walk all the trees of all the snapshots searching for inode X.  Which would
be probably extremely slow.  (Probably many hours to complete, in a typical
small business fileserver, linearly dependent on number of files in the
filesystem and number of snapshots to traverse.)

BTW, this is not an issue for Ontap.  Which is to say, yes it''s
absolutely
possible to maintain reverse inode lookup tables too, but apparently not
implemented in ... other filesystems including ZFS.

I think an inode --> name lookup table would be useful in something like
zhist.  And it''s one way of solving this problem without violating any
Ontap-style .snapshot directory patents.  So maybe this would be a good
thing to request in future versions of ZFS.

On 28.04.10 14:06, Edward Ned Harvey wrote:
>> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
>> bounces at opensolaris.org] On Behalf Of Edward Ned Harvey
>>
>> Look up the inode number of README.  (for example, ls -i README)
>>                  (suppose it?s inode 12345)
>> find /tank/.zfs/snapshot -inum 12345
>>
>> Problem is, the find command will run for a long time.
>>
>> Is there any faster way to find the file name(s) when all you know is
>> the inode number?  (Actually, all you know is all the info that?s in
>> the present directory, which is not limited to inode number; but, inode
>> number is the only information that I personally know could be useful.)
>
> Due to lack of response, and based on my personal knowledge, and lack of
any
> useful response anywhere else I''ve asked this question,
I''m drawing the
> conclusion it''s not possible to quickly lookup the name(s) of an
inode.
no - consider hard links. (and sorry for not answering sooner, this obvious 
one didn''t occur to me earlier).

Michael
-- 
michael.schuster at oracle.com	http://blogs.sun.com/recursion
Recursion, n.: see ''Recursion''

On 28 apr 2010, at 14.06, Edward Ned Harvey wrote:
>> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
>> bounces at opensolaris.org] On Behalf Of Edward Ned Harvey
>> 
>> Look up the inode number of README.  (for example, ls -i README)
>>                 (suppose it?s inode 12345)
>> find /tank/.zfs/snapshot -inum 12345
>> 
>> Problem is, the find command will run for a long time.
>> 
>> Is there any faster way to find the file name(s) when all you know is
>> the inode number?  (Actually, all you know is all the info that?s in
>> the present directory, which is not limited to inode number; but, inode
>> number is the only information that I personally know could be useful.)
> 
> Due to lack of response, and based on my personal knowledge, and lack of
any
> useful response anywhere else I''ve asked this question,
I''m drawing the
> conclusion it''s not possible to quickly lookup the name(s) of an
inode.
> 
> Which means in ZFS, if you want to find the previous snapshots of some
> file/directory that was renamed/moved, there are only two possibilities:
> 
> Assume the name and location didn''t change.  So you can quickly
"ls" or
> "stat" specific already-known name(s) in the snapshot
directories.
> - or -
> Walk all the trees of all the snapshots searching for inode X.  Which would
> be probably extremely slow.  (Probably many hours to complete, in a typical
> small business fileserver, linearly dependent on number of files in the
> filesystem and number of snapshots to traverse.)
> 
> BTW, this is not an issue for Ontap.  Which is to say, yes it''s
absolutely
> possible to maintain reverse inode lookup tables too, but apparently not
> implemented in ... other filesystems including ZFS.
What indicators do you have that ONTAP/WAFL has inode->name lookup
functionality? I don''t think it has any such thing - WAFL is pretty
much an UFS/FFS that does COW instead of in-place writing, the main
difference is that inodes are written to special inode files rather
than specific static areas. Directories I believe works very much like
UFS/FFS directories. But I may have misunderstood something and be
wrong.
> I think an inode --> name lookup table would be useful in something like
> zhist.  And it''s one way of solving this problem without violating
any
> Ontap-style .snapshot directory patents.  So maybe this would be a good
> thing to request in future versions of ZFS.
I must be missing something or maybe I am just plain stupid (both
are probable), and just out of curiosity: What is the exact problem
you are trying to solve? Providing a move/rename history? Is there
a specific use case?

/ragge

> From: Ragnar Sundblad [mailto:ragge at csc.kth.se]
> Sent: Wednesday, April 28, 2010 3:49 PM
> 
> What indicators do you have that ONTAP/WAFL has inode->name lookup
> functionality? 
I don''t have any such indicator, and if that''s the way my
words came out,
sorry for that.  Allow me to clarify:

In ontap, if you make a snapshot, and then move/rename directories all over
the place like crazy, and you''re now in the totally restructured
filesystem,
wanting to look at the previous version of some file or directory ... You
can still look in the .snapshot directory of the present filesystem, and you
can still see the contents of that subdirectory from a previous time.  There
is no difficulty trying to locate the old snapshots.

Actually, I''m glad you asked this question, because until now I assumed
that
meant ontap was somehow keeping track of "mv" operations, and linking
the
present .snapshot directory to whatever the old names used to be.  So you
could instantly get a listing of the present directory in a previous
filesystem snapshot, even though the present directory had a different name
or path name at some previous time.  But now that I''m explaining it out
loud, I realize that''s not necessary.

All you need to be able to do is to read the contents of inode X in a
previous snapshot, even though you might not know the name or pathname of
inode X within that snapshot.  Is this possible?  

Instead of "file = open(''/path/to/somefile'')"  Can
you "file openinode(12345)" instead?  Since you don''t
necessarily know the path to the
file, can you skip that step, and open an inode directly?  I''ll look
into
that, but I''ve never heard of it before, so if anyone already knows
how,
please share.

 
> I must be missing something or maybe I am just plain stupid (both
> are probable), and just out of curiosity: What is the exact problem
> you are trying to solve? Providing a move/rename history? Is there
> a specific use case?
Specific case:

In order to implement zhist, I''d very much like to instantly locate
previous
snapshot versions of a file or directory, even though you can''t assume
the
filename or path remained unchanged.

Let''s suppose you rename, move, or modify a file or directory.
	mkdir -p /tank/widgets/a/rel2049_773.13-4
	echo "hi" > /tank/widgets/a/rel2049_773.13-4/somefile.txt
	(create snapshot "mysnap")
	mkdir -p /tank/widgets/b/foogoo_release_1.9
	mv /tank/widgets/a/rel2049_773.13-4/somefile.txt
/tank/widgets/b/foogoo_release_1.9/README
	echo " there" >> /tank/widgets/b/foogoo_release_1.9/README

Let''s suppose you are now working on widget B, and you want to look at
the
past zfs snapshot of README, but you don''t remember where it came from.
That is, you don''t know the previous name or location where that file
used
to be.  One way you could do it would be:

Look up the inode number of README.  (for example, ls -i README)
                (suppose it''s inode 12345)
find /tank/.zfs/snapshot -inum 12345

It will produce the result eventually:
	/tank/.zfs/snapshot/mysnap/a/rel2049_773.13-4/somefile.txt

So at last, you can see the previous version of that file.  Unfortunately,
the find command will take a very long time.  You might grow gray hair.

On Apr 29, 2010, at 3:03 AM, Edward Ned Harvey wrote:
>> From: Ragnar Sundblad [mailto:ragge at csc.kth.se]
>> Sent: Wednesday, April 28, 2010 3:49 PM
>> 
>> What indicators do you have that ONTAP/WAFL has inode->name lookup
>> functionality? 
> 
> I don''t have any such indicator, and if that''s the way my
words came out,
> sorry for that.  Allow me to clarify:
> 
> In ontap, if you make a snapshot, and then move/rename directories all over
> the place like crazy, and you''re now in the totally restructured
filesystem,
> wanting to look at the previous version of some file or directory ... You
> can still look in the .snapshot directory of the present filesystem, and
you
> can still see the contents of that subdirectory from a previous time. 
There
> is no difficulty trying to locate the old snapshots.
> 
> Actually, I''m glad you asked this question, because until now I
assumed that
> meant ontap was somehow keeping track of "mv" operations, and
linking the
> present .snapshot directory to whatever the old names used to be.  So you
> could instantly get a listing of the present directory in a previous
> filesystem snapshot, even though the present directory had a different name
> or path name at some previous time.  But now that I''m explaining
it out
> loud, I realize that''s not necessary.
> 
> All you need to be able to do is to read the contents of inode X in a
> previous snapshot, even though you might not know the name or pathname of
> inode X within that snapshot.  Is this possible?  
> 
> Instead of "file = open(''/path/to/somefile'')" 
Can you "file > openinode(12345)" instead?  Since you
don''t necessarily know the path to the
> file, can you skip that step, and open an inode directly?  I''ll
look into
> that, but I''ve never heard of it before, so if anyone already
knows how,
> please share.
> 
> 
>> I must be missing something or maybe I am just plain stupid (both
>> are probable), and just out of curiosity: What is the exact problem
>> you are trying to solve? Providing a move/rename history? Is there
>> a specific use case?
> 
> Specific case:
> 
> In order to implement zhist, I''d very much like to instantly
locate previous
> snapshot versions of a file or directory, even though you can''t
assume the
> filename or path remained unchanged.
> 
> Let''s suppose you rename, move, or modify a file or directory.
> 	mkdir -p /tank/widgets/a/rel2049_773.13-4
> 	echo "hi" > /tank/widgets/a/rel2049_773.13-4/somefile.txt
> 	(create snapshot "mysnap")
> 	mkdir -p /tank/widgets/b/foogoo_release_1.9
> 	mv /tank/widgets/a/rel2049_773.13-4/somefile.txt
> /tank/widgets/b/foogoo_release_1.9/README
> 	echo " there" >> /tank/widgets/b/foogoo_release_1.9/README
> 
> Let''s suppose you are now working on widget B, and you want to
look at the
> past zfs snapshot of README, but you don''t remember where it came
from.
> That is, you don''t know the previous name or location where that
file used
> to be.  One way you could do it would be:
> 
> Look up the inode number of README.  (for example, ls -i README)
>                (suppose it''s inode 12345)
> find /tank/.zfs/snapshot -inum 12345
> 
> It will produce the result eventually:
> 	/tank/.zfs/snapshot/mysnap/a/rel2049_773.13-4/somefile.txt
> 
> So at last, you can see the previous version of that file.  Unfortunately,
> the find command will take a very long time.  You might grow gray hair.
There is a way to do this kind of object to name mapping, though
there''s no documented public interface for it. See zfs_obj_to_path()
function and ZFS_IOC_OBJ_TO_PATH ioctl.

I think it should also be possible to extend it to handle multiple names (in
case of multiple hardlinks) in some way, as id of parent directory is recorded
at the time of link creation in znode attributes

[...]
> There is a way to do this kind of object to name
> mapping, though there''s no documented public
> interface for it. See zfs_obj_to_path() function and
> ZFS_IOC_OBJ_TO_PATH ioctl.
> 
> I think it should also be possible to extend it to
> handle multiple names (in case of multiple hardlinks)
> in some way, as id of parent directory is recorded at
> the time of link creation in zone attributes
To add a bit: these sorts of things are _not_ required by any existing
standard, and may be limited to use by root (since they bypass directory
permissions).  So they''re typically private, undocumented, and subject
to change without notice.

Some other examples:

UFS _FIOIO ioctl: obtain a read-only file descriptor given an
existing file descriptor on the file system (to make the ioctl on)
and the inode number and generation number (keeps inode numbers
from being reused too quickly, mostly to make NFS happy I think) in
an argument to the ioctl.

Mac OS X /.vol directory: allows pre-OS X style access by
volume-ID/folder-ID/name triplet

Those are all hidden behind a particular library or application
that is the only supported way of using them.

It is perhaps unfortunate that there is no generic root-only way to
look up
fsid/inode
(problematic though due to hard links)
or
fsid/dir_inode/name
(could fail if name has been moved to another directory on the same filesystem)

but implementing a generic solution would likely be a lot of work
(requiring support from every filesystem, most of which were _not_
designed to do a reverse lookup, i.e. from inode back to name), and
the use cases seem to be very few indeed.  (As an example of that,
/.vol on a Mac is said to only work for HFS or HFS+ volumes, not old UFS
volumes (Macs used to support their own flavor of UFS, apparently; no
doubt one considerably different from on Solaris, so don''t go there)
In fact, I''m not sure that /.vol works at all on the latest Mac OS X.)
-- 
This message posted from opensolaris.org

On Wed, Apr 28, 2010 at 09:49:04PM +0200, Ragnar Sundblad
wrote:
> On 28 apr 2010, at 14.06, Edward Ned Harvey wrote:
> 
> What indicators do you have that ONTAP/WAFL has inode->name lookup
> functionality? I don''t think it has any such thing - WAFL is
pretty
> much an UFS/FFS that does COW instead of in-place writing, the main
> difference is that inodes are written to special inode files rather
> than specific static areas. Directories I believe works very much like
> UFS/FFS directories. But I may have misunderstood something and be
> wrong.
You''re correct that the FS is very UFS/FFS, but they''ve added
stuff.
The inode->name lookup is a bolt-on feature that was added in 7.1 (and
can be disabled).

# touch /net/tester/vol/ntap/file
# ls -li /net/tester/vol/ntap/file
    307107 -rw-r--r--   1 root     other          0 Apr 30 13:35
/net/tester/vol/ntap/file

tester*> inodepath -v ntap -a 307107
Inode 307107 in volume ntap (fsid 0x29428a) has 1 name.
Volume UUID is: 588b6ef0-5231-11de-9885-00a09800f026
[    1] Primary pathname = /vol/ntap/file

In general it is an internal feature and not readily exposed for user
operations.  I''ve seen it mentioned that it''s very handy for
their
interaction with virus scanners so it can pass a full path back to
them.  Also it helps with error messages.

It''s not really exposed via file ops.  The only time I''ve ever
used it
directly was when one of my servers was getting beaten up by clients.
The only diags were to scan the network and then go from the NFS FH to a
filename.  With ZFS on Solaris, I''m assuming that wouldn''t be
necessary
because you''d have dtrace to tell you exactly what files were being
accessed.

-- 
Darren