search for: ocfs2_delete_inod

Hello, I've hit a bug in OCFS2 delete code which results in inodes being left on disk without any links to them. The workload triggering this creates directories on one node and deletes them on another node in the cluster. The inode is not deleted because both nodes bail out from ocfs2_delete_inode() with: Skipping delete of 100405 because it is in use on other nodes The scenario which I think is happening is as follows: node1 node2 rmdir("d"); ocfs2_remote_dentry_delete() ocfs2_dentry_convert_worker() finishes ocfs2_unlink() eventually enters...

...rrno: -512 nfsd: non-standard errno: -512 nfsd: last server has exited nfsd: unexporting all filesystems And here are the strange messages: (6965,2):ocfs2_orphan_del:1869 ERROR: status = -2 (6965,2):ocfs2_remove_inode:614 ERROR: status = -2 (6965,2):ocfs2_wipe_inode:740 ERROR: status = -2 (6965,2):ocfs2_delete_inode:974 ERROR: status = -2 (6965,2):ocfs2_orphan_del:1869 ERROR: status = -2 (6965,2):ocfs2_remove_inode:614 ERROR: status = -2 (6965,2):ocfs2_wipe_inode:740 ERROR: status = -2 (6965,2):ocfs2_delete_inode:974 ERROR: status = -2 (6965,2):ocfs2_orphan_del:1869 ERROR: status = -2 (6965,2):ocfs2_remove_in...

...d allot bit(tells alloc group). and if its not stale(by above logic) we read it out using ocfs2_iget(). the second read should from cache. and also we have to add a per superblock nfs_sync_lock to cover the lock for alloc inode and that for inode in question. this is because ocfs2_get_dentry() and ocfs2_delete_inode() lock on them in reverse order. nfs_sync_lock is locked in EX mode in ocfs2_get_dentry() and in PR mode in ocfs2_delete_inode(). so that mutliple ocfs2_delete_inode() can run concurrently in normal case. Signed-off-by: Wengang Wang <wen.gang.wang at oracle.com> -- dlmglue.c | 45 ++...

...d allot bit(tells alloc group). and if its not stale(by above logic) we read it out using ocfs2_iget(). the second read should from cache. and also we have to add a per superblock nfs_sync_lock to cover the lock for alloc inode and that for inode in question. this is because ocfs2_get_dentry() and ocfs2_delete_inode() lock on them in reverse order. nfs_sync_lock is locked in EX mode in ocfs2_get_dentry() and in PR mode in ocfs2_delete_inode(). so that mutliple ocfs2_delete_inode() can run concurrently in normal case. Signed-off-by: Wengang Wang <wen.gang.wang at oracle.com> -- dlmglue.c | 45 ++...

...uation that (A) there is the in-memory inode and (B) this inode is without OCFS2_INODE_DELETE. For later operations on the stale inode, this may leads to crash because of the mismatch of the in-memory generation against the on-disk one if a new inode occupied the same block. for problem (2), in ocfs2_delete_inode(), after disk updates, ocfs2_remove_inode() doesn't sync/checkpiont to make sure the IO has finished. ocfs2_get_dentry() may read out a stale inode when JBD doesn't checkpoint yet(updates still only in memory). SOLUTION: (I) adds cross cluster lock for deletion and reading inode from nfs...

...d allot bit(tells alloc group). and if its not stale(by above logic) we read it out using ocfs2_iget(). the second read should from cache. and also we have to add a per superblock nfs_sync_lock to cover the lock for alloc inode and that for inode in question. this is because ocfs2_get_dentry() and ocfs2_delete_inode() lock on them in reverse order. nfs_sync_lock is locked in EX mode in ocfs2_get_dentry() and in PR mode in ocfs2_delete_inode(). so that mutliple ocfs2_delete_inode() can run concurrently in normal case. Signed-off-by: Wengang Wang <wen.gang.wang at oracle.com> -- diff --git a/fs/ocfs2/dlm...

...d allot bit(tells alloc group). and if its not stale(by above logic) we read it out using ocfs2_iget(). the second read should from cache. and also we have to add a per superblock nfs_sync_lock to cover the lock for alloc inode and that for inode in question. this is because ocfs2_get_dentry() and ocfs2_delete_inode() lock on them in reverse order. nfs_sync_lock is locked in EX mode in ocfs2_get_dentry() and in PR mode in ocfs2_delete_inode(). so that mutliple ocfs2_delete_inode() can run concurrently in normal case. Signed-off-by: Wengang Wang <wen.gang.wang at oracle.com> -- dlmglue.c | 46 ++...

...commit_truncate:7117 ERROR: status = -5 Jan 11 23:11:56 www1 kernel: [ 2339.658152] (mc,3305,0):ocfs2_truncate_for_delete:622 ERROR: status = -5 Jan 11 23:11:56 www1 kernel: [ 2339.659423] (mc,3305,0):ocfs2_wipe_inode:793 ERROR: status = -5 Jan 11 23:11:56 www1 kernel: [ 2339.660700] (mc,3305,0):ocfs2_delete_inode:1085 ERROR: status = -5 Jan 11 23:15:41 www1 kernel: [ 2565.101905] OCFS2: ERROR (device drbd1): ocfs2_commit_truncate: Inode 7418891 has an empty extent record, depth 2 Jan 11 23:15:41 www1 kernel: [ 2565.101908]. Jan 11 23:15:41 www1 kernel: [ 2565.105104] File system is now read-only due to...

The ocfs2 directory index updates two blocks when we remove an entry - the dx root and the dx leaf. OCFS2_DELETE_INODE_CREDITS was only accounting for the dx leaf. This shows up when ocfs2_delete_inode() runs out of credits in jbd2_journal_dirty_metadata() at "J_ASSERT_JH(jh, handle->h_buffer_credits > 0);". The test that caught this was running dirop_file_racer from the ocfs2-test suite with a 2...

...d allot bit(tells alloc group). and if its not stale(by above logic) we read it out using ocfs2_iget(). the second read should from cache. and also we have to add a per superblock nfs_sync_lock to cover the lock for alloc inode and that for inode in question. this is because ocfs2_get_dentry() and ocfs2_delete_inode() lock on them in reverse order. nfs_sync_lock is locked in EX mode in ocfs2_get_dentry() and in PR mode in ocfs2_delete_inode(). so that mutliple ocfs2_delete_inode() can run concurrently in normal case. this patch is based on 1.4 git. Signed-off-by: Wengang Wang <wen.gang.wang at oracle.com...

Anyone else seen this? (25736,1):ocfs2_query_inode_wipe:882 ERROR: Inode 129047 (on-disk 129047) not orphaned! Disk flags 0x1, inode flags 0x80 (25736,1):ocfs2_delete_inode:1010 ERROR: status = -17 Test case is my patched version of ocfs2-test/programs/dirop_file_racer that allows long filename prefixes. I ran it on two nodes in separate directories. Filesystem has a 512B blocksize, and I specified a filename prefix of 250 characters (thus brushing up against NAME...

...ish the orphan scan quickly. In general when umount starts, we will set a flag(Jan has added one named OCFS2_OSB_DROP_DENTRY_LOCK_IMMED which is set in kill_sb, I renamed it to OCFS2_OSB_UMOUNT_START so that we all can use it), and when ocfs2_evict_inode finds this flag, it will skip the process of ocfs2_delete_inode if the file is from orphan dir. We are safe to skip the delete process since it is in orphan dir, so it will be deleted eventually by other orphan scan, next mount or fsck. Signed-off-by: Tao Ma <tao.ma at oracle.com> --- fs/ocfs2/dcache.c | 4 ++-- fs/ocfs2/inode.c | 24 ++++++++++++...

...lkno = blkno; + args.fi_flags = 0; + args.fi_ino = ino_from_blkno(sb, blkno); + args.fi_sysfile_type = 0; + + return ilookup5(sb, blkno, ocfs2_find_actor, &args); +} struct inode *ocfs2_iget(struct ocfs2_super *osb, u64 blkno, unsigned flags, int sysfile_type) { @@ -949,6 +960,13 @@ void ocfs2_delete_inode(struct inode *in goto bail; } + /* Lock down the nfs_sync lock in PR mode */ + status = ocfs2_nfs_sync_lock(OCFS2_SB(inode->i_sb), 0); + if (status < 0) { + mlog(ML_ERROR, "getting nfs sync lock(PR) failed %d\n", status); + ocfs2_cleanup_delete_inode(inode, 0); + goto bai...

...lkno = blkno; + args.fi_flags = 0; + args.fi_ino = ino_from_blkno(sb, blkno); + args.fi_sysfile_type = 0; + + return ilookup5(sb, blkno, ocfs2_find_actor, &args); +} struct inode *ocfs2_iget(struct ocfs2_super *osb, u64 blkno, unsigned flags, int sysfile_type) { @@ -949,6 +960,13 @@ void ocfs2_delete_inode(struct inode *in goto bail; } + /* Lock down the nfs_sync lock in PR mode */ + status = ocfs2_nfs_sync_lock(OCFS2_SB(inode->i_sb), 0); + if (status < 0) { + mlog(ML_ERROR, "getting nfs sync lock(PR) failed %d\n", status); + ocfs2_cleanup_delete_inode(inode, 0); + goto bai...

...cate:6900 ERROR: status = -5 [94355.116364] (ocfs2_wq,5995,6):ocfs2_commit_truncate:7559 ERROR: status = -5 [94355.116370] (ocfs2_wq,5995,6):ocfs2_truncate_for_delete:597 ERROR: status = -5 [94355.116373] (ocfs2_wq,5995,6):ocfs2_wipe_inode:770 ERROR: status = -5 [94355.116376] (ocfs2_wq,5995,6):ocfs2_delete_inode:1062 ERROR: status = -5 ...although the particular extent block number varies somewhat. In addition, when I run "fsck.ocfs2 -y -f /dev/md0", I get an I/O error: dp-1:~ # fsck.ocfs2 -y -f /dev/md0 fsck.ocfs2 1.4.3 Checking OCFS2 filesystem in /dev/md0: Label: <NON...

...mutex_lock+0xd/0x31 Apr 29 11:01:18 node06 kernel: [2569440.616574] [<ffffffff8112c2b2>] ? dqget+0x2ce/0x318 Apr 29 11:01:18 node06 kernel: [2569440.616589] [<ffffffff8112cbad>] ? dquot_initialize+0x51/0x115 Apr 29 11:01:18 node06 kernel: [2569440.616611] [<ffffffffa0fcaab8>] ? ocfs2_delete_inode+0x0/0x1640 [ocfs2] Apr 29 11:01:18 node06 kernel: [2569440.616630] [<ffffffff810fee1f>] ? generic_delete_inode+0xd7/0x168 Apr 29 11:01:18 node06 kernel: [2569440.616652] [<ffffffffa0fca061>] ? ocfs2_drop_inode+0xc0/0x123 [ocfs2] Apr 29 11:01:18 node06 kernel: [2569440.616669] [<f...

Sorry i don`t copy everything: TEST-MAIL1# echo "ls //orphan_dir:0000"|debugfs.ocfs2 /dev/dm-0|wc debugfs.ocfs2 1.6.4 5239722 26198604 246266859 TEST-MAIL1# echo "ls //orphan_dir:0001"|debugfs.ocfs2 /dev/dm-0|wc debugfs.ocfs2 1.6.4 6074335 30371669 285493670 TEST-MAIL2 ~ # echo "ls //orphan_dir:0000"|debugfs.ocfs2 /dev/dm-0|wc debugfs.ocfs2 1.6.4 5239722 26198604

Found 15 patches (out of 162) that appeared relevant to ocfs2 1.4. Please review. Sunil

Hi, The following patches comprise the bulk of Ocfs2 updates for the 2.6.30 merge window. Aside from larger, more involved fixes, we're adding the following features, which I will describe in the order their patches are mailed. Sunil's exported some more state to our debugfs files, and consolidated some other aspects of our debugfs infrastructure. This will further aid us in debugging

ocfs2 currently uses the Journaled Block Device (JBD) for its journaling. This is a very stable and tested codebase. However, JBD is limited by architecture to 32bit block numbers. This means an ocfs2 filesystem is limited to 2^32 blocks. With a 4K blocksize, that's 16TB. People want larger volumes. Fortunately, there is now JBD2. JBD2 adds 64bit block number support and some other