thr3ads.net - Btrfs devel - [PATCH] snapshot, defragment and raid test cases for btrfs [Aug 2011]

If this information is useful, please help other people find it:
Share via:

Anand Jain

2011-Aug-05 07:59 UTC

[PATCH] snapshot, defragment and raid test cases for btrfs

Hi,

  Attached is the patch for the xfstests, which adds snapshot,
  defragment and volume management test cases for the btrfs
  (257, 258 and 259 respectively).

  This introduces a new user variable ''DISK_POOL'' which should
  be set to disks for the raid tests.

  An example of usage of these tests is as below.
------------
[root@localhost xfstests]# cat local.config
TEST_DEV="/dev/sdd"
TEST_DIR=/btrfs
SCRATCH_DEV="/dev/sde"
SCRATCH_MNT=/btrfs1
DISK_POOL="/dev/sdf /dev/sdg"
[root@localhost xfstests]#

[root@localhost xfstests]# ./check 257 258 259
FSTYP -- btrfs
PLATFORM -- Linux/i686 localhost 3.0.0-rc6+
MKFS_OPTIONS -- /dev/sde
MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/sde /btrfs1

257 8s
258 3s
259 33s
Ran: 257 258 259
Passed all 3 tests
----------------

  Thank you.

Anand

Amir Goldstein

2011-Aug-05 13:53 UTC

head link

Re: [PATCH] snapshot, defragment and raid test cases for btrfs

Hi Anand,

Can you please post the patch in the body of the message so we can
comment on it inline.

Aditya is also working on a btrfs snapshots xfstest (currently named
257) as part of
his Google summer of code project.

It would be great if you guys can cooperate your efforts.

BTW, while running Aditya''s test, Greg has stumbled upon a btrfs OOPS.
Greg can provide more details about it.

Cheers,
Amir.

On Fri, Aug 5, 2011 at 10:59 AM, Anand Jain <Anand.Jain@oracle.com>
wrote:>
> Hi,
>
>  Attached is the patch for the xfstests, which adds snapshot,
>  defragment and volume management test cases for the btrfs
>  (257, 258 and 259 respectively).
>
>  This introduces a new user variable ''DISK_POOL'' which
should
>  be set to disks for the raid tests.
>
>  An example of usage of these tests is as below.
> ------------
> [root@localhost xfstests]# cat local.config
> TEST_DEV="/dev/sdd"
> TEST_DIR=/btrfs
> SCRATCH_DEV="/dev/sde"
> SCRATCH_MNT=/btrfs1
> DISK_POOL="/dev/sdf /dev/sdg"
> [root@localhost xfstests]#
>
> [root@localhost xfstests]# ./check 257 258 259
> FSTYP -- btrfs
> PLATFORM -- Linux/i686 localhost 3.0.0-rc6+
> MKFS_OPTIONS -- /dev/sde
> MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/sde /btrfs1
>
> 257 8s
> 258 3s
> 259 33s
> Ran: 257 258 259
> Passed all 3 tests
> ----------------
>
>  Thank you.
>
> Anand
>--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Greg Freemyer

2011-Aug-05 15:40 UTC

head link

Re: [PATCH] snapshot, defragment and raid test cases for btrfs

On Fri, Aug 5, 2011 at 9:53 AM, Amir Goldstein <amir73il@gmail.com>
wrote:> Hi Anand,
>
> Can you please post the patch in the body of the message so we can
> comment on it inline.
>
> Aditya is also working on a btrfs snapshots xfstest (currently named
> 257) as part of
> his Google summer of code project.
>
> It would be great if you guys can cooperate your efforts.
>
> BTW, while running Aditya''s test, Greg has stumbled upon a btrfs
OOPS.
> Greg can provide more details about it.
>
> Cheers,
> Amir.
>
> On Fri, Aug 5, 2011 at 10:59 AM, Anand Jain <Anand.Jain@oracle.com>
wrote:
>>
>> Hi,
>>
>>  Attached is the patch for the xfstests, which adds snapshot,
>>  defragment and volume management test cases for the btrfs
>>  (257, 258 and 259 respectively).
>>
>>  This introduces a new user variable ''DISK_POOL''
which should
>>  be set to disks for the raid tests.
>>
>>  An example of usage of these tests is as below.
>> ------------
>> [root@localhost xfstests]# cat local.config
>> TEST_DEV="/dev/sdd"
>> TEST_DIR=/btrfs
>> SCRATCH_DEV="/dev/sde"
>> SCRATCH_MNT=/btrfs1
>> DISK_POOL="/dev/sdf /dev/sdg"
>> [root@localhost xfstests]#
>>
>> [root@localhost xfstests]# ./check 257 258 259
>> FSTYP -- btrfs
>> PLATFORM -- Linux/i686 localhost 3.0.0-rc6+
>> MKFS_OPTIONS -- /dev/sde
>> MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/sde /btrfs1
>>
>> 257 8s
>> 258 3s
>> 259 33s
>> Ran: 257 258 259
>> Passed all 3 tests
>> ----------------
>>
>>  Thank you.
>>
>> Anand
>>
Anand,

I''ve only reproduced the oops in 2.6.39 and older using openSUSE 11.4
userspace.

I have not tried with 3.0 or newer nor with newer userspace, so it may
or may not have been fixed.

It happened with a openSUSE kernel, so I put it in their bugzilla.

https://bugzilla.novell.com/show_bug.cgi?id=705039

Just yesterday, I put the actual full recreate details in the
bugzilla.  They are:

==I was working the xfstests git tree cloned via:

git clone https://github.com/adityadani/xfstests

# Then, get an older commit that apparently has a userspace bug that
triggers a kernel oops.

git checkout 1b712a4512d63502fab6

# Then prep for the test

export TEST_DEV=/dev/sda3
export TEST_DIR=/xfstests_mnt
export SCRATCH_DEV=/dev/sda1
export SCRATCH_MNT=/xfstests_mnt_scratch
export FSTYP=btrfs

umount $TEST_DIR
umount $SCRATCH_MNT

mkfs.btrfs $TEST_DEV
mkfs.btrfs $SCRATCH_DEV

mount $TEST_DEV $TEST_DIR
mount $SCRATCH_DEV $SCRATCH_MNT

# Then run the test

./check 255

I didn''t keep the oops, but it seems reproducible enough.

The oops was also put into the kde-oops catcher.
==
I don''t know what aspect of that test triggers the oops.  But the
latest version of test does not trigger it.

The latest version of that test in Aditya''s tree has been changed to
test 256.  Sounds like that will change to at least 260 before it gets
submitted.

Hope that helps,
Greg
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Greg Freemyer

2011-Aug-05 21:42 UTC

head link

Re: [PATCH] snapshot, defragment and raid test cases for btrfs

On Fri, Aug 5, 2011 at 11:40 AM, Greg Freemyer <greg.freemyer@gmail.com>
wrote:> On Fri, Aug 5, 2011 at 9:53 AM, Amir Goldstein <amir73il@gmail.com>
wrote:
>> Hi Anand,
>>
>> Can you please post the patch in the body of the message so we can
>> comment on it inline.
>>
>> Aditya is also working on a btrfs snapshots xfstest (currently named
>> 257) as part of
>> his Google summer of code project.
>>
>> It would be great if you guys can cooperate your efforts.
>>
>> BTW, while running Aditya''s test, Greg has stumbled upon a
btrfs OOPS.
>> Greg can provide more details about it.
>>
>> Cheers,
>> Amir.
>>
>> On Fri, Aug 5, 2011 at 10:59 AM, Anand Jain
<Anand.Jain@oracle.com> wrote:
>>>
>>> Hi,
>>>
>>>  Attached is the patch for the xfstests, which adds snapshot,
>>>  defragment and volume management test cases for the btrfs
>>>  (257, 258 and 259 respectively).
>>>
>>>  This introduces a new user variable ''DISK_POOL''
which should
>>>  be set to disks for the raid tests.
>>>
>>>  An example of usage of these tests is as below.
>>> ------------
>>> [root@localhost xfstests]# cat local.config
>>> TEST_DEV="/dev/sdd"
>>> TEST_DIR=/btrfs
>>> SCRATCH_DEV="/dev/sde"
>>> SCRATCH_MNT=/btrfs1
>>> DISK_POOL="/dev/sdf /dev/sdg"
>>> [root@localhost xfstests]#
>>>
>>> [root@localhost xfstests]# ./check 257 258 259
>>> FSTYP -- btrfs
>>> PLATFORM -- Linux/i686 localhost 3.0.0-rc6+
>>> MKFS_OPTIONS -- /dev/sde
>>> MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/sde
/btrfs1
>>>
>>> 257 8s
>>> 258 3s
>>> 259 33s
>>> Ran: 257 258 259
>>> Passed all 3 tests
>>> ----------------
>>>
>>>  Thank you.
>>>
>>> Anand
>>>
>
> Anand,
>
> I''ve only reproduced the oops in 2.6.39 and older using openSUSE
11.4 userspace.
>
> I have not tried with 3.0 or newer nor with newer userspace, so it may
> or may not have been fixed.
>
> It happened with a openSUSE kernel, so I put it in their bugzilla.
>
> https://bugzilla.novell.com/show_bug.cgi?id=705039
I just tested with kernel 3.0 and the oops is gone.

I closed that bugzilla as resolved.

Greg
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Dave Chinner

2011-Aug-06 14:06 UTC

head link

Re: [PATCH] snapshot, defragment and raid test cases for btrfs

On Fri, Aug 05, 2011 at 03:59:51PM +0800, Anand Jain
wrote:> 
> Hi,
> 
>  Attached is the patch for the xfstests, which adds snapshot,
>  defragment and volume management test cases for the btrfs
>  (257, 258 and 259 respectively).
A patch per test is the preferred form for review.
>  This introduces a new user variable ''DISK_POOL'' which
should
>  be set to disks for the raid tests.
It''s used as part of the scratch device, so it''d probaby be
better
to call it "SCRATCH_DEV_POOL". Indeed, if you do that, you could
then leave SCRATCH_DEV empty and fill it out with the first device
of the pool by default when setting up the scratch device in the
first place. That woul dthen get around all the mess you have where
you have to do "$SCRATCH_DEV $DISK_POOL" concatenations in all the
tests using the disk pool....
> 
>  An example of usage of these tests is as below.
> ------------
> [root@localhost xfstests]# cat local.config
> TEST_DEV="/dev/sdd"
> TEST_DIR=/btrfs
> SCRATCH_DEV="/dev/sde"
> SCRATCH_MNT=/btrfs1
> DISK_POOL="/dev/sdf /dev/sdg"
> [root@localhost xfstests]#
> 
> [root@localhost xfstests]# ./check 257 258 259
> FSTYP -- btrfs
> PLATFORM -- Linux/i686 localhost 3.0.0-rc6+
> MKFS_OPTIONS -- /dev/sde
> MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/sde /btrfs1
> 
> 257 8s
> 258 3s
> 259 33s
> Ran: 257 258 259
> Passed all 3 tests
> ----------------
> 
>  Thank you.
> 
> Anand
> From 62d90cd24afc192e091d9b81fb78d7c666237e97 Mon Sep 17 00:00:00 2001
> From: Anand Jain <Anand.Jain@oracle.com>
> Date: Fri, 5 Aug 2011 15:36:25 +0800
> Subject: [PATCH] Adding 257, 258, 259 to test snapshot, defragment and raid
in btrfs
> 
> Adding test case number 257, 258 and 259 to test snapshot, defragment
> and raid support in btrfs. This also adds a user input variable
> ''DISK_POOL''.
> 
> Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
> ---
>  257               |  198 ++++++++++++++++++++++++++++++++++++++++++
>  257.out           |    8 ++
>  258               |   75 ++++++++++++++++
>  258.out           |    2 +
>  259               |  220 +++++++++++++++++++++++++++++++++++++++++++++++
>  259.out           |    9 ++
>  common.rc         |   58 +++++++++++++
>  group             |    3 +
>  src/checksum_misc |   28 ++++++
>  src/devmgt        |   25 ++++++
>  src/fsmisc        |  247
+++++++++++++++++++++++++++++++++++++++++++++++++++++
>  src/randmisc      |   70 +++++++++++++++
>  src/snapmisc      |   73 ++++++++++++++++
Put all of these "misc" stuff in a common.snapshot file
or something similar. The src directory is for source code to
binaries that need to be built for other tests...
> diff --git a/257 b/257
> new file mode 100755
> index 0000000..2a75944
> --- /dev/null
> +++ b/257
> @@ -0,0 +1,198 @@
> +#! /bin/bash
> +# FS QA Test No. 257
> +#
> +# Extented btrfs snapshot test cases
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2011 Oracle  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#
> +#-----------------------------------------------------------------------
> +#
> +# creator
> +owner=anand.jain@oracle.com
> +
> +seq=`basename $0`
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +
> +_cleanup()
> +{
> +    rm -f $tmp.*
> +}
> +
> +trap "_cleanup ; exit \$status" 0 1 2 3 15
> +
> +# get standard environment, filters and checks
> +. ./common.rc
> +. ./common.filter
> +
> +_need_to_be_root
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +. ./src/checksum_misc
> +. ./src/randmisc
> +. ./src/fsmisc
> +. ./src/snapmisc
> +
> +_scratch_mkfs > /dev/null 2>&1
> +_scratch_mount
> +
> +#FS fill size
> +#name="DirDepth MinFiles MaxFiles MinFileSz MaxFileSz"
> +QUICK="1 4 4 4096 4096"
> +SMALL="1 10 100 4096 8192"
> +MEDIUM="10 30 1000 4096 8192"
> +LARGE="1000 1 1000 4096 1048576"
> +FDPROF=$QUICK
> +dolog=1
> +
> +# compare check sum of $1 with $2
> +_testa_snap_check()
> +{
> +	local volname
> +	volname=`echo $1 | rev | cut -d"/" -f1 | rev`
> +	#$here/src/checksum_misc save_checksum $1 $tmp.$volname.sum
> +	save_checksum $1 $tmp.$volname.sum
Please remove dead code (from eveywhere).
> +	#$here/src/checksum_misc verify_checksum $2 $tmp.$volname.sum
> +	verify_checksum $2 $tmp.$volname.sum
> +	[ $dolog = 1 ] && echo "testa_snap_check ok"
Please remove all the dolog stuff - the test requires the output to
pass, so it isn''t conditional.

As it is, i don''t think this is the best way to verify checksums.
See below as I get to the fsfill code....
> +_testb_file_append_check()
> +{
> +	local volname
> +
> +	volname=`echo $1 | rev | cut -d"/" -f1 | rev`
> +	#$here/src/checksum_misc save_checksum $1 $tmp.$volname.sum
> +	save_checksum $1 $tmp.$volname.sum
> +
> +	# modify the snap
> +	# $here/src/fsmisc modifyfs_fillblk ${2}
> +	modifyfs_fillblk ${2}
modify_fs_fillblk is only used here. It doesn''t need to be a generic
function. just open code it here and not force people to look in a
different place to find out what the test does.
> +	#$here/src/fsmisc check_sum $1 $tmp.$volname.sum
> +	verify_checksum $1 $tmp.$volname.sum
> +	echo "testb_file_append_check ok"
> +}
> +
> +
> +# arg1: Original subvolume
> +# arg2 : its snapshot
> +_testc_file_append2_check()
> +{
> +	local volname
> +
> +	volname=`echo $1 | rev | cut -d"/" -f1 | rev`
> +	#$here/src/checksum_misc save_checksum $1 $tmp.$volname.sum
> +	save_checksum $1 $tmp.$volname.sum
> +
> +	#$here/src/fsmisc modifyfs_append $2
> +	modifyfs_append $2
Ditto for modifyfs_append.
> +
> +	#$here/src/checksum_misc verify_checksum $1 $tmp.$volname.sum
> +	verify_checksum $1 $tmp.$volname.sum
> +	echo "testc_file_append2_check ok"
> +}
> +
> +# arg1: Original subvolume
> +# arg2 : its snapshot
> +_testd_read_modify_check()
> +{
> +	local volname
> +
> +	volname=`echo $1 | rev | cut -d"/" -f1 | rev`
> +	#$here/src/checksum_misc save_checksum $1 $tmp.$volname.sum
> +	save_checksum $1 $tmp.$volname.sum
> +	#$here/src/fsmisc modifyfs_readmodifywrite ${2}
> +	modifyfs_readmodifywrite ${2}
Ditto for modifyfs_readmodifywrite.
> +
> +	#$here/src/checksum_misc verify_checksum $1 $tmp.$volname.sum
> +	verify_checksum $1 $tmp.$volname.sum
> +	echo "testd_read_modify_check ok"
> +}
> +
> +# arg 1: depth
> +# arg 2: base vol
> +_teste_nested_snap_check()
> +{
> +	local n
> +	local x
> +
> +	# we need two additional snapshot to compare
> +	#$here/src/snapmisc create_nestedclones $(($1+2)) $2
> +	create_nestedclones $(($1+2)) $2
> +	#$here/src/snapmisc update_clonelist
> +	update_clonelist
> +
> +	n=${#CLONE_LIST[*]}
> +	if [ $n -lt $1 ]; then echo Clone List Error; status=1; exit; fi
> +
> +	dolog=0
> +	for i in `seq 1 $1`
> +	do
> +		n=$(($n-1))
> +		_testa_snap_check ${CLONE_LIST[$(($n-1))]} ${CLONE_LIST[$n]}
> +	done
> +	dolog=1
> +	echo "teste_nested_snap_check ok"
> +}
> +
> +_testf_delete_file_check()
> +{
> +	local org=$1
> +	local org_n=`echo $org | rev | cut -d"/" -f1 | rev`
> +	local afile
> +
> +	#$here/src/checksum_misc save_checksum $org $tmp.${org_n}.sum
> +	save_checksum $org $tmp.${org_n}.sum
> +
> +	#afile=`$here/src/randmisc picka rfile ${2}`
> +	afile=`picka rfile ${2}`
> +	#$here/src/fsmisc modifyfs_delete $afile
> +	modifyfs_delete $afile
Same for modifyfs_delete - all it does is run ''rm -f
<file>''!
> +
> +	#$here/src/checksum_misc verify_checksum $org $tmp.${org_n}.sum
> +	verify_checksum $org $tmp.${org_n}.sum
> +
> +	echo "testf_delete_file_check ok"
> +}
> +
> +# real QA test starts here
> +
> +firstvol="$SCRATCH_MNT/sv1"
> +btrfs subvolume create $firstvol | _filter_scratch
> +#$here/src/fsmisc fillfs $firstvol $FDPROF
> +fillfs $firstvol $FDPROF
If you are using the QUICK config, then you aren''t using any of the
randomness that fillfs actually provides, so why don''t you use
_populate_fs? From common.rc:

# Populate a filesystem with inodes for performance experiments
#
# usage: populate [-v] [-n ndirs] [-f nfiles] [-d depth] [-r root] [-s size]

(yeah, I know, the function is called _populate_fs, not populate)

One of the reasons I suggest for usingthis is that I don''t think
randomness is a good property for a regression test - you cannot
repeat the same test twice in a row, which defeats the purpose of
having a regression test. See comments on the fillfs function below.
> +SNAPNAME=0
> +#$here/src/snapmisc create_snap $firstvol $SCRATCH_MNT
> +create_snap $firstvol $SCRATCH_MNT
> +# Single clone test
> +_testa_snap_check $firstvol $SNAPNAME
> +_testb_file_append_check $firstvol $SNAPNAME
> +_testc_file_append2_check $firstvol $SNAPNAME
> +_testd_read_modify_check $firstvol $SNAPNAME
> +# nested clone test
> +_teste_nested_snap_check  7 $firstvol  # as of now don''t do
beyond 7 - btrfs bug
> +SNAPNAME=0
> +create_snap $firstvol $SCRATCH_MNT
> +_testf_delete_file_check $firstvol $SNAPNAME
Please separate the different tests by a line of whitespace. i.e.
everywhere you''ve got a comment, put a line of whitespce before it.


....

[defrag test]
> +#FS fill size
> +#name="DirDepth MinFiles MaxFiles MinFileSz MaxFileSz"
> +QUICK="1 4 4 4096 4096"
> +SMALL="1 10 100 4096 8192"
> +MEDIUM="10 30 1000 4096 8192"
> +LARGE="1000 1 1000 4096 1048576"
> +FDPROF=$QUICK
You only use one of these test setups, so  why even have two levels
of indirection and a bunch of unused configurations?
> +
> +_testa_defrag()
> +{
> +        btrfs filesystem defragment $1
> +        if [ ! $? ]; then echo "Error: Defrag failed"; exit; fi
The normal thing to do here is:

        btrfs filesystem defragment $1 || _fail "Defrag failed with
$?"
> +        echo "testa_defrag.... ok"
> +}
> +
> +fillfs $SCRATCH_MNT $FDPROF
> +_testa_defrag $SCRATCH_MNT
You don''t need a function here. The test is effectively:

echo "silence is golden"

# fillfs DirDepth MinFiles MaxFiles MinFileSz MaxFileSz
fillfs 1 4 4 4096 4096
btrfs filesystem defragment $1 || _fail "Defrag failed with $?"

That''s 2 lines of code for the test, and zero indirection.
> diff --git a/258.out b/258.out
> new file mode 100644
> index 0000000..6ccf5cf
> --- /dev/null
> +++ b/258.out
> @@ -0,0 +1,2 @@
> +QA output created by 258+silence is golden

.....> +# btrfs vol tests
.....> +#FS fill size
> +#name="DirDepth MinFiles MaxFiles MinFileSz MaxFileSz"
> +QUICK="1 4 4 4096 4096"
> +SMALL="1 10 100 4096 8192"
> +MEDIUM="10 30 1000 4096 8192"
> +LARGE="1000 1 1000 4096 1048576"
> +FDPROF=$QUICK
> +
> +FSDEV="$SCRATCH_DEV $DISK_POOL"
Why?
> +if_error()
> +{
> +        if [[ $? -ne 0 ]]; then
> +                echo "$1"
> +                exit
> +        fi
> +}
Seriously?
> +fun_error()
> +{
> +        echo $1
> +        exit
> +}
dead code
> +test1_default()
> +{
> +	local i
dead.
> +	mk_fs $1 "$2"
Oh, no, you didn''t implement your own mkfs function as well?

Please have a look at common.rc and the scratch_mkfs functions and
how that supports different filesystems. Add your multiple device
mkfs operations to the btrfs part of that code.

Oh, and to mount it, use ''scratch_mount'' and you
don''t need to pass
the mountpoint to every function...


[....]
> +test6_add()
> +{
> +	local i
> +	local devs[]="( $2 )"
> +	local n=${#devs[@]}
ugh.
> +
> +	n=$(($n-1))
> +
> +	_trace_wipe "$2"
What the does this do? it appears to dd zeros over the first
400k of the devices passed to it. Why? And if it''s needed for
mkfs.btrfs to work successfully, then why wasn''t this done in your
mk_fs function? What bug in btrfs is this working around?
> +	mk_fs $1 "${devs[0]}"
> +	fillfs $1 $FDPROF
> +	for i in `seq 1 $n`
> +	do
> +		# echo i=$i ${devs[$i]} $1
> +		btrfs device add ${devs[$i]} $1 2>&1 > /dev/null
> +	done
You do all that nasty array stuff, when all you need to is this:

	while [ $# -gt 0 ]; do
		btrfs device add $1 $SCRATCH_MNT 2>&1 > /dev/null
		shift;
	done

(assuming you aren''t passing the mount point as $1 anymore)
> +	btrfs filesystem balance $1
> +	btrfs filesystem show 2>&1 | egrep devid |awk ''{
> +		if ( $6 == "0.00" ) {
> +			print "Error: Device "$8" balance failed"
> +			exit
> +		}
> +	}''
> +	if_error "Error: test6_add... failed"
	[ $? -eq 0 ] || _fail "test6_add failed with $?"
> +	echo "test6_add ok"
> +}
> +
> +test7_replace()
> +{
> +	local i
> +	local x
> +	local devs=( $2 )
> +	local n=${#devs[@]}
> +	local ds
> +	local DEVHTL=""
> +
> +	if [ $n -lt 3 ]; then echo "Error: Need atleast 3 physical disks for
the vol:test7_replace"; exit; fi
This test should not be here - it should in the main test setup
section and use ''_not_run "Requires at least 3 scratch
devices"''.
That is, not having 3 devices is not a failure, it''s a reason not to
run the test.

Indeed, because none of the other tests have this requirement, you
shoul dput this into it''s own test so that the other tests here can
still be run even if this is not met.

> +	n=$(($n-1))
> +	ds=${devs[@]:0:$n}
Ah, what?

I can''t work out what this....
> +	_trace_wipe "$2"
> +	mk_fs $1 "$ds"
> +	fillfs $1 $FDPROF
> +
> +	#fail a disk
> +	ds=${devs[@]:$(($n-1)):1}
... and this are doing even after spending 5 minutes reading the
bash man page.

Seriously, if you''re going to write code that looks like line noise,
comment it so people who aren''t bash experts can understand what
your intent is here. I''ll have to refer to the bash man page every
time i look at this code, so I really can''t tell if it''s doing
what
you meant or whether it''s broken.

Write dumb code - it''s far easier to review and maintain....
> +	devmgt remove ${ds}
> +
> +	btrfs fi show | egrep "Some devices missing" > /dev/null
2>&1
> +	if_error "Error: btrfs did not report device missing"
> +	# if [ ! $? ]; then echo "Error: btrfs did not report device
missing"; exit; fi
> +
> +	# add a new disk to btrfs
> +	ds=${devs[@]:$(($n)):1}
> +	btrfs device add ${ds} ${1}
> +	btrfs fi balance $1
> +	btrfs filesystem show 2>&1 | egrep devid |awk ''{
> +		if ( $6 == "0.00" ) {
> +			print "Error: Device "$8" balance failed"
> +			exit
> +		}
> +	}''
> +	if_error "Error: test7_repalce... failed"
> +	echo "test7_replace ok"
> +
> +	# cleaup. add the removed disk
> +	umount $1 > /dev/null 2>&1
> +	devmgt add "${DEVHTL}"
> +}
> +
> +function test8_remove
> +{
> +	local res="ok"
> +	_trace_wipe "$2"
> +	mk_fs $1 "$2"
> +	fillfs $1 $FDPROF
> +
> +	dev_del=`echo ${2} | awk ''{print $NF}''`
Why use a different method for calulating the number of devices to
test 7? This version is far easier to understand, though I still
don''t know if it is correct because there''s nothing that
describes
what the test is supposed tobe doing....
> +	btrfs device delete $dev_del $1 || if_error "Error: delete
failed"
> +
> +	btrfs fi balance $1
> +	btrfs filesystem show 2>&1 | egrep devid |awk ''{
> +		if ( $6 == "0.00" ) {
> +			print "Error: Device "$8" balance failed"
> +			exit
> +		}
> +	} ''
	btrfs filesystem show 2>&1 | awk ''/devid/ {
> +	if_error "test8_remove... failed"
> +	echo "test8_remove ok"
> +	umount $1 > /dev/null 2>&1
scratch_umount
> +	_trace_wipe "$2"
why?
> +}
> +
> +test1_default $SCRATCH_MNT "$FSDEV"
> +test2_raid0 $SCRATCH_MNT "$FSDEV"
> +test3_raid1 $SCRATCH_MNT "$FSDEV"
> +test4_raid10 $SCRATCH_MNT "$FSDEV"
> +test5_single $SCRATCH_MNT "$FSDEV"
> +test6_add $SCRATCH_MNT "$FSDEV"
> +test7_replace $SCRATCH_MNT "$FSDEV"
> +test8_remove $SCRATCH_MNT "$FSDEV"
> +
> +status=0
> +exit
> diff --git a/259.out b/259.out
> new file mode 100644
> index 0000000..b3bd62a
> --- /dev/null
> +++ b/259.out
> @@ -0,0 +1,9 @@
> +QA output created by 259
> +test1_default ok
> +test2_raid0 ok
> +test3_raid1 ok
> +test4_raid10 ok
> +test5_single ok
> +test6_add ok
> +test7_replace ok
> +test8_remove ok
> diff --git a/common.rc b/common.rc
> index cb23a02..a354f69 100644
> --- a/common.rc
> +++ b/common.rc
> @@ -1559,7 +1559,65 @@ _test_inode_extsz()
>      echo $blocks
>  }
>  
> +_require_disk_pool()
> +{
> +	local i
> +	case "$FSTYP" in
> +	btrfs)
> +		if [ -z "$DISK_POOL" ]
> +		then
> +			_notrun "this test requires a valid \$DISK_POOL"
> +		fi
The usual format is:

		if [ -z "$DISK_POOL" ]; then
			.....
		fi
> +		if [ "`echo $DISK_POOL|wc -w`" -lt 2 ]
> +		then
> +			_notrun "this test needs more than 1 disk in DISK_POOL"
> +		fi
> +
> +		for i in $DISK_POOL
> +		do
Same:		for i in $DISK_POOL; do
> +			if [ "`_is_block_dev $i`" = "" ]
> +			then
> +				_notrun "this test requires valid block disk $i"
> +			fi
> +			if [ "`_is_block_dev $i`" = "`_is_block_dev
$TEST_DEV`" ]
> +			then
> +				_notrun "$i is part of TEST_DEV, this test requires unique
disks"
> +			fi
> +			if [ "`_is_block_dev $i`" = "`_is_block_dev
$SCRATCH_DEV`" ]
> +			then
> +				_notrun "$i is part of SCRATCH_DEV, this test requires unique
disks"
> +			fi
> +			if _mount | grep -q $i
> +			then
> +				if ! $UMOUNT_PROG $i
> +			        then
> +			            echo "failed to unmount $i - aborting"
> +			            exit 1
> +			        fi
> +			fi
> +			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
Umm, why is this zeroing here? Once again, if this is necessary then
it should be done only when necessary by the code that needs to do
it (i.e. the mkfs or device add/remove commands).
> +		done
> +	;;
> +	esac
> +}
Needs to _notrun for all FSTYPs other than btrfs.
> +_trace_wipe()
> +{
> +	local i
> +	case "$FSTYP" in
> +	btrfs)
> +		for i in $1
> +		do
> +			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
> +			if [ $? != 0 ]
> +			then
> +				echo "disk $i access failed"
> +				exit 1
> +			fi
> +		done
> +	;;
> +	esac
> +}
This needs to be folded into whatever btrfs mkfs code that needs it.
> 
################################################################################
>  
>  if [ "$iam" != new -a "$iam" != bench ]
> diff --git a/group b/group
> index 0c746c8..62eebfb 100644
> --- a/group
> +++ b/group
> @@ -370,3 +370,6 @@ deprecated
>  254 auto quick
>  255 auto quick prealloc
>  256 auto quick
> +257 auto quick
> +258 auto quick
> +259 auto quick
> diff --git a/src/checksum_misc b/src/checksum_misc
> new file mode 100644
> index 0000000..42488cb
> --- /dev/null
> +++ b/src/checksum_misc
> @@ -0,0 +1,28 @@
> +#! /bin/bash
> +# To Create the sha256 sum for the arg1
> +# arg1 FS to generate sha256
> +# arg2 File name to save the sha256 output
> +function save_checksum()
> +{
> +	local i=0
> +	>$2
> +	cd $1
> +	#echo "$1"
> +	for i in `find . -type f`; do sha256sum $i >> $2; done
> +	cd $OLDPWD
> +}
> +
> +
> +
> +# To check the sha256 for the TESTFS
> +# arg1 FS to be tested
> +# arg2 sha256 file
> +function verify_checksum()
> +{
> +	cd $1
> +	if [ ! -f $2 ]; then echo "$2 file not found"; status=1;exit;
fi
> +	#$SHA256SUM --status -c $2 | $GREP "FAILED"
> +	sha256sum -c $2 | grep "FAILED"
> +	if [ $? == 0 ]; then status=1;exit; fi
> +	cd $OLDPWD
> +}
This checksum stuff only appears to be used by a single
test, so put it in that test.

Further, it you use files of a known contents (rather that reading
from /dev/urandom), you only need to dump the checksum into the
golden output and you don''t need a verify step at all as the golden
image check at the end of the test will do it for you. (i.e. if the
checksum is different to that in the golden output, the test will
fail.) See 242 and 252 for examples of this style of checksum
verification.
> diff --git a/src/devmgt b/src/devmgt
> new file mode 100644
> index 0000000..d86b5c9
> --- /dev/null
> +++ b/src/devmgt
> @@ -0,0 +1,25 @@
> +#! /bin/bash
> +# arg 1 remove/add
> +# arg 2 /dev/sdx or return of devmgt resply
> +function devmgt
> +{
> +	local x
> +	local d
> +
> +	if [ ${1} == "remove" ]; then
> +		d=`echo $2|cut -d"/" -f3`
> +		x=`ls -l /sys/class/block/${d} | cut -d "/" -f12 | sed
''s/:/ /g''`
> +		#log_event log "Removing the disk ${d} ${x} ..."
> +		echo "scsi remove-single-device ${x}" > /proc/scsi/scsi
I don''t have a /proc/scsi on any of my machines, even those with SAS
disks. So tests that use this are going to need a test for having
support for this API and _not_run if it doesn''t exist.
> +		if [ ! $? ]; then echo "Error: Remove a disk failed"; exit 1;
fi
> +		DEVHTL=${x}
> +		#log_event done
> +		return
> +	else
> +		#log_event log "cleaning.. adding back the removed disk ${2} ..
"
> +		echo "scsi add-single-device ${2}" > /proc/scsi/scsi
> +		if [ ! $? ]; then echo "Error: Remove a disk failed"; exit 1;
fi
> +		#log_event done
> +		return
I was going to ask what device this operated on, but I see that the
global DEVHTL is passed back into the add function. needs
documentation that the device needs to be stored by the caller....
> +	fi
> +}
As it is, this function is only by a single test - you should move
it into that test given the _notrun check requirement. And it
probably shoul dbe two functions - one for remove and one for add...
> diff --git a/src/fsmisc b/src/fsmisc
> new file mode 100644
> index 0000000..2ee942d
> --- /dev/null
> +++ b/src/fsmisc
> @@ -0,0 +1,247 @@
> +#! /bin/bash
> +# Create Dir tree and files in it.
> +# arg1 basedir
> +# arg2 dir depth
> +# arg3 nfile_min
> +# arg4 nfile_max
> +# arg5 fsize_min
> +# arg6 fsize_max
> +
> +function fillfs()
> +{
> +	umask 000
> +	local j
> +	local i
> +	local DIRP
> +	local FCNT
> +	local FILEP
> +	local SCNT
> +	local BCNT
> +	#log_event log "filling $1 with dir=$2 files=$3 to $4 of size=$5 to
$6....."
> +	DIRP=$1
> +	for ((j=0; j<$2; j++))
> +	do
> +		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
> +		FCNT=$(r_ranged $3 $4)
> +		for ((i=0; i<$FCNT; i++))
> +		do
> +			FILEP=`mktemp -q $DIRP/file.XXXXXX`
> +			SCNT=$(r_ranged $5 $6)
> +			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 status=noxfer
2>/dev/null &
> +			# if some kernel may not support non aligned blocks well
> +			#BCNT=$(($SCNT/4096))
> +			#dd if=/dev/urandom of=$FILEP bs=4096 count=$BCNT status=noxfer
2>/dev/null &
> +		done
> +	done
> +	wait $!
> +	#log_event done
> +}
As mentioned previously, this give unrepeatable fill patterns
because it is random.  While that might be good for testing some
corner cases, it''s bad for reproducing problems when they occur.
This is the reason when se a known seed for things that generate
random operations (like fsx) so when a problem is found we can
reproduce the workload -exactly-.

But given the small test set sizes (even for large), the randomness
is not really buying you anything that a fixed configuration or two
would not test. Hence I''d drop the randomness altogether and use
known patterns and filenames instead to make the test code simpler
and allows you to put the checksums directly into the golden output
for simple verification.

Remember, once a test is integrated into xfstests it is no longer a
standalone test, so having lots of different configurations that are
never used just makes it harder to understand and maintain. Indeed,
if all the randomness goes away, all you need to do is make sure
_populate_fs writes patterned files rather than zeroes. e.g. uses
something like:

	$XFS_IO_PROG -f -c "pwrite -S 0xa55ac33c 0 $fsize" $outfile

instead of dd if=/dev/zero....

FWIW, that also gets around the performance problem of read lots of
data from /dev/urandom - I get about 10MB/s throughput from
/dev/urandom, which means writing large files will take quite some
time - far slower than disk performance....

> +function modifyfs_fillblk()
> +{
> +	local FSIZE
> +	local BLKS
> +	local NBLK
> +	local FALLOC
> +	local WS
> +
> +	#log_event log "filling the allocated blocks....."
> +	for i in `find /$1 -type f`
> +	do
> +		FSIZE=`stat -t $i | cut -d" " -f2`
> +		BLKS=`stat -c "%B" $i`
> +		NBLK=`stat -c "%b" $i`
> +		FALLOC=$(($BLKS * $NBLK))
> +		WS=$(($FALLOC - $FSIZE))
WS is unused, and given that you know the files aren''t sparse, why
wouldn''t you simple overwrite from offset 0 to $FSIZE?
> +		#echo $FSIZE $BLKS $NBLK $FALLOC $WS
> +		dd if=/dev/urandom of=$i obs=$FALLOC count=1 status=noxfer
2>/dev/null &
		xfs_io -f -c "pwrite -S 0xbeefbabe 0 $FSIZE" $f
> +	done
> +	wait $!
> +	#log_event done
> +}
> +
> +
> +# Append a random size to the files
> +# arg1 : FS in question
> +
> +function modifyfs_append()
> +{
> +	local FSIZE
> +	local X
> +	local N
> +	local i
> +	#log_event log "appending the files......"
> +	N=0
> +	for i in `find $1 -type f`
> +	do
> +		if [ $N == 0 ]; then
> +			N=$(($N+1))
You do this just to execute this on the first loop? Just put it
outside the loop and drop N altogether
> +			X=$i
> +			FSIZE=`stat -t $X | cut -d" " -f2`
> +			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer
2>/dev/null &
so it doubles the size of the file.

	xfs_io -a -f "pwrite -S 0xa55ac33c" $FSIZE $FSIZE" $i
> +			continue
> +		fi
> +		FSIZE=`stat -t $i | cut -d" " -f2`
> +		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer
2>/dev/null &
and that copies from the start of the previous file to the end of
the current file. IOWs, if we are using a fixed pattern, it''s the
same operation as the N=0 case, without the risk that the source
file is too short and we end up with a short read. It also means
this ends up as:

	for f in `find $1 -type f`; do
		FSIZE=`stat -t $i | cut -d" " -f2`
		xfs_io -a -f "pwrite -S 0xa55ac33c" $FSIZE $FSIZE" $i
	done;

.....
> +# To have a clean FS to test
> +# arg 1: mount point
> +# arg 2: metadata Raid Type/devs
> +# arg 3: data Raid Type/none
> +# arg 4: devices/none
> +#or
> +# arg 1: mount point
> +# arg 2: devices
> +function mk_fs()
> +{
> +	local ndev
> +
> +	umount $1 > /dev/null 2>&1
> +
> +	if [ $# == 2 ]; then
> +		# use default options to create the FS.
> +		ndev=( $2 )
> +		#echo ${ndev[@]}
> +		mkfs.btrfs ${ndev[@]} > /dev/null 2>&1
> +		#mkfs.btrfs ${ndev[@]} 2>&1 | _filter_scratch
> +		if [ $? != 0 ]; then echo mkfs failed ; exit; fi
> +	else if [ $# == 4 ]; then
> +		ndev=( $4 )
> +		mkfs.btrfs -m $2 -d $3 ${ndev[@]} > /dev/null 2>&1
> +		#mkfs.btrfs -m $2 -d $3 ${ndev[@]} 2>&1 | _filter_scratch
> +		if [ $? != 0 ]; then echo mkfs failed; exit; fi
> +	else
> +		echo "Coding Error 234: xfstests alias would like to hear about
it"
> +	fi
> +	fi
> +
> +	mount -t btrfs ${ndev[0]} $1
> +}
See my comments about folding this into btrfs specific branches of
scratch_mkfs.
> +# arg1: mntpoint
> +# arg2: Number of devs to use
> +# arg3: devs in array
> +function mk_fs_loopdev()
> +{
> +	local ndev
> +
> +	#log_event log "creating a new btrfs fs on $3...."
> +	umount $1 > /dev/null 2>&1
> +
> +	ndev=( $3 )
> +	#mkfs.btrfs -m $2 -d $3 ${ndev[@]} > ./tmp/out
> +	#if [ $? != 0 ]; then echo mkfs failed ; cat ./tmp/out; exit 1; fi
> +	#mount -t btrfs ${ndev[0]} $1
> +
> +#	echo
> +#	echo $# ndev=${ndev[@]}
> +	#log_event done
> +}
Unused, does nothing. Please remove.
> +
> +
> +function leaned_fs
> +{
> +	echo $1 $2
> +}
Unused, please remove.
> +#arg 1: mntpt
> +#arg 2: phy disks
> +#arg 3: Number of total loop devs
> +#arg 4: initial mkfs devs
> +
> +function fs_on_loopdev()
> +{
> +	local sz
> +	local n=0
> +	local devname
> +	local devs
> +
> +	mk_fs $1 "$2"
> +
> +	sz=$((`df -k $1 | egrep "$1" | awk ''{print
$4}''`/$(($3+1))))
> +	sz=$(($sz*1024))
> +	if [ $sz -lt 2671771648 ]; then echo "Error: Need at least 2G space
for this test"; exit 1; fi
> +
> +	for n in $(seq 1 $3)
> +	do
> +		devname=`mktemp -q $1/dev.XXXXXX`
> +		dd if=/dev/zero of=$devname bs=8192 count=64000 status=noxfer
2>/dev/null
> +		losetup -f $devname
> +		if [ $? != 0 ]; then echo "Error losetup failed";exit 1;fi
> +		LOOPDEV[$(($n-1))]=`losetup -a | grep $devname|cut -d":" -f1`
> +		LOOPFILE[$(($n-1))]=$devname
> +	done
> +
> +	mkdir -p $1/mnt
> +	devs=${LOOPDEV[@]:0:$4}
> +	mk_fs $1/mnt "$devs"
> +}
Unused, please remove.
> +
> +#arg 1: mntpt
> +function clean_loopdev()
> +{
> +	local i
> +
> +	#log_event log "Cleaning loop devices....."
> +
> +	local n=${#LOOPDEV[@]}
> +	if [ ! $n ]; then return; fi
> +
> +	n=$(($n-1))
> +
> +	umount ${LOOPDEV[0]}
> +	if [ $? != 0 ]; then echo "Error: umount loopdev failed"; exit
1;fi
> +
> +	for i in `seq 0 $n`
> +	do
> +		losetup -d ${LOOPDEV[$i]}
> +		unlink ${LOOPFILE[$i]}
> +		#echo ${LOOPDEVS[$i]}
> +		#ls -l $MNTPT/fd$i
> +	done
> +
> +	#log_event done
> +}
Also unused.

IOWs, this file can pretty much go away.
> diff --git a/src/randmisc b/src/randmisc
> new file mode 100644
> index 0000000..a0a065b
> --- /dev/null
> +++ b/src/randmisc
> @@ -0,0 +1,70 @@
> +#! /bin/bash
> +
> +# Generate Random number in a range
> +# arg1 min
> +# arg2 max
> +
> +function r_ranged()
> +{
> +	local X
> +	local Y
> +	local S
> +	if [ $2 == 0 ]; then echo 0; return; fi
> +	Y=$RANDOM
> +	((X = $2 - $1 + 1))
> +	((X = $Y % $X))
> +	((X = $X + $1))
> +	echo $X
> +}
only used by the fillfs function and the picka function below.
Given that fillfs isn''t actually random with the proposed
configuration (and is better off without the randomness), and picka
is used only once, this can move to whereever the picka function
goes...
> +
> +# Picks a rand(r)/first(f)/last(l) file or subvol
> +# arg1: what
> +# arg2: FS
> +function picka()
> +{
> +	local MAX
> +	local X
> +	local R
> +	local i
> +
> +	case $1 in
> +	rfile)
> +		MAX=`find $2 -type f | wc -l`
> +		X=`r_ranged 1 $MAX`
> +		R=0
> +		for i in `find $2 -type f`; do
> +			if [ "$X" == "$R" ]; then echo $i; return; fi
> +			R=$(($R+1))
> +		done
> +	;;
> +	rsnap)
> +		MAX=`$BTRFS subvolume list $2 | wc -l`
> +		X=`r_ranged 1 $MAX`
> +		R=0
> +		for i in `$BTRFS subvolume list $2`; do
> +			if [ $X == $R ]; then echo $i; return; fi
> +			R=$(($R+1))
> +		done
> +	;;
> +	ffile)
> +		for i in `find $2 -type f`; do
> +			echo $i
> +			break
> +		done
> +	;;
> +	fsnap)
> +		for i in `$BTRFS subvolume list $2`; do
> +			echo $i;
> +			break
> +		done
> +	;;
> +	lfile)
> +		for i in `find $2 -type f`; do
> +			sleep 1
> +		done
> +		echo $i
> +	;;
> +	lsnap)
> +	;;
> +	esac
> +}
picka is used only once, for the rfile case. Move the function into
that test, remove all the unused code. can probably even just be
open coded in that test...
> diff --git a/src/snapmisc b/src/snapmisc
> new file mode 100644
> index 0000000..7bd059f
> --- /dev/null
> +++ b/src/snapmisc
> @@ -0,0 +1,73 @@
> +#! /bin/bash
> +# Create a snapshot
> +# arg1 source
> +# arg2 dest dir
> +
> +# Return snapshot name in the SNAPNAME
> +function create_snap()
> +{
> +	local x
> +	if [ ! -d $2 ]; then echo Destination dir $2 not present; fi
> +	SNAPNAME=`mktemp -u $2/snap.XXXXXX`
> +	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null
> +	if [ $? != 0 ]; then echo Error snapshot create failed; status=1;exit; fi
> +	return
> +}
I''m not sure this really needs a wrapper, especially as $2 is always
$SCRATCHMNT and will be present. i.e, it only needs to be:

	btrfs subvolume snapshot $1 `mktemp -u $2/snap.XXXXXX` \
		> /dev/null || _fail "snapshot create failed with $?"
> +
> +# Destroy a snapshot
> +# arg 1: snapshot to be deleted
> +
> +function destroy_snap()
> +{
> +	#log_event log "deleting snapshot $1....."
> +	btrfs subvolume delete $1
> +	#log_event done done
> +}
That certainly doesn''t need a wrapper.
> +
> +# Creates n clones
> +# arg 1: number of clones required
> +# arg 2: soruce of which clone has to be taken
> +
> +function create_nclones()
> +{
> +	local i
> +
> +	for i in `seq 1 $1`
> +	do
> +		create_snap $2 $SCRATCH_MNT
> +	done
> +}
unused, please remove.
> +
> +function update_clonelist()
> +{
> +	local i
> +	# Is there a way btrfs can distinguish sv and ss ?
> +	local n=0
> +	for i in `btrfs subvolume list $SCRATCH_MNT | rev|cut -d" "
-f1|rev`
Why do you reverse the stream, cut it and then reverse it again?

> +	do
> +		#echo i=$i n=$n
> +		CLONE_LIST[$n]="${SCRATCH_MNT}/${i}"
> +                n=$((n+1))
> +        done
> +	#echo n=$n
> +	#for i in `seq 0 $((n-1))`; do echo i=$i ${CLONE_LIST[$i]}; done
kill all the debug.
> +}
> +
> +# arg1: number of clones required
> +# arg2: base subvol
> +function create_nestedclones()
> +{
> +	local i
> +	local x
> +	local w
> +	local SNAPNAME=0
> +
> +	x=$2
> +	for i in `seq 1 $1`
> +	do
> +		create_snap $x $SCRATCH_MNT
> +		x=$SNAPNAME
If you open code creat_snap, this global variable abuse goes away.
> +		#w=`btrfs subvolume list $SCRATCH_MNT|wc -l`
> +		#echo i=$i x=$x w=$w
> +	done
> +}
Both create_nestedclones() and  update_clonelist() are use donly
once, and one is called directly after the other. Just open code it
in the test - much simpler and easier to understand...

Cheers,

Dave.


-- 
Dave Chinner
david@fromorbit.com
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Aug-11 19:52 UTC

head link

Re: [PATCH] snapshot, defragment and raid test cases for btrfs

Dave,

Thanks for the great review comments, I have accepted almost all
comments and sent out a new set of patches.  Kindly review further,
let me know.

- Anand

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Aug-11 20:01 UTC

head link

[PATCH 1/3] Added test case 257 for btrfs extended snapshot tests

This adds the test case 257 to test the snapshot feature in:wq btrfs.
This also added a new user set variable SCRATCH_DEV_POOL, which should
be set to dev pool for btrfs.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
  257       |  192 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  257.out   |    2 +
  common.rc |   86 +++++++++++++++++++++++++++
  group     |    1 +
  4 files changed, 281 insertions(+), 0 deletions(-)
  create mode 100755 257
  create mode 100644 257.out

diff --git a/257 b/257
new file mode 100755
index 0000000..9d7c8ff
--- /dev/null
+++ b/257
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 257
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot 
create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null
&
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer 
2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 
2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 
2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts 
here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume 
create $firstvol failed"
+_fillfs 1 10 100 4096 8192 $firstvol
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit
diff --git a/257.out b/257.out
new file mode 100644
index 0000000..cc3693f
--- /dev/null
+++ b/257.out
@@ -0,0 +1,2 @@
+QA output created by 257
+Silence is golden
diff --git a/common.rc b/common.rc
index cb23a02..db7c2dd 100644
--- a/common.rc
+++ b/common.rc
@@ -1559,7 +1559,93 @@ _test_inode_extsz()
      echo $blocks
  }

+# Generate Random number in a range
+# arg1 min, arg2 max
+_rand_range()
+{
+	local X
+	local Y
+	if [ $2 == 0 ]; then echo 0; return; fi
+	RANDOM=1
+	Y=$RANDOM
+	((X = $2 - $1 + 1))
+	((X = $Y % $X))
+	((X = $X + $1))
+	echo $X
+}
+
+# Create Dir tree and files in it.
+# arg1 basedir
+# arg2 dir depth
+# arg3 nfile_min
+# arg4 nfile_max
+# arg5 fsize_min
+# arg6 fsize_max
+_fillfs()
+{
+	umask 000
+	local j
+	local i
+	local DIRP
+	local FCNT
+	local FILEP
+	local SCNT
+	local BCNT
+	DIRP=$6
+	for ((j=0; j<$1; j++)); do
+		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
+		FCNT=$(_rand_range $2 $3)
+		for ((i=0; i<$FCNT; i++)); do
+			FILEP=`mktemp -q $DIRP/file.XXXXXX`
+			SCNT=$(_rand_range $4 $5)
+			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 status=noxfer 
2>/dev/null &
+		done
+	done
+	wait $!
+}
+
+# scratch_dev_pool should contain the disks pool for the btrfs raid
+_require_scratch_dev_pool()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		if [ -z "$SCRATCH_DEV_POOL" ]
+		then
+			_notrun "this test requires a valid \$SCRATCH_DEV_POOL"
+		fi
+		if [ "`echo $SCRATCH_DEV_POOL|wc -w`" -lt 2 ]
+		then
+			_notrun "this test needs 2 or more disks in SCRATCH_DEV_POOL"
+		fi

+		for i in $SCRATCH_DEV_POOL
+		do
+			if [ "`_is_block_dev $i`" = "" ]
+			then
+				_notrun "this test requires valid block disk $i"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev $TEST_DEV`"
]
+			then
+				_notrun "$i is part of TEST_DEV, this test requires unique disks"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev
$SCRATCH_DEV`" ]
+			then
+				_notrun "$i is part of SCRATCH_DEV, this test requires unique
disks"
+			fi
+			if _mount | grep -q $i
+			then
+				if ! $UMOUNT_PROG $i
+			        then
+			            echo "failed to unmount $i - aborting"
+			            exit 1
+			        fi
+			fi
+			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
+		done
+	;;
+	esac
+}
 
################################################################################

  if [ "$iam" != new -a "$iam" != bench ]
diff --git a/group b/group
index 0c746c8..84c45da 100644
--- a/group
+++ b/group
@@ -370,3 +370,4 @@ deprecated
  254 auto quick
  255 auto quick prealloc
  256 auto quick
+257 auto quick
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Aug-11 20:01 UTC

head link

[PATCH 2/3] Added test case 258 for btrfs defragmentation

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
  258     |   60 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  258.out |    2 ++
  group   |    1 +
  3 files changed, 63 insertions(+), 0 deletions(-)
  create mode 100755 258
  create mode 100644 258.out

diff --git a/258 b/258
new file mode 100755
index 0000000..798bbaf
--- /dev/null
+++ b/258
@@ -0,0 +1,60 @@
+#! /bin/bash
+# FS QA Test No. 258
+#
+# btrfs defragmentation tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+_scratch_mount
+
+_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+btrfs filesystem defragment $SCRATCH_MNT
+[ $? = 20 ] || _fail "Defragment failed"
+
+umount $SCRATCH_MNT || _fail "umount failed"
+
+echo "Silence is golden"
+status=0; exit
diff --git a/258.out b/258.out
new file mode 100644
index 0000000..9d47016
--- /dev/null
+++ b/258.out
@@ -0,0 +1,2 @@
+QA output created by 258
+Silence is golden
diff --git a/group b/group
index 84c45da..739f806 100644
--- a/group
+++ b/group
@@ -371,3 +371,4 @@ deprecated
  255 auto quick prealloc
  256 auto quick
  257 auto quick
+258 auto quick
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Aug-11 20:01 UTC

head link

[PATCH 3/3] Added test case 259 for the btrfs raid features

Added test case 259 for the btrfs raid features. SCRATCH_DEV_POOL must
be set to 2 or more disks.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
  259       |  186 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  259.out   |    2 +
  common.rc |    6 ++
  group     |    1 +
  4 files changed, 195 insertions(+), 0 deletions(-)
  create mode 100755 259
  create mode 100644 259.out

diff --git a/259 b/259
new file mode 100755
index 0000000..b4ba403
--- /dev/null
+++ b/259
@@ -0,0 +1,186 @@
+#! /bin/bash
+# FS QA Test No. 259
+#
+# btrfs vol tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool
+_require_proc_scsi
+
+# arg 1 remove/add
+# arg 2 /dev/sdx or return of devmgt resply
+_devmgt()
+{
+	local x
+	local d
+
+	if [ $1 == "remove" ]; then
+		d=`echo $2|cut -d"/" -f3`
+		x=`ls -l /sys/class/block/${d} | cut -d "/" -f12 | sed
''s/:/ /g''`
+		echo "scsi remove-single-device ${x}" > /proc/scsi/scsi || _fail
"Remove disk failed"
+		DEVHTL=${x}
+	else
+		echo "scsi add-single-device ${2}" > /proc/scsi/scsi || _fail
"Add
disk failed"
+	fi
+}
+
+# Test cases related to raid in btrfs
+_test_raid0()
+{
+	export MKFS_OPTIONS="-m raid0 -d raid0"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_raid1()
+{
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_raid10()
+{
+	export MKFS_OPTIONS="-m raid10 -d raid10"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_single()
+{
+	export MKFS_OPTIONS="-m single -d single"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_add()
+{
+	local i
+	local devs[]="( $SCRATCH_DEV_POOL )"
+	local n=${#devs[@]}
+
+	n=$(($n-1))
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	for i in `seq 1 $n`
+	do
+		btrfs device add ${devs[$i]} $SCRATCH_MNT > /dev/null 2>&1 || _fail
"device add failed"
+	done
+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk ''{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}'' || _fail "test6_add... failed"
+	umount $SCRATCH_MNT
+}
+
+_test_replace()
+{
+	local i
+	local x
+	local devs=( $SCRATCH_DEV_POOL )
+	local n=${#devs[@]}
+	local ds
+	local DEVHTL=""
+
+	# exclude the last disk in the disk pool
+	n=$(($n-1))
+	ds=${devs[@]:0:$n}
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs "$ds" > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+	#pick the 2nd last disk
+	ds=${devs[@]:$(($n-1)):1}
+
+	#fail disk
+	_devmgt remove ${ds}
+
+	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" >
/dev/null
|| _fail \
+							"btrfs did not report device missing"
+
+	# add a new disk to btrfs
+	ds=${devs[@]:$(($n)):1}
+	btrfs device add ${ds} $SCRATCH_MNT > /dev/null 2>&1 || _fail
"dev add
failed"
+
+	# cleaup. add the removed disk
+	umount $SCRATCH_MNT
+	_devmgt add "${DEVHTL}"
+}
+
+_test_remove()
+{
+	_scratch_mkfs "$SCRATCH_DEV_POOL" > /dev/null 2>&1 ||
_fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+	# pick last dev in the list
+	dev_del=`echo ${SCRATCH_DEV_POOL} | awk ''{print $NF}''`
+	btrfs device delete $dev_del $SCRATCH_MNT || _fail "btrfs device 
delete failed"
+	btrfs fi show $SCRATCH_DEV 2>&1 | grep $dev_del > /dev/null
&& _fail
"btrfs still shows the deleted dev"
+	umount $SCRATCH_MNT
+}
+
+_test_raid0
+_test_raid1
+_test_raid10
+_test_single
+_test_add
+_test_replace
+_test_remove
+
+echo "Silence is golden"
+status=0; exit
diff --git a/259.out b/259.out
new file mode 100644
index 0000000..bfbd2de
--- /dev/null
+++ b/259.out
@@ -0,0 +1,2 @@
+QA output created by 259
+Silence is golden
diff --git a/common.rc b/common.rc
index db7c2dd..aa45a7e 100644
--- a/common.rc
+++ b/common.rc
@@ -1646,6 +1646,12 @@ _require_scratch_dev_pool()
  	;;
  	esac
  }
+
+# we need this to test removing a dev from the system
+_require_proc_scsi()
+{
+	[ -e /proc/scsi/scsi ]  || _notrun "/proc/scsi/scsi is not present"
+}
 
################################################################################

  if [ "$iam" != new -a "$iam" != bench ]
diff --git a/group b/group
index 739f806..62eebfb 100644
--- a/group
+++ b/group
@@ -372,3 +372,4 @@ deprecated
  256 auto quick
  257 auto quick
  258 auto quick
+259 auto quick
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Aug-15 07:52 UTC

head link

[PATCH v2 1/3] Added test case 257 for btrfs extended snapshot tests

Added extended test cases for btrfs snapshot. This adds a new
user-variable SCRATCH_DEV_POOL which should be set to a disk set.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
  257       |  192 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  257.out   |    2 +
  common.rc |   86 +++++++++++++++++++++++++++
  group     |    1 +
  4 files changed, 281 insertions(+), 0 deletions(-)
  create mode 100755 257
  create mode 100644 257.out

diff --git a/257 b/257
new file mode 100755
index 0000000..9d7c8ff
--- /dev/null
+++ b/257
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 257
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot 
create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null
&
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer 
2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 
2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 
2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts 
here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume 
create $firstvol failed"
+_fillfs 1 10 100 4096 8192 $firstvol
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit
diff --git a/257.out b/257.out
new file mode 100644
index 0000000..cc3693f
--- /dev/null
+++ b/257.out
@@ -0,0 +1,2 @@
+QA output created by 257
+Silence is golden
diff --git a/common.rc b/common.rc
index cb23a02..db7c2dd 100644
--- a/common.rc
+++ b/common.rc
@@ -1559,7 +1559,93 @@ _test_inode_extsz()
      echo $blocks
  }

+# Generate Random number in a range
+# arg1 min, arg2 max
+_rand_range()
+{
+	local X
+	local Y
+	if [ $2 == 0 ]; then echo 0; return; fi
+	RANDOM=1
+	Y=$RANDOM
+	((X = $2 - $1 + 1))
+	((X = $Y % $X))
+	((X = $X + $1))
+	echo $X
+}
+
+# Create Dir tree and files in it.
+# arg1 basedir
+# arg2 dir depth
+# arg3 nfile_min
+# arg4 nfile_max
+# arg5 fsize_min
+# arg6 fsize_max
+_fillfs()
+{
+	umask 000
+	local j
+	local i
+	local DIRP
+	local FCNT
+	local FILEP
+	local SCNT
+	local BCNT
+	DIRP=$6
+	for ((j=0; j<$1; j++)); do
+		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
+		FCNT=$(_rand_range $2 $3)
+		for ((i=0; i<$FCNT; i++)); do
+			FILEP=`mktemp -q $DIRP/file.XXXXXX`
+			SCNT=$(_rand_range $4 $5)
+			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 status=noxfer 
2>/dev/null &
+		done
+	done
+	wait $!
+}
+
+# scratch_dev_pool should contain the disks pool for the btrfs raid
+_require_scratch_dev_pool()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		if [ -z "$SCRATCH_DEV_POOL" ]
+		then
+			_notrun "this test requires a valid \$SCRATCH_DEV_POOL"
+		fi
+		if [ "`echo $SCRATCH_DEV_POOL|wc -w`" -lt 2 ]
+		then
+			_notrun "this test needs 2 or more disks in SCRATCH_DEV_POOL"
+		fi

+		for i in $SCRATCH_DEV_POOL
+		do
+			if [ "`_is_block_dev $i`" = "" ]
+			then
+				_notrun "this test requires valid block disk $i"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev $TEST_DEV`"
]
+			then
+				_notrun "$i is part of TEST_DEV, this test requires unique disks"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev
$SCRATCH_DEV`" ]
+			then
+				_notrun "$i is part of SCRATCH_DEV, this test requires unique
disks"
+			fi
+			if _mount | grep -q $i
+			then
+				if ! $UMOUNT_PROG $i
+			        then
+			            echo "failed to unmount $i - aborting"
+			            exit 1
+			        fi
+			fi
+			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
+		done
+	;;
+	esac
+}
 
################################################################################

  if [ "$iam" != new -a "$iam" != bench ]
diff --git a/group b/group
index 0c746c8..84c45da 100644
--- a/group
+++ b/group
@@ -370,3 +370,4 @@ deprecated
  254 auto quick
  255 auto quick prealloc
  256 auto quick
+257 auto quick
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Sep-02 08:45 UTC

head link

Re: [PATCH 1/3] Added test case 257 for btrfs extended snapshot tests

Dave, Alex:

any objections to these patches?  I not I''m going to put them in this
weekend.

On Fri, Aug 12, 2011 at 04:01:08AM +0800, Anand Jain
wrote:> This adds the test case 257 to test the snapshot feature in:wq btrfs.
> This also added a new user set variable SCRATCH_DEV_POOL, which should
> be set to dev pool for btrfs.
> 
> Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
> ---
>  257       |  192
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  257.out   |    2 +
>  common.rc |   86 +++++++++++++++++++++++++++
>  group     |    1 +
>  4 files changed, 281 insertions(+), 0 deletions(-)
>  create mode 100755 257
>  create mode 100644 257.out
> 
> diff --git a/257 b/257
> new file mode 100755
> index 0000000..9d7c8ff
> --- /dev/null
> +++ b/257
> @@ -0,0 +1,192 @@
> +#!/bin/bash
> +# FS QA Test No. 257
> +#
> +# Extented btrfs snapshot test cases
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2011 Oracle  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#
> +#-----------------------------------------------------------------------
> +#
> +# creator
> +owner=anand.jain@oracle.com
> +
> +seq=`basename $0`
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +
> +_cleanup()
> +{
> +    rm -f $tmp.*
> +}
> +
> +trap "_cleanup ; exit \$status" 0 1 2 3 15
> +
> +# get standard environment, filters and checks
> +. ./common.rc
> +. ./common.filter
> +
> +_need_to_be_root
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail
"mkfs failed"
> +_scratch_mount
> +
> +# Create and save sha256sum
> +# arg1 FS to generate sha256
> +# arg2 File name to save the sha256 output
> +_save_checksum()
> +{
> +	local i=0
> +	>$2
> +	cd $1
> +	for i in `find . -type f`; do sha256sum $i >> $2; done
> +	cd $OLDPWD
> +}
> +
> +# Verify the sha256sum for a FS
> +# arg1 FS to be tested
> +# arg2 sha256 file
> +_verify_checksum()
> +{
> +	cd $1
> +	[ -f $2 ] || _fail "checksum file $2 not found"
> +	sha256sum -c $2 | grep "FAILED"
> +	cd $OLDPWD
> +}
> +
> +# Create a snapshot
> +# arg1 dest dir
> +# Return snapshot name in the SNAPNAME
> +_create_snap()
> +{
> +	local x
> +	[ -d $1 ] || _fail "Destination dir $1 not present"
> +	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
> +	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail
> "snapshot create failed"
> +}
> +
> +# Reads and writes new data but does not allocate new blocks
> +# arg1 FS to be modified
> +_read_modify_write()
> +{
> +	local i
> +	local FSIZE
> +	for i in `find $1 -type f`
> +	do
> +		FSIZE=`stat -t $i | cut -d" " -f2`
> +		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null
&
> +		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null
&
> +	done
> +	wait $!
> +}
> +
> +# Fills the allocated blocks
> +# arg1 FS in question
> +_fill_blk()
> +{
> +	local FSIZE
> +	local BLKS
> +	local NBLK
> +	local FALLOC
> +	local WS
> +
> +	for i in `find /$1 -type f`
> +	do
> +		FSIZE=`stat -t $i | cut -d" " -f2`
> +		BLKS=`stat -c "%B" $i`
> +		NBLK=`stat -c "%b" $i`
> +		FALLOC=$(($BLKS * $NBLK))
> +		WS=$(($FALLOC - $FSIZE))
> +		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1
> status=noxfer 2>/dev/null &
> +	done
> +	wait $!
> +}
> +
> +
> +# Append a random size to the files
> +# arg1 : FS in question
> +_append_file()
> +{
> +	local FSIZE
> +	local X
> +	local N
> +	local i
> +	N=0
> +	for i in `find $1 -type f`
> +	do
> +		if [ $N == 0 ]; then
> +			X=$i
> +			FSIZE=`stat -t $X | cut -d" " -f2`
> +			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer
> 2>/dev/null &
> +			N=$(($N+1))
> +			continue
> +		fi
> +		FSIZE=`stat -t $i | cut -d" " -f2`
> +		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer
> 2>/dev/null &
> +		X=$i
> +	done
> +	wait $!
> +}
> +
> +##################### real QA test starts
> here###################################
> +# sv1 - is just a name nothing spl
> +firstvol="$SCRATCH_MNT/sv1"
> +btrfs subvolume create $firstvol > /dev/null || _fail "btrfs
> subvolume create $firstvol failed"
> +_fillfs 1 10 100 4096 8192 $firstvol
> +SNAPNAME=0
> +_create_snap $firstvol
> +_save_checksum $firstvol $tmp.sv1.sum
> +_verify_checksum $SNAPNAME $tmp.sv1.sum
> +
> +#Append1 the files
> +_fill_blk $SNAPNAME
> +_verify_checksum $firstvol $tmp.sv1.sum
> +
> +#Append2 the files
> +_append_file $SNAPNAME
> +_verify_checksum $firstvol $tmp.sv1.sum
> +
> +#read modify write
> +_read_modify_write $SNAPNAME
> +_verify_checksum $firstvol $tmp.sv1.sum
> +
> +#nested snapshot test
> +src_vol=$firstvol
> +for i in `seq 1 7`; do
> +	SNAPNAME=0
> +	_create_snap $src_vol
> +	_verify_checksum $SNAPNAME $tmp.sv1.sum
> +	src_vol=$SNAPNAME
> +done
> +
> +# file delete test
> +SNAPNAME=0
> +_create_snap $firstvol
> +tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
> +_save_checksum $SNAPNAME $tmp.$tname.sum
> +\rm -rf $firstvol/*
> +_verify_checksum $SNAPNAME $tmp.$tname.sum
> +
> +umount $SCRATCH_DEV || _fail "unmount failed"
> +
> +echo "Silence is golden"
> +status=0; exit
> diff --git a/257.out b/257.out
> new file mode 100644
> index 0000000..cc3693f
> --- /dev/null
> +++ b/257.out
> @@ -0,0 +1,2 @@
> +QA output created by 257
> +Silence is golden
> diff --git a/common.rc b/common.rc
> index cb23a02..db7c2dd 100644
> --- a/common.rc
> +++ b/common.rc
> @@ -1559,7 +1559,93 @@ _test_inode_extsz()
>      echo $blocks
>  }
> 
> +# Generate Random number in a range
> +# arg1 min, arg2 max
> +_rand_range()
> +{
> +	local X
> +	local Y
> +	if [ $2 == 0 ]; then echo 0; return; fi
> +	RANDOM=1
> +	Y=$RANDOM
> +	((X = $2 - $1 + 1))
> +	((X = $Y % $X))
> +	((X = $X + $1))
> +	echo $X
> +}
> +
> +# Create Dir tree and files in it.
> +# arg1 basedir
> +# arg2 dir depth
> +# arg3 nfile_min
> +# arg4 nfile_max
> +# arg5 fsize_min
> +# arg6 fsize_max
> +_fillfs()
> +{
> +	umask 000
> +	local j
> +	local i
> +	local DIRP
> +	local FCNT
> +	local FILEP
> +	local SCNT
> +	local BCNT
> +	DIRP=$6
> +	for ((j=0; j<$1; j++)); do
> +		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
> +		FCNT=$(_rand_range $2 $3)
> +		for ((i=0; i<$FCNT; i++)); do
> +			FILEP=`mktemp -q $DIRP/file.XXXXXX`
> +			SCNT=$(_rand_range $4 $5)
> +			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 status=noxfer
> 2>/dev/null &
> +		done
> +	done
> +	wait $!
> +}
> +
> +# scratch_dev_pool should contain the disks pool for the btrfs raid
> +_require_scratch_dev_pool()
> +{
> +	local i
> +	case "$FSTYP" in
> +	btrfs)
> +		if [ -z "$SCRATCH_DEV_POOL" ]
> +		then
> +			_notrun "this test requires a valid \$SCRATCH_DEV_POOL"
> +		fi
> +		if [ "`echo $SCRATCH_DEV_POOL|wc -w`" -lt 2 ]
> +		then
> +			_notrun "this test needs 2 or more disks in SCRATCH_DEV_POOL"
> +		fi
> 
> +		for i in $SCRATCH_DEV_POOL
> +		do
> +			if [ "`_is_block_dev $i`" = "" ]
> +			then
> +				_notrun "this test requires valid block disk $i"
> +			fi
> +			if [ "`_is_block_dev $i`" = "`_is_block_dev
$TEST_DEV`" ]
> +			then
> +				_notrun "$i is part of TEST_DEV, this test requires unique
disks"
> +			fi
> +			if [ "`_is_block_dev $i`" = "`_is_block_dev
$SCRATCH_DEV`" ]
> +			then
> +				_notrun "$i is part of SCRATCH_DEV, this test requires unique
disks"
> +			fi
> +			if _mount | grep -q $i
> +			then
> +				if ! $UMOUNT_PROG $i
> +			        then
> +			            echo "failed to unmount $i - aborting"
> +			            exit 1
> +			        fi
> +			fi
> +			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
> +		done
> +	;;
> +	esac
> +}
> 
>
################################################################################
> 
>  if [ "$iam" != new -a "$iam" != bench ]
> diff --git a/group b/group
> index 0c746c8..84c45da 100644
> --- a/group
> +++ b/group
> @@ -370,3 +370,4 @@ deprecated
>  254 auto quick
>  255 auto quick prealloc
>  256 auto quick
> +257 auto quick
> -- 
> 1.7.1
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs---end quoted text---
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Sep-02 08:49 UTC

head link

Re: [PATCH 3/3] Added test case 259 for the btrfs raid features

On Fri, Aug 12, 2011 at 04:01:33AM +0800, Anand Jain
wrote:> Added test case 259 for the btrfs raid features. SCRATCH_DEV_POOL must
> be set to 2 or more disks.
Any chance you can document how SCRATCH_DEV_POOL is supposed to be
used in the README file?  An addition patch is fine, no need to update
the existing ones.    Also is there a chance you could allow setting
only SCRATCH_DEV_POOL for btrfs, and derive SCRATCH_DEV for that as an
additional step?
> +# arg 1 remove/add
> +# arg 2 /dev/sdx or return of devmgt resply
> +_devmgt()
> +{
> +	local x
> +	local d
> +
> +	if [ $1 == "remove" ]; then
> +		d=`echo $2|cut -d"/" -f3`
> +		x=`ls -l /sys/class/block/${d} | cut -d "/" -f12 | sed
''s/:/ /g''`
> +		echo "scsi remove-single-device ${x}" > /proc/scsi/scsi ||
_fail
> "Remove disk failed"
> +		DEVHTL=${x}
> +	else
> +		echo "scsi add-single-device ${2}" > /proc/scsi/scsi ||
_fail
> "Add disk failed"
> +	fi
> +}
Please use the sysfs interface instead of the deprecated /proc/scsi/scsi
interface.  I would also suggest to split this routine into two for
removing and adding, and move them to the common helper library, so it
could be used for other tests.
> +# we need this to test removing a dev from the system
> +_require_proc_scsi()
> +{
> +	[ -e /proc/scsi/scsi ]  || _notrun "/proc/scsi/scsi is not
present"
> +}
The _require need really is that the device you want to work on is a
SCSI device.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-03 11:25 UTC

head link

Re: [PATCH 3/3] Added test case 259 for the btrfs raid features

Christoph,

  apologies for the delay I was on a long leave.

> Also is there a chance you could allow setting
> only SCRATCH_DEV_POOL for btrfs, and derive SCRATCH_DEV for that as an
> additional step?
  There is some challenge to set SCRATCH_DEV based on FSTYP since
  SCRATCH_DEV is used even before FSTYP is set.

  common.config uses SCRATCH_DEV and we set FSTYP in common.
  check calls these two script files in the following logic.
  --------------------------
  # we need common.config
  if ! . ./common.config
  then
     echo "$iam: failed to source common.config"
     exit 1
  fi

  # we need common
  . ./common
  ---------------------------

  to avoid major changes what we could do is to find-out FSTYP on our
  own in the file common.config and set SCRATCH_DEV based on
  SCRATCH_DEV_POOL, but thats a bit ugly way. I am open to any
  suggestions. thanks.


> Please use the sysfs interface instead of the deprecated /proc/scsi/scsi
> interface.  I would also suggest to split this routine into two for
> removing and adding, and move them to the common helper library, so it
> could be used for other tests.
yes. will get something like below code. in the common

-----
remove:
echo 1 > 
/sys/class/scsi_device/${host}:${channel}:${id}:${lun}/device/delete

add:
echo "$channel $id $lun" > /sys/class/scsi_host/host${host}/scan
-----

>> +# we need this to test removing a dev from the system
>> +_require_proc_scsi()
>> +{
>> +	[ -e /proc/scsi/scsi ]  || _notrun "/proc/scsi/scsi is not
present"
>> +}
> 
> The _require need really is that the device you want to work on is a
> SCSI device.
will get this.

Thanks -Anand
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-10 09:58 UTC

head link

[PATCH] Changes to received review comments

Now SCRATCH_DEV is derived from SCRATCH_DEV_POOL.
  Moved code to delete and scan block device to common.rc and
  updated README.


Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
  259           |   29 ++++++++---------------------
  README        |    9 +++++++--
  common.config |   14 ++++++++++++++
  common.rc     |   34 ++++++++++++++++++++++++++++++++--
  4 files changed, 61 insertions(+), 25 deletions(-)

diff --git a/259 b/259
index 522191d..fe90147 100755
--- a/259
+++ b/259
@@ -46,24 +46,7 @@ _supported_fs btrfs
  _supported_os Linux
  _require_scratch
  _require_scratch_dev_pool
-_require_proc_scsi
-
-# arg 1 remove/add
-# arg 2 /dev/sdx or return of devmgt resply
-_devmgt()
-{
-	local x
-	local d
-
-	if [ $1 == "remove" ]; then
-		d=`echo $2|cut -d"/" -f3`
-		x=`ls -l /sys/class/block/${d} | cut -d "/" -f12 | sed
''s/:/ /g''`
-		echo "scsi remove-single-device ${x}" > /proc/scsi/scsi || _fail
"Remove disk failed"
-		DEVHTL=${x}
-	else
-		echo "scsi add-single-device ${2}" > /proc/scsi/scsi || _fail
"Add
disk failed"
-	fi
-}
+_require_deletable_scratch_dev_pool

  # Test cases related to raid in btrfs
  _test_raid0()
@@ -128,10 +111,10 @@ _test_add()
  _test_replace()
  {
  	local i
-	local x
  	local devs=( $SCRATCH_DEV_POOL )
  	local n=${#devs[@]}
  	local ds
+	local d
  	local DEVHTL=""

  	# exclude the last disk in the disk pool
@@ -146,8 +129,12 @@ _test_replace()
  	#pick the 2nd last disk
  	ds=${devs[@]:$(($n-1)):1}

+	# retrive the HTL for this scsi disk
+	d=`echo $ds|cut -d"/" -f3`
+	DEVHTL=`ls -l /sys/class/block/${d} | rev | cut -d "/" -f 3 | rev`
+
  	#fail disk
-	_devmgt remove ${ds}
+	_devmgt_remove ${DEVHTL}

  	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" >
/dev/null
|| _fail \
  							"btrfs did not report device missing"
@@ -162,7 +149,7 @@ _test_replace()

  	# cleaup. add the removed disk
  	umount $SCRATCH_MNT
-	_devmgt add "${DEVHTL}"
+	_devmgt_add "${DEVHTL}"
  }

  _test_remove()
diff --git a/README b/README
index 5367be6..407888a 100644
--- a/README
+++ b/README
@@ -34,14 +34,19 @@ Preparing system for tests (IRIX and Linux):
              - leave empty and expect this partition to be clobbered
                by some tests.  If this is not provided, many tests will
                not be run.
-
+
          (these must be two DIFFERENT partitions)
+
+    - for btrfs only: some tests would need 3 or more independent 
SCRATCH disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+

      - setup your environment
          - setenv TEST_DEV "device containing TEST PARTITION"
          - setenv TEST_DIR "mount point of TEST PARTITION"
         	- optionally:
               - setenv SCRATCH_DEV "device containing SCRATCH
PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for 
testing btrfs"
               - setenv SCRATCH_MNT "mount point for SCRATCH
PARTITION"
               - setenv TAPE_DEV "tape device for testing xfsdump"
               - setenv RMT_TAPE_DEV "remote tape device for testing 
xfsdump"
@@ -63,7 +68,7 @@ Preparing system for tests (IRIX and Linux):
        tape which can be overwritten.

      - make sure $TEST_DEV is a mounted XFS partition
-    - make sure that $SCRATCH_DEV contains nothing useful
+    - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing 
useful

  Running tests:

diff --git a/common.config b/common.config
index 3642139..7ee255e 100644
--- a/common.config
+++ b/common.config
@@ -228,6 +228,20 @@ if [ ! -d "$TEST_DIR" ]; then
      exit 1
  fi

+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of 
its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward 
compatibility
+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi
+if [ "$FSTYP_tmp" == "btrfs" ]; then
+    if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+	SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | cut -d" " -f 1`
+	SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | cut -d" " -f 2-`
+    fi
+fi
+	
  echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
  if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a
"$?" != "0" ]; then
      echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a 
block device or a NFS filesystem"
diff --git a/common.rc b/common.rc
index 02dde11..a614090 100644
--- a/common.rc
+++ b/common.rc
@@ -1609,9 +1609,19 @@ _require_scratch_dev_pool()
  	esac
  }

-_require_proc_scsi()
+# We will check if the device is virtual (eg: loop device) since it 
does not
+# have the delete entry-point. Otherwise SCSI and USB devices are fine.
+_require_deletable_scratch_dev_pool()
  {
-	[ -e /proc/scsi/scsi ]  || _notrun "/proc/scsi/scsi is not present"
+	local i
+	local x
+	for i in $SCRATCH_DEV_POOL; do
+		x=`echo $i | cut -d"/" -f 3`
+		ls -l /sys/class/block/${x} | grep -q "virtual"
+		if [ $? == "0" ]; then
+			_notrun "$i is a virtual device which is not deletable"
+		fi
+	done
  }

  # Generate Random number in a range
@@ -1659,6 +1669,26 @@ _fillfs()
  	wait $!
  }

+# arg 1 is dev to remove and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_remove()
+{
+	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail "Remove 
disk failed"
+}
+
+# arg 1 is dev to add and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_add()
+{
+	local h
+	local tdl
+	# arg 1 will be in h:t:d:l format now in the h and "t d l" format
+	h=`echo ${1} | cut -d":" -f 1`
+	tdl=`echo ${1} | cut -d":" -f 2-|sed ''s/:/ /g''`
+
+	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add disk 
failed"
+}
+
 
################################################################################
  if [ "$iam" != new -a "$iam" != bench ]
  then
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Oct-10 11:21 UTC

head link

Re: [PATCH] Changes to received review comments

On Mon, Oct 10, 2011 at 05:58:47PM +0800, Anand Jain
wrote:>  Now SCRATCH_DEV is derived from SCRATCH_DEV_POOL.
>  Moved code to delete and scan block device to common.rc and
>  updated README.
Can you resend the whole patch?   Note that xfstests is up to test 262
in the meantime, so it will also need to be renumbered.

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-11 11:25 UTC

head link

Re: [PATCH] Changes to received review comments

Christoph,

  Email following this will have patches, kindly find them.

Thanks, Anand


On 10/10/2011 07:21 PM, Christoph Hellwig wrote:> On Mon, Oct 10, 2011 at 05:58:47PM +0800, Anand Jain wrote:
>>   Now SCRATCH_DEV is derived from SCRATCH_DEV_POOL.
>>   Moved code to delete and scan block device to common.rc and
>>   updated README.
>
> Can you resend the whole patch?   Note that xfstests is up to test 262
> in the meantime, so it will also need to be renumbered.
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-11 11:26 UTC

head link

[PATCH 1/3] 263: Functional test case for the btrfs snapshot

Create snapshots in various ways, modify the data around the block and
file boundaries and verify the data integrity.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
  263           |  192 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  263.out       |    2 +
  README        |    7 ++-
  common.config |   14 ++++
  common.rc     |  122 ++++++++++++++++++++++++++++++++++++
  group         |    1 +
  6 files changed, 337 insertions(+), 1 deletions(-)
  create mode 100755 263
  create mode 100644 263.out

diff --git a/263 b/263
new file mode 100755
index 0000000..f26a677
--- /dev/null
+++ b/263
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 263
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot 
create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null
&
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer 
2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 
2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 
2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts 
here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume 
create $firstvol failed"
+_fillfs 1 10 100 4096 8192 $firstvol
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit
diff --git a/263.out b/263.out
new file mode 100644
index 0000000..7a26e35
--- /dev/null
+++ b/263.out
@@ -0,0 +1,2 @@
+QA output created by 263
+Silence is golden
diff --git a/README b/README
index 5367be6..7c135c7 100644
--- a/README
+++ b/README
@@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
                not be run.

          (these must be two DIFFERENT partitions)
+
+    - for btrfs only: some tests would need 3 or more independent 
SCRATCH disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+

      - setup your environment
          - setenv TEST_DEV "device containing TEST PARTITION"
          - setenv TEST_DIR "mount point of TEST PARTITION"
         	- optionally:
               - setenv SCRATCH_DEV "device containing SCRATCH
PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for 
testing btrfs"
               - setenv SCRATCH_MNT "mount point for SCRATCH
PARTITION"
               - setenv TAPE_DEV "tape device for testing xfsdump"
               - setenv RMT_TAPE_DEV "remote tape device for testing 
xfsdump"
@@ -63,7 +68,7 @@ Preparing system for tests (IRIX and Linux):
        tape which can be overwritten.

      - make sure $TEST_DEV is a mounted XFS partition
-    - make sure that $SCRATCH_DEV contains nothing useful
+    - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing 
useful

  Running tests:

diff --git a/common.config b/common.config
index e94624e..4212fc8 100644
--- a/common.config
+++ b/common.config
@@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
      exit 1
  fi

+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of 
its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward 
compatibility
+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi
+if [ "$FSTYP_tmp" == "btrfs" ]; then
+    if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+	SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | cut -d" " -f 1`
+	SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | cut -d" " -f 2-`
+    fi
+fi
+	
  echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
  if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a
"$?" != "0" ]; then
      echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a 
block device or a NFS filesystem"
diff --git a/common.rc b/common.rc
index e948169..861f1f8 100644
--- a/common.rc
+++ b/common.rc
@@ -1591,6 +1591,128 @@ _test_inode_extsz()
      echo $blocks
  }

+# scratch_dev_pool should contain the disks pool for the btrfs raid
+_require_scratch_dev_pool()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		if [ -z "$SCRATCH_DEV_POOL" ]
+		then
+			_notrun "this test requires a valid \$SCRATCH_DEV_POOL"
+		fi
+		if [ "`echo $SCRATCH_DEV_POOL|wc -w`" -lt 2 ]
+		then
+			_notrun "this test needs 2 or more disks in SCRATCH_DEV_POOL"
+		fi
+
+		for i in $SCRATCH_DEV_POOL
+		do
+			if [ "`_is_block_dev $i`" = "" ]
+			then
+				_notrun "this test requires valid block disk $i"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev $TEST_DEV`"
]
+			then
+				_notrun "$i is part of TEST_DEV, this test requires unique disks"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev
$SCRATCH_DEV`" ]
+			then
+				_notrun "$i is part of SCRATCH_DEV, this test requires unique
disks"
+			fi
+			if _mount | grep -q $i
+			then
+				if ! $UMOUNT_PROG $i
+			        then
+			            echo "failed to unmount $i - aborting"
+			            exit 1
+			        fi
+			fi
+			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
+		done
+	;;
+	esac
+}
+
+# We will check if the device is virtual (eg: loop device) since it 
does not
+# have the delete entry-point. Otherwise SCSI and USB devices are fine.
+_require_deletable_scratch_dev_pool()
+{
+	local i
+	local x
+	for i in $SCRATCH_DEV_POOL; do
+		x=`echo $i | cut -d"/" -f 3`
+		ls -l /sys/class/block/${x} | grep -q "virtual"
+		if [ $? == "0" ]; then
+			_notrun "$i is a virtual device which is not deletable"
+		fi
+	done
+}
+
+# Generate Random number in a range
+# arg1 min, arg2 max
+_rand_range()
+{
+	local X
+	local Y
+	if [ $2 == 0 ]; then echo 0; return; fi
+	RANDOM=1
+	Y=$RANDOM
+	((X = $2 - $1 + 1))
+	((X = $Y % $X))
+	((X = $X + $1))
+	echo $X
+}
+
+# Create Dir tree and files in it.
+# arg1 basedir
+# arg2 dir depth
+# arg3 nfile_min
+# arg4 nfile_max
+# arg5 fsize_min
+# arg6 fsize_max
+_fillfs()
+{
+	umask 000
+	local j
+	local i
+	local DIRP
+	local FCNT
+	local FILEP
+	local SCNT
+	local BCNT
+	DIRP=$6
+	for ((j=0; j<$1; j++)); do
+		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
+		FCNT=$(_rand_range $2 $3)
+		for ((i=0; i<$FCNT; i++)); do
+			FILEP=`mktemp -q $DIRP/file.XXXXXX`
+			SCNT=$(_rand_range $4 $5)
+			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 status=noxfer 
2>/dev/null &
+		done
+	done
+	wait $!
+}
+
+# arg 1 is dev to remove and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_remove()
+{
+	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail "Remove 
disk failed"
+}
+
+# arg 1 is dev to add and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_add()
+{
+	local h
+	local tdl
+	# arg 1 will be in h:t:d:l format now in the h and "t d l" format
+	h=`echo ${1} | cut -d":" -f 1`
+	tdl=`echo ${1} | cut -d":" -f 2-|sed ''s/:/ /g''`
+
+	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add disk 
failed"
+}

 
################################################################################

diff --git a/group b/group
index 17466a1..d86624f 100644
--- a/group
+++ b/group
@@ -376,3 +376,4 @@ deprecated
  260 auto quick trim
  261 auto quick quota
  262 auto quick quota
+263 auto quick
-- 
1.7.1


--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-11 11:27 UTC

head link

[PATCH 2/3] 264: Functional test case for the btrfs de-fragmentation

To verify the btrfs de-fragmentation does not fail

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
  264     |   60 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  264.out |    2 ++
  group   |    1 +
  3 files changed, 63 insertions(+), 0 deletions(-)
  create mode 100755 264
  create mode 100644 264.out

diff --git a/264 b/264
new file mode 100755
index 0000000..61efeb8
--- /dev/null
+++ b/264
@@ -0,0 +1,60 @@
+#! /bin/bash
+# FS QA Test No. 264
+#
+# btrfs defragmentation tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+_scratch_mount
+
+_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+btrfs filesystem defragment $SCRATCH_MNT
+[ $? = 20 ] || _fail "Defragment failed"
+
+umount $SCRATCH_MNT || _fail "umount failed"
+
+echo "Silence is golden"
+status=0; exit
diff --git a/264.out b/264.out
new file mode 100644
index 0000000..eca790b
--- /dev/null
+++ b/264.out
@@ -0,0 +1,2 @@
+QA output created by 264
+Silence is golden
diff --git a/group b/group
index d86624f..b5863be 100644
--- a/group
+++ b/group
@@ -377,3 +377,4 @@ deprecated
  261 auto quick quota
  262 auto quick quota
  263 auto quick
+264 auto quick
-- 
1.7.1


--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-11 11:28 UTC

head link

[PATCH 3/3] 265: Functional test case for the btrfs raid operations

This will verify the various raid features in btrfs and device
replacement functionality

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
  265     |  187 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  265.out |    2 +
  group   |    1 +
  3 files changed, 190 insertions(+), 0 deletions(-)
  create mode 100755 265
  create mode 100644 265.out

diff --git a/265 b/265
new file mode 100755
index 0000000..4e35f6b
--- /dev/null
+++ b/265
@@ -0,0 +1,187 @@
+#! /bin/bash
+# FS QA Test No. 265
+#
+# btrfs vol tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+dev_removed=0
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+    if [ $dev_removed == 1 ]; then
+	umount $SCRATCH_MNT
+        _devmgt_add "${DEVHTL}"
+    fi
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool
+_require_deletable_scratch_dev_pool
+
+# Test cases related to raid in btrfs
+_test_raid0()
+{
+	export MKFS_OPTIONS="-m raid0 -d raid0"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_raid1()
+{
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_raid10()
+{
+	export MKFS_OPTIONS="-m raid10 -d raid10"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_single()
+{
+	export MKFS_OPTIONS="-m single -d single"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_add()
+{
+	local i
+	local devs[]="( $SCRATCH_DEV_POOL )"
+	local n=${#devs[@]}
+
+	n=$(($n-1))
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	for i in `seq 1 $n`
+	do
+		btrfs device add ${devs[$i]} $SCRATCH_MNT > /dev/null 2>&1 || _fail
"device add failed"
+	done
+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk ''{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}'' || _fail "test6_add... failed"
+	umount $SCRATCH_MNT
+}
+
+_test_replace()
+{
+	local i
+	local devs=( $SCRATCH_DEV_POOL )
+	local n=${#devs[@]}
+	local ds
+	local d
+	local DEVHTL=""
+
+	# exclude the last disk in the disk pool
+	n=$(($n-1))
+	ds=${devs[@]:0:$n}
+
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs "$ds" > /dev/null 2>&1 || _fail "tr:
mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+	#pick the 2nd last disk
+	ds=${devs[@]:$(($n-1)):1}
+
+	# retrive the HTL for this scsi disk
+	d=`echo $ds|cut -d"/" -f3`
+	DEVHTL=`ls -l /sys/class/block/${d} | rev | cut -d "/" -f 3 | rev`
+
+	#fail disk
+	_devmgt_remove ${DEVHTL}
+	dev_removed=1
+
+	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" >
/dev/null
|| _fail \
+							"btrfs did not report device missing"
+
+	# add a new disk to btrfs
+	ds=${devs[@]:$(($n)):1}
+	btrfs device add ${ds} $SCRATCH_MNT > /dev/null 2>&1 || _fail
"dev add
failed"
+	# in some system balance fails if there is no delay (a bug)
+	# putting sleep 10 to work around as of now
+	# sleep 10
+	btrfs fi balance $SCRATCH_MNT || _fail "dev balance failed"
+	#btrfs filesystem show | egrep devid |awk ''{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}'' || _fail "btrfs balance failed"
+
+	# cleaup. add the removed disk
+	umount $SCRATCH_MNT
+	_devmgt_add "${DEVHTL}"
+	dev_removed=0
+}
+
+_test_remove()
+{
+	_scratch_mkfs "$SCRATCH_DEV_POOL" > /dev/null 2>&1 ||
_fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+	# pick last dev in the list
+	dev_del=`echo ${SCRATCH_DEV_POOL} | awk ''{print $NF}''`
+	btrfs device delete $dev_del $SCRATCH_MNT || _fail "btrfs device 
delete failed"
+	btrfs fi show $SCRATCH_DEV 2>&1 | grep $dev_del > /dev/null
&& _fail
"btrfs still shows the deleted dev"
+	umount $SCRATCH_MNT
+}
+
+_test_raid0
+_test_raid1
+_test_raid10
+_test_single
+_test_add
+_test_replace
+_test_remove
+
+echo "Silence is golden"
+status=0; exit
diff --git a/265.out b/265.out
new file mode 100644
index 0000000..9fa4eb0
--- /dev/null
+++ b/265.out
@@ -0,0 +1,2 @@
+QA output created by 265
+Silence is golden
diff --git a/group b/group
index b5863be..b991b24 100644
--- a/group
+++ b/group
@@ -378,3 +378,4 @@ deprecated
  262 auto quick quota
  263 auto quick
  264 auto quick
+265 auto
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

David Sterba

2011-Oct-11 11:38 UTC

head link

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

please please fix your mail client not to wrap long lines ...

On Tue, Oct 11, 2011 at 07:26:57PM +0800, Anand Jain
wrote:> Create snapshots in various ways, modify the data around the block and
> file boundaries and verify the data integrity.
> 
> Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
> ---
>  263           |  192 
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  263.out       |    2 +
>  README        |    7 ++-
>  common.config |   14 ++++
>  common.rc     |  122 ++++++++++++++++++++++++++++++++++++
>  group         |    1 +
>  6 files changed, 337 insertions(+), 1 deletions(-)
>  create mode 100755 263
>  create mode 100644 263.out
> 
> diff --git a/263 b/263
> new file mode 100755
> index 0000000..f26a677
> --- /dev/null
> +++ b/263
> @@ -0,0 +1,192 @@
> +#!/bin/bash
> +# FS QA Test No. 263
> +#
> +# Extented btrfs snapshot test cases
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2011 Oracle  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#
> +#-----------------------------------------------------------------------
> +#
> +# creator
> +owner=anand.jain@oracle.com
> +
> +seq=`basename $0`
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +
> +_cleanup()
> +{
> +    rm -f $tmp.*
> +}
> +
> +trap "_cleanup ; exit \$status" 0 1 2 3 15
> +
> +# get standard environment, filters and checks
> +. ./common.rc
> +. ./common.filter
> +
> +_need_to_be_root
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail
"mkfs failed"
> +_scratch_mount
> +
> +# Create and save sha256sum
> +# arg1 FS to generate sha256
> +# arg2 File name to save the sha256 output
> +_save_checksum()
> +{
> +	local i=0
> +	>$2
> +	cd $1
> +	for i in `find . -type f`; do sha256sum $i >> $2; done
> +	cd $OLDPWD
> +}
> +
> +# Verify the sha256sum for a FS
> +# arg1 FS to be tested
> +# arg2 sha256 file
> +_verify_checksum()
> +{
> +	cd $1
> +	[ -f $2 ] || _fail "checksum file $2 not found"
> +	sha256sum -c $2 | grep "FAILED"
> +	cd $OLDPWD
> +}
> +
> +# Create a snapshot
> +# arg1 dest dir
> +# Return snapshot name in the SNAPNAME
> +_create_snap()
> +{
> +	local x
> +	[ -d $1 ] || _fail "Destination dir $1 not present"
> +	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
> +	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail
"snapshot
> create failed"
> +}
> +
> +# Reads and writes new data but does not allocate new blocks
> +# arg1 FS to be modified
> +_read_modify_write()
> +{
> +	local i
> +	local FSIZE
> +	for i in `find $1 -type f`
> +	do
> +		FSIZE=`stat -t $i | cut -d" " -f2`
> +		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 
> 2>/dev/null &
> +		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 
> 2>/dev/null &
> +	done
> +	wait $!
> +}
> +
> +# Fills the allocated blocks
> +# arg1 FS in question
> +_fill_blk()
> +{
> +	local FSIZE
> +	local BLKS
> +	local NBLK
> +	local FALLOC
> +	local WS
> +
> +	for i in `find /$1 -type f`
> +	do
> +		FSIZE=`stat -t $i | cut -d" " -f2`
> +		BLKS=`stat -c "%B" $i`
> +		NBLK=`stat -c "%b" $i`
> +		FALLOC=$(($BLKS * $NBLK))
> +		WS=$(($FALLOC - $FSIZE))
> +		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 
> status=noxfer 2>/dev/null &
...
> +	done
> +	wait $!
> +}
> +
> +
> +# Append a random size to the files
> +# arg1 : FS in question
> +_append_file()
> +{
> +	local FSIZE
> +	local X
> +	local N
> +	local i
> +	N=0
> +	for i in `find $1 -type f`
> +	do
> +		if [ $N == 0 ]; then
> +			X=$i
> +			FSIZE=`stat -t $X | cut -d" " -f2`
> +			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 
> status=noxfer 2>/dev/null &
> +			N=$(($N+1))
> +			continue
> +		fi
> +		FSIZE=`stat -t $i | cut -d" " -f2`
> +		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 
> status=noxfer 2>/dev/null &
> +		X=$i
> +	done
> +	wait $!
> +}
> +
> +##################### real QA test starts 
> here###################################
> +# sv1 - is just a name nothing spl
> +firstvol="$SCRATCH_MNT/sv1"
> +btrfs subvolume create $firstvol > /dev/null || _fail "btrfs
subvolume
> create $firstvol failed"
> +_fillfs 1 10 100 4096 8192 $firstvol
> +SNAPNAME=0
> +_create_snap $firstvol
> +_save_checksum $firstvol $tmp.sv1.sum
> +_verify_checksum $SNAPNAME $tmp.sv1.sum
> +
> +#Append1 the files
> +_fill_blk $SNAPNAME
> +_verify_checksum $firstvol $tmp.sv1.sum
> +
> +#Append2 the files
> +_append_file $SNAPNAME
> +_verify_checksum $firstvol $tmp.sv1.sum
> +
> +#read modify write
> +_read_modify_write $SNAPNAME
> +_verify_checksum $firstvol $tmp.sv1.sum
> +
> +#nested snapshot test
> +src_vol=$firstvol
> +for i in `seq 1 7`; do
> +	SNAPNAME=0
> +	_create_snap $src_vol
> +	_verify_checksum $SNAPNAME $tmp.sv1.sum
> +	src_vol=$SNAPNAME
> +done
> +
> +# file delete test
> +SNAPNAME=0
> +_create_snap $firstvol
> +tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
> +_save_checksum $SNAPNAME $tmp.$tname.sum
> +\rm -rf $firstvol/*
> +_verify_checksum $SNAPNAME $tmp.$tname.sum
> +
> +umount $SCRATCH_DEV || _fail "unmount failed"
> +
> +echo "Silence is golden"
> +status=0; exit
> diff --git a/263.out b/263.out
> new file mode 100644
> index 0000000..7a26e35
> --- /dev/null
> +++ b/263.out
> @@ -0,0 +1,2 @@
> +QA output created by 263
> +Silence is golden
> diff --git a/README b/README
> index 5367be6..7c135c7 100644
> --- a/README
> +++ b/README
> @@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
>                not be run.
> 
>          (these must be two DIFFERENT partitions)
> +
> +    - for btrfs only: some tests would need 3 or more independent 
> SCRATCH disks,
> +      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
> +
> 
>      - setup your environment
>          - setenv TEST_DEV "device containing TEST PARTITION"
>          - setenv TEST_DIR "mount point of TEST PARTITION"
>         	- optionally:
>               - setenv SCRATCH_DEV "device containing SCRATCH
PARTITION"
> +             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for 
> testing btrfs"
>               - setenv SCRATCH_MNT "mount point for SCRATCH
PARTITION"
>               - setenv TAPE_DEV "tape device for testing xfsdump"
>               - setenv RMT_TAPE_DEV "remote tape device for testing 
> xfsdump"
> @@ -63,7 +68,7 @@ Preparing system for tests (IRIX and Linux):
>        tape which can be overwritten.
> 
>      - make sure $TEST_DEV is a mounted XFS partition
> -    - make sure that $SCRATCH_DEV contains nothing useful
> +    - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing 
> useful
> 
>  Running tests:
> 
> diff --git a/common.config b/common.config
> index e94624e..4212fc8 100644
> --- a/common.config
> +++ b/common.config
> @@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
>      exit 1
>  fi
> 
> +# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of 
> its dev
> +# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward 
> compatibility
> +if [ "$HOSTOS" == "Linux" ]; then
> +    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
> +else
> +    FSTYP_tmp=xfs
> +fi
> +if [ "$FSTYP_tmp" == "btrfs" ]; then
> +    if [ ! -z "$SCRATCH_DEV_POOL" ]; then
> +	SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | cut -d" " -f 1`
> +	SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | cut -d" " -f 2-`
> +    fi
> +fi
> +	
>  echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
>  if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a
"$?" != "0" ]; then
>      echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a
> block device or a NFS filesystem"
> diff --git a/common.rc b/common.rc
> index e948169..861f1f8 100644
> --- a/common.rc
> +++ b/common.rc
> @@ -1591,6 +1591,128 @@ _test_inode_extsz()
>      echo $blocks
>  }
> 
> +# scratch_dev_pool should contain the disks pool for the btrfs raid
> +_require_scratch_dev_pool()
> +{
> +	local i
> +	case "$FSTYP" in
> +	btrfs)
> +		if [ -z "$SCRATCH_DEV_POOL" ]
> +		then
> +			_notrun "this test requires a valid 
> \$SCRATCH_DEV_POOL"
> +		fi
> +		if [ "`echo $SCRATCH_DEV_POOL|wc -w`" -lt 2 ]
> +		then
> +			_notrun "this test needs 2 or more disks in 
> SCRATCH_DEV_POOL"
> +		fi
> +
> +		for i in $SCRATCH_DEV_POOL
> +		do
> +			if [ "`_is_block_dev $i`" = "" ]
> +			then
> +				_notrun "this test requires valid block disk 
> $i"
> +			fi
> +			if [ "`_is_block_dev $i`" = "`_is_block_dev 
> $TEST_DEV`" ]
> +			then
> +				_notrun "$i is part of TEST_DEV, this test 
> requires unique disks"
> +			fi
> +			if [ "`_is_block_dev $i`" = "`_is_block_dev 
> $SCRATCH_DEV`" ]
> +			then
> +				_notrun "$i is part of SCRATCH_DEV, this 
> test requires unique disks"
> +			fi
> +			if _mount | grep -q $i
> +			then
> +				if ! $UMOUNT_PROG $i
> +			        then
> +			            echo "failed to unmount $i - aborting"
> +			            exit 1
> +			        fi
> +			fi
> +			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 
> 2>&1
> +		done
> +	;;
> +	esac
> +}
> +
> +# We will check if the device is virtual (eg: loop device) since it 
> does not
> +# have the delete entry-point. Otherwise SCSI and USB devices are fine.
> +_require_deletable_scratch_dev_pool()
> +{
> +	local i
> +	local x
> +	for i in $SCRATCH_DEV_POOL; do
> +		x=`echo $i | cut -d"/" -f 3`
> +		ls -l /sys/class/block/${x} | grep -q "virtual"
> +		if [ $? == "0" ]; then
> +			_notrun "$i is a virtual device which is not 
> deletable"
> +		fi
> +	done
> +}
> +
> +# Generate Random number in a range
> +# arg1 min, arg2 max
> +_rand_range()
> +{
> +	local X
> +	local Y
> +	if [ $2 == 0 ]; then echo 0; return; fi
> +	RANDOM=1
> +	Y=$RANDOM
> +	((X = $2 - $1 + 1))
> +	((X = $Y % $X))
> +	((X = $X + $1))
> +	echo $X
> +}
> +
> +# Create Dir tree and files in it.
> +# arg1 basedir
> +# arg2 dir depth
> +# arg3 nfile_min
> +# arg4 nfile_max
> +# arg5 fsize_min
> +# arg6 fsize_max
> +_fillfs()
> +{
> +	umask 000
> +	local j
> +	local i
> +	local DIRP
> +	local FCNT
> +	local FILEP
> +	local SCNT
> +	local BCNT
> +	DIRP=$6
> +	for ((j=0; j<$1; j++)); do
> +		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
> +		FCNT=$(_rand_range $2 $3)
> +		for ((i=0; i<$FCNT; i++)); do
> +			FILEP=`mktemp -q $DIRP/file.XXXXXX`
> +			SCNT=$(_rand_range $4 $5)
> +			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 
> status=noxfer 2>/dev/null &
> +		done
> +	done
> +	wait $!
> +}
> +
> +# arg 1 is dev to remove and is output of the below eg.
> +# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
> +_devmgt_remove()
> +{
> +	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail
"Remove
> disk failed"
> +}
> +
> +# arg 1 is dev to add and is output of the below eg.
> +# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
> +_devmgt_add()
> +{
> +	local h
> +	local tdl
> +	# arg 1 will be in h:t:d:l format now in the h and "t d l"
format
> +	h=`echo ${1} | cut -d":" -f 1`
> +	tdl=`echo ${1} | cut -d":" -f 2-|sed ''s/:/
/g''`
> +
> +	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add
disk
> failed"
> +}
> 
> 
>
################################################################################
> 
> diff --git a/group b/group
> index 17466a1..d86624f 100644
> --- a/group
> +++ b/group
> @@ -376,3 +376,4 @@ deprecated
>  260 auto quick trim
>  261 auto quick quota
>  262 auto quick quota
> +263 auto quick
> -- 
> 1.7.1
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Oct-11 11:42 UTC

head link

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

On Tue, Oct 11, 2011 at 01:38:38PM +0200, David Sterba
wrote:> please please fix your mail client not to wrap long lines ...
And while we''re at it please fix yours to not full quote :)

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

David Sterba

2011-Oct-11 11:47 UTC

head link

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

On Tue, Oct 11, 2011 at 07:42:45AM -0400, Christoph Hellwig
wrote:> On Tue, Oct 11, 2011 at 01:38:38PM +0200, David Sterba wrote:
> > please please fix your mail client not to wrap long lines ...
> 
> And while we''re at it please fix yours to not full quote :)
It was on purpose, I have marked some of those long lines.

david
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-12 04:52 UTC

head link

[PATCH 0/3] xfstest patch

Anand Jain (3):
  263: Functional test case for the btrfs snapshot
  264: Functional test case for the btrfs de-fragmentation
  265: Functional test case for the btrfs raid operations

 263           |  192 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 263.out       |    2 +
 264           |   60 ++++++++++++++++++
 264.out       |    2 +
 265           |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out       |    2 +
 README        |    7 ++-
 common.config |   14 ++++
 common.rc     |  122 ++++++++++++++++++++++++++++++++++++
 group         |    3 +
 10 files changed, 590 insertions(+), 1 deletions(-)
 create mode 100755 263
 create mode 100644 263.out
 create mode 100755 264
 create mode 100644 264.out
 create mode 100755 265
 create mode 100644 265.out

-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-12 04:52 UTC

head link

[PATCH 1/3] 263: Functional test case for the btrfs snapshot

Create snapshots in various ways, modify the data around the block and
file boundaries and verify the data integrity.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 263           |  192 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 263.out       |    2 +
 README        |    7 ++-
 common.config |   14 ++++
 common.rc     |  122 ++++++++++++++++++++++++++++++++++++
 group         |    1 +
 6 files changed, 337 insertions(+), 1 deletions(-)
 create mode 100755 263
 create mode 100644 263.out

diff --git a/263 b/263
new file mode 100755
index 0000000..f26a677
--- /dev/null
+++ b/263
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 263
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot
create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null
&
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer
2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer
2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer
2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts
here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume
create $firstvol failed"
+_fillfs 1 10 100 4096 8192 $firstvol
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit
diff --git a/263.out b/263.out
new file mode 100644
index 0000000..7a26e35
--- /dev/null
+++ b/263.out
@@ -0,0 +1,2 @@
+QA output created by 263
+Silence is golden
diff --git a/README b/README
index 5367be6..7c135c7 100644
--- a/README
+++ b/README
@@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
               not be run.
               
         (these must be two DIFFERENT partitions)
+
+    - for btrfs only: some tests would need 3 or more independent SCRATCH
disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+              
               
     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"   
        	- optionally:
              - setenv SCRATCH_DEV "device containing SCRATCH
PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for testing
btrfs"
              - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION"
              - setenv TAPE_DEV "tape device for testing xfsdump"
              - setenv RMT_TAPE_DEV "remote tape device for testing
xfsdump"
@@ -63,7 +68,7 @@ Preparing system for tests (IRIX and Linux):
       tape which can be overwritten.
           
     - make sure $TEST_DEV is a mounted XFS partition
-    - make sure that $SCRATCH_DEV contains nothing useful
+    - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing useful
     
 Running tests:
 
diff --git a/common.config b/common.config
index e94624e..4212fc8 100644
--- a/common.config
+++ b/common.config
@@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward
compatibility
+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi
+if [ "$FSTYP_tmp" == "btrfs" ]; then
+    if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+	SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | cut -d" " -f 1`
+	SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | cut -d" " -f 2-`
+    fi
+fi
+	
 echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a
"$?" != "0" ]; then
     echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a
block device or a NFS filesystem"
diff --git a/common.rc b/common.rc
index e948169..861f1f8 100644
--- a/common.rc
+++ b/common.rc
@@ -1591,6 +1591,128 @@ _test_inode_extsz()
     echo $blocks
 }
 
+# scratch_dev_pool should contain the disks pool for the btrfs raid
+_require_scratch_dev_pool()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		if [ -z "$SCRATCH_DEV_POOL" ]
+		then
+			_notrun "this test requires a valid \$SCRATCH_DEV_POOL"
+		fi
+		if [ "`echo $SCRATCH_DEV_POOL|wc -w`" -lt 2 ]
+		then
+			_notrun "this test needs 2 or more disks in SCRATCH_DEV_POOL"
+		fi
+
+		for i in $SCRATCH_DEV_POOL
+		do
+			if [ "`_is_block_dev $i`" = "" ]
+			then
+				_notrun "this test requires valid block disk $i"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev $TEST_DEV`"
]
+			then
+				_notrun "$i is part of TEST_DEV, this test requires unique disks"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev
$SCRATCH_DEV`" ]
+			then
+				_notrun "$i is part of SCRATCH_DEV, this test requires unique
disks"
+			fi
+			if _mount | grep -q $i
+			then
+				if ! $UMOUNT_PROG $i
+			        then
+			            echo "failed to unmount $i - aborting"
+			            exit 1
+			        fi
+			fi
+			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
+		done
+	;;
+	esac
+}
+
+# We will check if the device is virtual (eg: loop device) since it does not
+# have the delete entry-point. Otherwise SCSI and USB devices are fine. 
+_require_deletable_scratch_dev_pool()
+{
+	local i
+	local x
+	for i in $SCRATCH_DEV_POOL; do
+		x=`echo $i | cut -d"/" -f 3`
+		ls -l /sys/class/block/${x} | grep -q "virtual" 
+		if [ $? == "0" ]; then
+			_notrun "$i is a virtual device which is not deletable"
+		fi
+	done
+}
+
+# Generate Random number in a range
+# arg1 min, arg2 max
+_rand_range()
+{
+	local X
+	local Y
+	if [ $2 == 0 ]; then echo 0; return; fi
+	RANDOM=1
+	Y=$RANDOM
+	((X = $2 - $1 + 1))
+	((X = $Y % $X))
+	((X = $X + $1))
+	echo $X
+}
+
+# Create Dir tree and files in it.
+# arg1 basedir
+# arg2 dir depth
+# arg3 nfile_min
+# arg4 nfile_max
+# arg5 fsize_min
+# arg6 fsize_max
+_fillfs()
+{
+	umask 000
+	local j
+	local i
+	local DIRP
+	local FCNT
+	local FILEP
+	local SCNT
+	local BCNT
+	DIRP=$6
+	for ((j=0; j<$1; j++)); do
+		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
+		FCNT=$(_rand_range $2 $3)
+		for ((i=0; i<$FCNT; i++)); do
+			FILEP=`mktemp -q $DIRP/file.XXXXXX`
+			SCNT=$(_rand_range $4 $5)
+			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 status=noxfer 2>/dev/null
&
+		done
+	done
+	wait $!
+}
+
+# arg 1 is dev to remove and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_remove()
+{
+	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail "Remove
disk failed"
+}
+
+# arg 1 is dev to add and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_add()
+{
+	local h
+	local tdl
+	# arg 1 will be in h:t:d:l format now in the h and "t d l" format
+	h=`echo ${1} | cut -d":" -f 1`
+	tdl=`echo ${1} | cut -d":" -f 2-|sed ''s/:/ /g''`
+
+	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add disk
failed"
+}
 
 ################################################################################
 
diff --git a/group b/group
index 17466a1..d86624f 100644
--- a/group
+++ b/group
@@ -376,3 +376,4 @@ deprecated
 260 auto quick trim
 261 auto quick quota
 262 auto quick quota
+263 auto quick
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-12 04:52 UTC

head link

[PATCH 2/3] 264: Functional test case for the btrfs de-fragmentation

To verify the btrfs de-fragmentation does not fail

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 264     |   60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out |    2 ++
 group   |    1 +
 3 files changed, 63 insertions(+), 0 deletions(-)
 create mode 100755 264
 create mode 100644 264.out

diff --git a/264 b/264
new file mode 100755
index 0000000..61efeb8
--- /dev/null
+++ b/264
@@ -0,0 +1,60 @@
+#! /bin/bash
+# FS QA Test No. 264
+#
+# btrfs defragmentation tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+_scratch_mount
+
+_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+btrfs filesystem defragment $SCRATCH_MNT
+[ $? = 20 ] || _fail "Defragment failed"
+
+umount $SCRATCH_MNT || _fail "umount failed"
+
+echo "Silence is golden"
+status=0; exit
diff --git a/264.out b/264.out
new file mode 100644
index 0000000..eca790b
--- /dev/null
+++ b/264.out
@@ -0,0 +1,2 @@
+QA output created by 264
+Silence is golden
diff --git a/group b/group
index d86624f..b5863be 100644
--- a/group
+++ b/group
@@ -377,3 +377,4 @@ deprecated
 261 auto quick quota
 262 auto quick quota
 263 auto quick
+264 auto quick
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-12 04:52 UTC

head link

[PATCH 3/3] 265: Functional test case for the btrfs raid operations

This will verify the various raid features in btrfs and device
replacement functionality

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 265     |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out |    2 +
 group   |    1 +
 3 files changed, 190 insertions(+), 0 deletions(-)
 create mode 100755 265
 create mode 100644 265.out

diff --git a/265 b/265
new file mode 100755
index 0000000..4e35f6b
--- /dev/null
+++ b/265
@@ -0,0 +1,187 @@
+#! /bin/bash
+# FS QA Test No. 265
+#
+# btrfs vol tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+dev_removed=0
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+    if [ $dev_removed == 1 ]; then
+	umount $SCRATCH_MNT
+        _devmgt_add "${DEVHTL}"
+    fi
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool
+_require_deletable_scratch_dev_pool
+
+# Test cases related to raid in btrfs
+_test_raid0()
+{
+	export MKFS_OPTIONS="-m raid0 -d raid0"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_raid1()
+{
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_raid10()
+{
+	export MKFS_OPTIONS="-m raid10 -d raid10"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_single()
+{
+	export MKFS_OPTIONS="-m single -d single"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_add()
+{
+	local i
+	local devs[]="( $SCRATCH_DEV_POOL )"
+	local n=${#devs[@]}
+
+	n=$(($n-1))
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	for i in `seq 1 $n`
+	do
+		btrfs device add ${devs[$i]} $SCRATCH_MNT > /dev/null 2>&1 || _fail
"device add failed"
+	done
+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk ''{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}'' || _fail "test6_add... failed"
+	umount $SCRATCH_MNT
+}
+
+_test_replace()
+{
+	local i
+	local devs=( $SCRATCH_DEV_POOL )
+	local n=${#devs[@]}
+	local ds
+	local d
+	local DEVHTL=""
+
+	# exclude the last disk in the disk pool
+	n=$(($n-1))
+	ds=${devs[@]:0:$n}
+
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs "$ds" > /dev/null 2>&1 || _fail "tr:
mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+	#pick the 2nd last disk 
+	ds=${devs[@]:$(($n-1)):1}
+
+	# retrive the HTL for this scsi disk
+	d=`echo $ds|cut -d"/" -f3`
+	DEVHTL=`ls -l /sys/class/block/${d} | rev | cut -d "/" -f 3 | rev`
+
+	#fail disk
+	_devmgt_remove ${DEVHTL}
+	dev_removed=1
+
+	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" >
/dev/null || _fail \
+							"btrfs did not report device missing"
+
+	# add a new disk to btrfs
+	ds=${devs[@]:$(($n)):1}
+	btrfs device add ${ds} $SCRATCH_MNT > /dev/null 2>&1 || _fail
"dev add failed"
+	# in some system balance fails if there is no delay (a bug)
+	# putting sleep 10 to work around as of now
+	# sleep 10
+	btrfs fi balance $SCRATCH_MNT || _fail "dev balance failed"
+	#btrfs filesystem show | egrep devid |awk ''{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}'' || _fail "btrfs balance failed"
+
+	# cleaup. add the removed disk
+	umount $SCRATCH_MNT
+	_devmgt_add "${DEVHTL}"
+	dev_removed=0
+}
+
+_test_remove()
+{
+	_scratch_mkfs "$SCRATCH_DEV_POOL" > /dev/null 2>&1 ||
_fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+	# pick last dev in the list
+	dev_del=`echo ${SCRATCH_DEV_POOL} | awk ''{print $NF}''`
+	btrfs device delete $dev_del $SCRATCH_MNT || _fail "btrfs device delete
failed"
+	btrfs fi show $SCRATCH_DEV 2>&1 | grep $dev_del > /dev/null
&& _fail "btrfs still shows the deleted dev"
+	umount $SCRATCH_MNT
+}
+
+_test_raid0
+_test_raid1
+_test_raid10
+_test_single
+_test_add
+_test_replace
+_test_remove
+
+echo "Silence is golden"
+status=0; exit
diff --git a/265.out b/265.out
new file mode 100644
index 0000000..9fa4eb0
--- /dev/null
+++ b/265.out
@@ -0,0 +1,2 @@
+QA output created by 265
+Silence is golden
diff --git a/group b/group
index b5863be..b991b24 100644
--- a/group
+++ b/group
@@ -378,3 +378,4 @@ deprecated
 262 auto quick quota
 263 auto quick
 264 auto quick
+265 auto
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Dave Chinner

2011-Oct-13 00:56 UTC

head link

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

On Wed, Oct 12, 2011 at 12:52:52PM +0800, Anand Jain
wrote:> Create snapshots in various ways, modify the data around the block and
> file boundaries and verify the data integrity.....
> +
> +# Create Dir tree and files in it.
> +# arg1 basedir
> +# arg2 dir depth
> +# arg3 nfile_min
> +# arg4 nfile_max
> +# arg5 fsize_min
> +# arg6 fsize_max
> +_fillfs()
> +{
> +	umask 000
> +	local j
> +	local i
> +	local DIRP
> +	local FCNT
> +	local FILEP
> +	local SCNT
> +	local BCNT
> +	DIRP=$6
> +	for ((j=0; j<$1; j++)); do
> +		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
> +		FCNT=$(_rand_range $2 $3)
> +		for ((i=0; i<$FCNT; i++)); do
> +			FILEP=`mktemp -q $DIRP/file.XXXXXX`
> +			SCNT=$(_rand_range $4 $5)
> +			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 status=noxfer
2>/dev/null &
> +		done
> +	done
> +	wait $!
> +}
Please, no, not -another- new and just a bit different "fill
filesystem" function.

There''s already been one added recently in test 256 which I
commented at the time could use _populate_fs, and I said the same
thing in my original review of this series, too.

So please modify the _populate_fs function to do what you need.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-18 06:28 UTC

head link

[PATCH 0/3] xfstests patches

Looks like there are no further comments.
Changes per Dave pointing out to reuse _populate_fs.
Thanks for the review.

Anand Jain (3):
  264: Functional test case for the btrfs snapshot
  265: Functional test case for the btrfs de-fragmentation
  266: Functional test case for the btrfs raid operations

 264           |  192 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out       |    2 +
 265           |   60 ++++++++++++++++++
 265.out       |    2 +
 266           |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 266.out       |    2 +
 README        |    7 ++-
 common.config |   14 ++++
 common.rc     |   95 +++++++++++++++++++++++++++-
 group         |    3 +
 10 files changed, 559 insertions(+), 5 deletions(-)
 create mode 100755 264
 create mode 100644 264.out
 create mode 100755 265
 create mode 100644 265.out
 create mode 100755 266
 create mode 100644 266.out

-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-18 06:28 UTC

head link

[PATCH 1/3] 264: Functional test case for the btrfs snapshot

Create snapshots in various ways, modify the data around the block and
file boundaries and verify the data integrity.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 264           |  192 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out       |    2 +
 README        |    7 ++-
 common.config |   14 ++++
 common.rc     |   95 +++++++++++++++++++++++++++-
 group         |    1 +
 6 files changed, 306 insertions(+), 5 deletions(-)
 create mode 100755 264
 create mode 100644 264.out

diff --git a/264 b/264
new file mode 100755
index 0000000..cd7aae2
--- /dev/null
+++ b/264
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 263
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot
create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null
&
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer
2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer
2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer
2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts
here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume
create $firstvol failed"
+_populate_fs -n 1 -f 20 -d 10 -r $firstvol -s 10 -x
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit
diff --git a/264.out b/264.out
new file mode 100644
index 0000000..7a26e35
--- /dev/null
+++ b/264.out
@@ -0,0 +1,2 @@
+QA output created by 263
+Silence is golden
diff --git a/README b/README
index 5367be6..7c135c7 100644
--- a/README
+++ b/README
@@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
               not be run.
               
         (these must be two DIFFERENT partitions)
+
+    - for btrfs only: some tests would need 3 or more independent SCRATCH
disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+              
               
     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"   
        	- optionally:
              - setenv SCRATCH_DEV "device containing SCRATCH
PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for testing
btrfs"
              - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION"
              - setenv TAPE_DEV "tape device for testing xfsdump"
              - setenv RMT_TAPE_DEV "remote tape device for testing
xfsdump"
@@ -63,7 +68,7 @@ Preparing system for tests (IRIX and Linux):
       tape which can be overwritten.
           
     - make sure $TEST_DEV is a mounted XFS partition
-    - make sure that $SCRATCH_DEV contains nothing useful
+    - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing useful
     
 Running tests:
 
diff --git a/common.config b/common.config
index e94624e..4212fc8 100644
--- a/common.config
+++ b/common.config
@@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward
compatibility
+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi
+if [ "$FSTYP_tmp" == "btrfs" ]; then
+    if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+	SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | cut -d" " -f 1`
+	SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | cut -d" " -f 2-`
+    fi
+fi
+	
 echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a
"$?" != "0" ]; then
     echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a
block device or a NFS filesystem"
diff --git a/common.rc b/common.rc
index e948169..a59d5f0 100644
--- a/common.rc
+++ b/common.rc
@@ -1498,7 +1498,11 @@ _nfiles()
                 file=f$f
                 echo > $file
                 if [ $size -gt 0 ]; then
-                    dd if=/dev/zero of=$file bs=1024 count=$size
+		    if [ $randomdata == false ]; then
+	                dd if=/dev/zero of=$file bs=1024 count=$size 2>&1 |
_filter_dd
+		    else
+                    	dd if=/dev/urandom of=$file bs=1024 count=$size
2>&1 | _filter_dd
+		    fi
                 fi
 		let f=$f+1
         done
@@ -1508,7 +1512,11 @@ _nfiles()
 _descend()
 {
         dirname=$1; depth=$2
-        mkdir $dirname  || die "mkdir $dirname failed"
+	if [ -d $dirname ]; then
+		dirname=`mktemp -dq $dirname/dir.XXXXXX`
+	else
+	        mkdir $dirname  || die "mkdir $dirname failed"
+	fi
         cd $dirname
 
         _nfiles $files           # files for this dir
@@ -1529,7 +1537,7 @@ _descend()
 
 # Populate a filesystem with inodes for performance experiments
 #
-# usage: populate [-v] [-n ndirs] [-f nfiles] [-d depth] [-r root] [-s size]
+# usage: populate [-v] [-n ndirs] [-f nfiles] [-d depth] [-r root] [-s size]
[-x ]
 #
 _populate_fs()
 {
@@ -1540,8 +1548,9 @@ _populate_fs()
     depth=2         # depth of tree from root to leaves
     verbose=false
     root=root       # path of initial root of directory tree
+    randomdata=false # -x Use Random data instead of zero
 
-    while getopts "d:f:n:r:s:v" c
+    while getopts "d:f:n:r:s:v:x" c
     do
         case $c in
         d)      depth=$OPTARG;;
@@ -1550,6 +1559,7 @@ _populate_fs()
         s)      size=$OPTARG;;
         v)      verbose=true;;
         r)      root=$OPTARG;;
+	x)	randomdata=true;;
         esac
     done
 
@@ -1591,6 +1601,83 @@ _test_inode_extsz()
     echo $blocks
 }
 
+# scratch_dev_pool should contain the disks pool for the btrfs raid
+_require_scratch_dev_pool()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		if [ -z "$SCRATCH_DEV_POOL" ]
+		then
+			_notrun "this test requires a valid \$SCRATCH_DEV_POOL"
+		fi
+		if [ "`echo $SCRATCH_DEV_POOL|wc -w`" -lt 2 ]
+		then
+			_notrun "this test needs 2 or more disks in SCRATCH_DEV_POOL"
+		fi
+
+		for i in $SCRATCH_DEV_POOL
+		do
+			if [ "`_is_block_dev $i`" = "" ]
+			then
+				_notrun "this test requires valid block disk $i"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev $TEST_DEV`"
]
+			then
+				_notrun "$i is part of TEST_DEV, this test requires unique disks"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev
$SCRATCH_DEV`" ]
+			then
+				_notrun "$i is part of SCRATCH_DEV, this test requires unique
disks"
+			fi
+			if _mount | grep -q $i
+			then
+				if ! $UMOUNT_PROG $i
+			        then
+			            echo "failed to unmount $i - aborting"
+			            exit 1
+			        fi
+			fi
+			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
+		done
+	;;
+	esac
+}
+
+# We will check if the device is virtual (eg: loop device) since it does not
+# have the delete entry-point. Otherwise SCSI and USB devices are fine. 
+_require_deletable_scratch_dev_pool()
+{
+	local i
+	local x
+	for i in $SCRATCH_DEV_POOL; do
+		x=`echo $i | cut -d"/" -f 3`
+		ls -l /sys/class/block/${x} | grep -q "virtual" 
+		if [ $? == "0" ]; then
+			_notrun "$i is a virtual device which is not deletable"
+		fi
+	done
+}
+
+# arg 1 is dev to remove and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_remove()
+{
+	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail "Remove
disk failed"
+}
+
+# arg 1 is dev to add and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_add()
+{
+	local h
+	local tdl
+	# arg 1 will be in h:t:d:l format now in the h and "t d l" format
+	h=`echo ${1} | cut -d":" -f 1`
+	tdl=`echo ${1} | cut -d":" -f 2-|sed ''s/:/ /g''`
+
+	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add disk
failed"
+}
 
 ################################################################################
 
diff --git a/group b/group
index 2a8970c..cfbae8c 100644
--- a/group
+++ b/group
@@ -377,3 +377,4 @@ deprecated
 261 auto quick quota
 262 auto quick quota
 263 rw auto quick
+264 auto quick
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-18 06:28 UTC

head link

[PATCH 2/3] 265: Functional test case for the btrfs de-fragmentation

To verify the btrfs de-fragmentation does not fail.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 265     |   60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out |    2 ++
 group   |    1 +
 3 files changed, 63 insertions(+), 0 deletions(-)
 create mode 100755 265
 create mode 100644 265.out

diff --git a/265 b/265
new file mode 100755
index 0000000..81ec69d
--- /dev/null
+++ b/265
@@ -0,0 +1,60 @@
+#! /bin/bash
+# FS QA Test No. 265
+#
+# btrfs defragmentation tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+_scratch_mount
+
+_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+
+btrfs filesystem defragment $SCRATCH_MNT
+[ $? = 20 ] || _fail "Defragment failed"
+
+umount $SCRATCH_MNT || _fail "umount failed"
+
+echo "Silence is golden"
+status=0; exit
diff --git a/265.out b/265.out
new file mode 100644
index 0000000..9fa4eb0
--- /dev/null
+++ b/265.out
@@ -0,0 +1,2 @@
+QA output created by 265
+Silence is golden
diff --git a/group b/group
index cfbae8c..6bac8b5 100644
--- a/group
+++ b/group
@@ -378,3 +378,4 @@ deprecated
 262 auto quick quota
 263 rw auto quick
 264 auto quick
+265 auto quick
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-18 06:28 UTC

head link

[PATCH 3/3] 266: Functional test case for the btrfs raid operations

This will verify the various raid features in btrfs and device
replacement functionality.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 266     |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 266.out |    2 +
 group   |    1 +
 3 files changed, 190 insertions(+), 0 deletions(-)
 create mode 100755 266
 create mode 100644 266.out

diff --git a/266 b/266
new file mode 100755
index 0000000..3e70630
--- /dev/null
+++ b/266
@@ -0,0 +1,187 @@
+#! /bin/bash
+# FS QA Test No. 266
+#
+# btrfs vol tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+dev_removed=0
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+    if [ $dev_removed == 1 ]; then
+	umount $SCRATCH_MNT
+        _devmgt_add "${DEVHTL}"
+    fi
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool
+_require_deletable_scratch_dev_pool
+
+# Test cases related to raid in btrfs
+_test_raid0()
+{
+	export MKFS_OPTIONS="-m raid0 -d raid0"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_raid1()
+{
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_raid10()
+{
+	export MKFS_OPTIONS="-m raid10 -d raid10"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_single()
+{
+	export MKFS_OPTIONS="-m single -d single"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_add()
+{
+	local i
+	local devs[]="( $SCRATCH_DEV_POOL )"
+	local n=${#devs[@]}
+
+	n=$(($n-1))
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+	for i in `seq 1 $n`
+	do
+		btrfs device add ${devs[$i]} $SCRATCH_MNT > /dev/null 2>&1 || _fail
"device add failed"
+	done
+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk ''{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}'' || _fail "test6_add... failed"
+	umount $SCRATCH_MNT
+}
+
+_test_replace()
+{
+	local i
+	local devs=( $SCRATCH_DEV_POOL )
+	local n=${#devs[@]}
+	local ds
+	local d
+	local DEVHTL=""
+
+	# exclude the last disk in the disk pool
+	n=$(($n-1))
+	ds=${devs[@]:0:$n}
+
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs "$ds" > /dev/null 2>&1 || _fail "tr:
mkfs failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+
+	#pick the 2nd last disk 
+	ds=${devs[@]:$(($n-1)):1}
+
+	# retrive the HTL for this scsi disk
+	d=`echo $ds|cut -d"/" -f3`
+	DEVHTL=`ls -l /sys/class/block/${d} | rev | cut -d "/" -f 3 | rev`
+
+	#fail disk
+	_devmgt_remove ${DEVHTL}
+	dev_removed=1
+
+	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" >
/dev/null || _fail \
+							"btrfs did not report device missing"
+
+	# add a new disk to btrfs
+	ds=${devs[@]:$(($n)):1}
+	btrfs device add ${ds} $SCRATCH_MNT > /dev/null 2>&1 || _fail
"dev add failed"
+	# in some system balance fails if there is no delay (a bug)
+	# putting sleep 10 to work around as of now
+	# sleep 10
+	btrfs fi balance $SCRATCH_MNT || _fail "dev balance failed"
+	#btrfs filesystem show | egrep devid |awk ''{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}'' || _fail "btrfs balance failed"
+
+	# cleaup. add the removed disk
+	umount $SCRATCH_MNT
+	_devmgt_add "${DEVHTL}"
+	dev_removed=0
+}
+
+_test_remove()
+{
+	_scratch_mkfs "$SCRATCH_DEV_POOL" > /dev/null 2>&1 ||
_fail "mkfs failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+
+	# pick last dev in the list
+	dev_del=`echo ${SCRATCH_DEV_POOL} | awk ''{print $NF}''`
+	btrfs device delete $dev_del $SCRATCH_MNT || _fail "btrfs device delete
failed"
+	btrfs fi show $SCRATCH_DEV 2>&1 | grep $dev_del > /dev/null
&& _fail "btrfs still shows the deleted dev"
+	umount $SCRATCH_MNT
+}
+
+_test_raid0
+_test_raid1
+_test_raid10
+_test_single
+_test_add
+_test_replace
+_test_remove
+
+echo "Silence is golden"
+status=0; exit
diff --git a/266.out b/266.out
new file mode 100644
index 0000000..54ba60f
--- /dev/null
+++ b/266.out
@@ -0,0 +1,2 @@
+QA output created by 266
+Silence is golden
diff --git a/group b/group
index 6bac8b5..817d245 100644
--- a/group
+++ b/group
@@ -379,3 +379,4 @@ deprecated
 263 rw auto quick
 264 auto quick
 265 auto quick
+266 auto
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Oct-19 09:42 UTC

head link

Re: [PATCH 1/3] 264: Functional test case for the btrfs snapshot

On Tue, Oct 18, 2011 at 02:28:54PM +0800, Anand Jain
wrote:> Create snapshots in various ways, modify the data around the block and
> file boundaries and verify the data integrity.
The test itselt looks good enough, but I have some comments on the
pool infrastructure changes.  I also think they should probably be
a separate preparatory patch, or at least documented in the changelog
as well.
> index 5367be6..7c135c7 100644
> --- a/README
> +++ b/README
> @@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
>                not be run.
>                
>          (these must be two DIFFERENT partitions)
> +
> +    - for btrfs only: some tests would need 3 or more independent SCRATCH
disks,
> +      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
> +              
>                
>      - setup your environment
>          - setenv TEST_DEV "device containing TEST PARTITION"
>          - setenv TEST_DIR "mount point of TEST PARTITION"   
>         	- optionally:
>               - setenv SCRATCH_DEV "device containing SCRATCH
PARTITION"
> +             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for
testing btrfs"
How does one find out what the pool name is?  You''ll also need to
document how to create the pool from disks.
> @@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
>      exit 1
>  fi
>  
> +# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its
dev
> +# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward
compatibility
> +if [ "$HOSTOS" == "Linux" ]; then
> +    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
> +else
> +    FSTYP_tmp=xfs
> +fi
Why do we need a second FSTYP detection?  If the existing one isn''t
early enough make sure it''s done early enough instead of duplicating
it.
> --- a/common.rc
> +++ b/common.rc
> @@ -1498,7 +1498,11 @@ _nfiles()
>                  file=f$f
>                  echo > $file
>                  if [ $size -gt 0 ]; then
> -                    dd if=/dev/zero of=$file bs=1024 count=$size
> +		    if [ $randomdata == false ]; then
> +	                dd if=/dev/zero of=$file bs=1024 count=$size 2>&1
| _filter_dd
> +		    else
> +                    	dd if=/dev/urandom of=$file bs=1024 count=$size
2>&1 | _filter_dd
> +		    fi
I''d rather see the randomdata flag passed down explicitly to _descend
and
_nfiles rather than setting a magic environment variable.
> @@ -1508,7 +1512,11 @@ _nfiles()
>  _descend()
>  {
>          dirname=$1; depth=$2
> -        mkdir $dirname  || die "mkdir $dirname failed"
> +	if [ -d $dirname ]; then
> +		dirname=`mktemp -dq $dirname/dir.XXXXXX`
> +	else
> +	        mkdir $dirname  || die "mkdir $dirname failed"
> +	fi
Why would the directory here already exist?  This at least needs
very good documentation.  Also the indentation seems off compared
to the surrounding code.
> @@ -1550,6 +1559,7 @@ _populate_fs()
>          s)      size=$OPTARG;;
>          v)      verbose=true;;
>          r)      root=$OPTARG;;
> +	x)	randomdata=true;;
indendation is off again.
> +# scratch_dev_pool should contain the disks pool for the btrfs raid
> +_require_scratch_dev_pool()
> +{
> +	local i
> +	case "$FSTYP" in
> +	btrfs)
> +		if [ -z "$SCRATCH_DEV_POOL" ]
> +		then
For new code I''d generally prefer the more readable

		if [ ... ]; then

although the above form unfortunately still is fairly common in
xfsprogs.
> +# We will check if the device is virtual (eg: loop device) since it does
not
> +# have the delete entry-point. Otherwise SCSI and USB devices are fine. 
> +_require_deletable_scratch_dev_pool()
> +{
> +	local i
> +	local x
> +	for i in $SCRATCH_DEV_POOL; do
> +		x=`echo $i | cut -d"/" -f 3`
> +		ls -l /sys/class/block/${x} | grep -q "virtual" 
> +		if [ $? == "0" ]; then
> +			_notrun "$i is a virtual device which is not deletable"
> +		fi
> +	done
> +}
> +
> +# arg 1 is dev to remove and is output of the below eg.
> +# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
> +_devmgt_remove()
> +{
> +	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail
"Remove disk failed"
> +}
> +
> +# arg 1 is dev to add and is output of the below eg.
> +# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
> +_devmgt_add()
> +{
> +	local h
> +	local tdl
> +	# arg 1 will be in h:t:d:l format now in the h and "t d l"
format
> +	h=`echo ${1} | cut -d":" -f 1`
> +	tdl=`echo ${1} | cut -d":" -f 2-|sed ''s/:/
/g''`
> +
> +	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add
disk failed"
> +}
This code looks a bit fragile to me, but I think we can fix it on the go
if we encouter issues.
> diff --git a/group b/group
> index 2a8970c..cfbae8c 100644
> --- a/group
> +++ b/group
> @@ -377,3 +377,4 @@ deprecated
>  261 auto quick quota
>  262 auto quick quota
>  263 rw auto quick
> +264 auto quick
It might be worth to add pool or snaphot groups if you add more
tests like this.

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Oct-19 09:43 UTC

head link

Re: [PATCH 2/3] 265: Functional test case for the btrfs de-fragmentation

On Tue, Oct 18, 2011 at 02:28:55PM +0800, Anand Jain
wrote:> To verify the btrfs de-fragmentation does not fail.
Any reason you can''t simply fold btrfs defragmentation testing into
the existing common defragmentation test (218)?
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Oct-19 09:45 UTC

head link

Re: [PATCH 3/3] 266: Functional test case for the btrfs raid operations

> +    if [ $dev_removed == 1 ]; then
> +	umount $SCRATCH_MNT
> +        _devmgt_add "${DEVHTL}"
> +    fi
messy indendation.
> +	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
> +	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk
''{
> +	#	if ( $6 == "0.00" ) { exit 1 }
> +	#}'' || _fail "test6_add... failed"
what is the point of the commented out code?

otherwise looks good to me.
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-20 15:31 UTC

head link

Re: [PATCH 1/3] 264: Functional test case for the btrfs snapshot

comments in line.

On 19/10/2011 17:42, Christoph Hellwig wrote:> On Tue, Oct 18, 2011 at 02:28:54PM +0800, Anand Jain wrote:
>> Create snapshots in various ways, modify the data around the block and
>> file boundaries and verify the data integrity.
> 
> The test itselt looks good enough, but I have some comments on the
> pool infrastructure changes.  I also think they should probably be
> a separate preparatory patch, or at least documented in the changelog
> as well.
> 
>> index 5367be6..7c135c7 100644
>> --- a/README
>> +++ b/README
>> @@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
>>                not be run.
>>                
>>          (these must be two DIFFERENT partitions)
>> +
>> +    - for btrfs only: some tests would need 3 or more independent
SCRATCH disks,
>> +      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
>> +              
>>                
>>      - setup your environment
>>          - setenv TEST_DEV "device containing TEST PARTITION"
>>          - setenv TEST_DIR "mount point of TEST PARTITION"   
>>         	- optionally:
>>               - setenv SCRATCH_DEV "device containing SCRATCH
PARTITION"
>> +             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for
testing btrfs"
> 
> How does one find out what the pool name is?  You''ll also need to
> document how to create the pool from disks.
> 
  agreed.
>> @@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
>>      exit 1
>>  fi
>>  
>> +# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of
its dev
>> +# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward
compatibility
>> +if [ "$HOSTOS" == "Linux" ]; then
>> +    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
>> +else
>> +    FSTYP_tmp=xfs
>> +fi
> 
> Why do we need a second FSTYP detection?  If the existing one
isn''t
> early enough make sure it''s done early enough instead of
duplicating
> it.
  looks like its ok not to have FSTYP checked here, it will follow the
  following logic..

   btrfs FS OR any FS
     SCRATCH_DEV_POOL is unset and SCRATCH_DEV is set
      . test-case with _require_scratch_dev_pool will not run
      . test-case without _require_scratch_dev_pool will run

     SCRATCH_DEV_POOL is set and SCRATCH_DEV is unset
      . test-case with _require_scratch_dev_pool
          - runs only if FSTYP=btrfs
      . test-case without _require_scratch_dev_pool will run using first
        dev in the SCRATCH_DEV_POOL as a SCRATCH_DEV
          - if FSTYP=btrfs it includes SCRATCH_DEV_POOL disks to the FS
          - if FSTYP=non-btrfs SCRATCH_DEV_POOL is ignored

     SCRATCH_DEV_POOL is set and SCRATCH_DEV is set
      . reports error in the config

     SCRATCH_DEV_POOL is unset and SCRATCH_DEV is unset
      . no change

>> --- a/common.rc
>> +++ b/common.rc
>> @@ -1498,7 +1498,11 @@ _nfiles()
>>                  file=f$f
>>                  echo > $file
>>                  if [ $size -gt 0 ]; then
>> -                    dd if=/dev/zero of=$file bs=1024 count=$size
>> +		    if [ $randomdata == false ]; then
>> +	                dd if=/dev/zero of=$file bs=1024 count=$size
2>&1 | _filter_dd
>> +		    else
>> +                    	dd if=/dev/urandom of=$file bs=1024 count=$size
2>&1 | _filter_dd
>> +		    fi
> 
> I''d rather see the randomdata flag passed down explicitly to
_descend and
> _nfiles rather than setting a magic environment variable.
makes sense. added.

>> @@ -1508,7 +1512,11 @@ _nfiles()
>>  _descend()
>>  {
>>          dirname=$1; depth=$2
>> -        mkdir $dirname  || die "mkdir $dirname failed"
>> +	if [ -d $dirname ]; then
>> +		dirname=`mktemp -dq $dirname/dir.XXXXXX`
>> +	else
>> +	        mkdir $dirname  || die "mkdir $dirname failed"
>> +	fi
> 
> Why would the directory here already exist?  This at least needs
> very good documentation.  Also the indentation seems off compared
> to the surrounding code.
  Hmm. Changing this back to the original design - where calling
  function has to ensure a new directory name which does not exists.
>> @@ -1550,6 +1559,7 @@ _populate_fs()
>>          s)      size=$OPTARG;;
>>          v)      verbose=true;;
>>          r)      root=$OPTARG;;
>> +	x)	randomdata=true;;
> 
> indendation is off again.
oh! thks for pointing.
>> +# scratch_dev_pool should contain the disks pool for the btrfs raid
>> +_require_scratch_dev_pool()
>> +{
>> +	local i
>> +	case "$FSTYP" in
>> +	btrfs)
>> +		if [ -z "$SCRATCH_DEV_POOL" ]
>> +		then
> 
> For new code I''d generally prefer the more readable
> 
> 		if [ ... ]; then
> 
> although the above form unfortunately still is fairly common in
> xfsprogs.
got it.
>> +# We will check if the device is virtual (eg: loop device) since it
does not
>> +# have the delete entry-point. Otherwise SCSI and USB devices are
fine.
>> +_require_deletable_scratch_dev_pool()
>> +{
>> +	local i
>> +	local x
>> +	for i in $SCRATCH_DEV_POOL; do
>> +		x=`echo $i | cut -d"/" -f 3`
>> +		ls -l /sys/class/block/${x} | grep -q "virtual" 
>> +		if [ $? == "0" ]; then
>> +			_notrun "$i is a virtual device which is not deletable"
>> +		fi
>> +	done
>> +}
>> +
>> +# arg 1 is dev to remove and is output of the below eg.
>> +# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
>> +_devmgt_remove()
>> +{
>> +	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail
"Remove disk failed"
>> +}
>> +
>> +# arg 1 is dev to add and is output of the below eg.
>> +# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
>> +_devmgt_add()
>> +{
>> +	local h
>> +	local tdl
>> +	# arg 1 will be in h:t:d:l format now in the h and "t d l"
format
>> +	h=`echo ${1} | cut -d":" -f 1`
>> +	tdl=`echo ${1} | cut -d":" -f 2-|sed ''s/:/
/g''`
>> +
>> +	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail
"Add disk failed"
>> +}
> 
> This code looks a bit fragile to me, but I think we can fix it on the go
> if we encouter issues.
  ok.
>> diff --git a/group b/group
>> index 2a8970c..cfbae8c 100644
>> --- a/group
>> +++ b/group
>> @@ -377,3 +377,4 @@ deprecated
>>  261 auto quick quota
>>  262 auto quick quota
>>  263 rw auto quick
>> +264 auto quick
> 
> It might be worth to add pool or snaphot groups if you add more
> tests like this.
  this can be considered when appropriate I think, as of now there is
  one test case in each of them.
Thanks

Anand
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-20 15:32 UTC

head link

Re: [PATCH 2/3] 265: Functional test case for the btrfs de-fragmentation

On 19/10/2011 17:43, Christoph Hellwig wrote:> On Tue, Oct 18, 2011 at 02:28:55PM +0800, Anand Jain wrote:
>> To verify the btrfs de-fragmentation does not fail.
> 
> Any reason you can''t simply fold btrfs defragmentation testing
into
> the existing common defragmentation test (218)?
  oh! 218 ! let me defer defrag for now. will get the other test-cases
  going. thanks
anand
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-20 15:32 UTC

head link

Re: [PATCH 3/3] 266: Functional test case for the btrfs raid operations

On 19/10/2011 17:45, Christoph Hellwig wrote:>> +    if [ $dev_removed == 1 ]; then
>> +	umount $SCRATCH_MNT
>> +        _devmgt_add "${DEVHTL}"
>> +    fi
> 
> messy indendation.
  got it. thanks.
>> +	btrfs filesystem balance $SCRATCH_MNT || _fail "balance
failed"
>> +	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk
''{
>> +	#	if ( $6 == "0.00" ) { exit 1 }
>> +	#}'' || _fail "test6_add... failed"
> 
> what is the point of the commented out code?
> 
> otherwise looks good to me.
  yes that has to be removed.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-20 15:41 UTC

head link

[PATCH 0/5] xfstests enhancement and bug fix

Anand Jain (5):
  updating to fill files with random data
  Added SCRATCH_DEV_POOL to specify multiple disks for the btrfs RAID
  264: Functional test case for the btrfs snapshot
  265: Functional test case for the btrfs raid operations
  _populate_fs should use OPTIND when getopts is used

 264           |  193 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out       |    2 +
 265           |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out       |    2 +
 README        |   16 ++++-
 common.config |   11 +++
 common.rc     |   96 ++++++++++++++++++++++++++--
 group         |    2 +
 8 files changed, 498 insertions(+), 11 deletions(-)
 create mode 100755 264
 create mode 100644 264.out
 create mode 100755 265
 create mode 100644 265.out

-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-20 15:41 UTC

head link

[PATCH 1/5] fill files with random data

snapshot data integrity test-case needs filesystem with random data.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 common.rc |   21 ++++++++++++++-------
 1 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/common.rc b/common.rc
index e948169..37379a5 100644
--- a/common.rc
+++ b/common.rc
@@ -1490,6 +1490,7 @@ _die()
         exit 1
 }
 
+#takes files, randomdata
 _nfiles()
 {
         f=0
@@ -1498,20 +1499,24 @@ _nfiles()
                 file=f$f
                 echo > $file
                 if [ $size -gt 0 ]; then
-                    dd if=/dev/zero of=$file bs=1024 count=$size
+                    if [ $2 == false ]; then
+                        dd if=/dev/zero of=$file bs=1024 count=$size
2>&1 | _filter_dd
+                    else
+                        dd if=/dev/urandom of=$file bs=1024 count=$size
2>&1 | _filter_dd
+                    fi
                 fi
 		let f=$f+1
         done
 }
 
-# takes dirname, depth
+# takes dirname, depth, randomdata
 _descend()
 {
-        dirname=$1; depth=$2
+        dirname=$1; depth=$2; randomdata=$3
         mkdir $dirname  || die "mkdir $dirname failed"
         cd $dirname
 
-        _nfiles $files           # files for this dir
+        _nfiles $files $randomdata          # files for this dir and data type
 
         [ $depth -eq 0 ] && return
 	let deep=$depth-1 # go 1 down
@@ -1529,7 +1534,7 @@ _descend()
 
 # Populate a filesystem with inodes for performance experiments
 #
-# usage: populate [-v] [-n ndirs] [-f nfiles] [-d depth] [-r root] [-s size]
+# usage: populate [-v] [-n ndirs] [-f nfiles] [-d depth] [-r root] [-s size]
[-x]
 #
 _populate_fs()
 {
@@ -1540,8 +1545,9 @@ _populate_fs()
     depth=2         # depth of tree from root to leaves
     verbose=false
     root=root       # path of initial root of directory tree
+    randomdata=false # -x data type urandom or zero
 
-    while getopts "d:f:n:r:s:v" c
+    while getopts "d:f:n:r:s:v:x" c
     do
         case $c in
         d)      depth=$OPTARG;;
@@ -1550,10 +1556,11 @@ _populate_fs()
         s)      size=$OPTARG;;
         v)      verbose=true;;
         r)      root=$OPTARG;;
+        x)      randomdata=true;;
         esac
     done
 
-    _descend $root $depth
+    _descend $root $depth $randomdata
     wait
 
     cd $here
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-20 15:41 UTC

head link

[PATCH 2/5] Added SCRATCH_DEV_POOL to specify multiple disks for the btrfs RAID

SCRATCH_DEV takes single disk as the scratch place for testing. New
SCRATCH_DEV_POOL can used to specify multiple disks for the scratch
btrfs filesystem.

Using SCRATCH_DEV and or SCRATCH_DEV_POOL will follow the following logic.
  btrfs FS OR any FS
    SCRATCH_DEV_POOL is unset and SCRATCH_DEV is set
     . test-case with _require_scratch_dev_pool will not run
     . test-case without _require_scratch_dev_pool will run

    SCRATCH_DEV_POOL is set and SCRATCH_DEV is unset
     . test-case with _require_scratch_dev_pool
         - runs only if FSTYP=btrfs
     . test-case without _require_scratch_dev_pool will run using first
       dev in the SCRATCH_DEV_POOL as a SCRATCH_DEV
         - if FSTYP=btrfs it includes SCRATCH_DEV_POOL disks to the FS
         - if FSTYP=non-btrfs SCRATCH_DEV_POOL is ignored

    SCRATCH_DEV_POOL is set and SCRATCH_DEV is set
     . reports error in the config

    SCRATCH_DEV_POOL is unset and SCRATCH_DEV is unset
     . no change

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 README        |   16 +++++++++---
 common.config |   11 ++++++++
 common.rc     |   76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 98 insertions(+), 5 deletions(-)

diff --git a/README b/README
index 5367be6..0f52f02 100644
--- a/README
+++ b/README
@@ -34,14 +34,22 @@ Preparing system for tests (IRIX and Linux):
             - leave empty and expect this partition to be clobbered
               by some tests.  If this is not provided, many tests will
               not be run.
-              
-        (these must be two DIFFERENT partitions)
+              (SCRATCH and TEST must be two DIFFERENT partitions)
+              OR
+        - for btrfs only: some btrfs test cases will need 3 or more independent
+              SCRATCH disks which should be set using SCRATCH_DEV_POOL (for eg:
+              SCRATCH_DEV_POOL="/dev/sda /dev/sdb /dev/sdc") with
which
+              SCRATCH_DEV should be unused by the tester, and for the legacy
+              support SCRATCH_DEV will be set to the first disk of the
+              SCRATCH_DEV_POOL by xfstests script.
               
     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"   
        	- optionally:
-             - setenv SCRATCH_DEV "device containing SCRATCH
PARTITION"
+             - setenv SCRATCH_DEV "device containing SCRATCH
PARTITION" OR
+               (btrfs only) setenv SCRATCH_DEV_POOL "to 3 or more SCRATCH
disks for
+               testing btrfs raid concepts"
              - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION"
              - setenv TAPE_DEV "tape device for testing xfsdump"
              - setenv RMT_TAPE_DEV "remote tape device for testing
xfsdump"
@@ -63,7 +71,7 @@ Preparing system for tests (IRIX and Linux):
       tape which can be overwritten.
           
     - make sure $TEST_DEV is a mounted XFS partition
-    - make sure that $SCRATCH_DEV contains nothing useful
+    - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing useful
     
 Running tests:
 
diff --git a/common.config b/common.config
index e94624e..1df2bbd 100644
--- a/common.config
+++ b/common.config
@@ -229,6 +229,17 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward
compatibility
+if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+    if [ ! -z "$SCRATCH_DEV" ]; then
+        echo "common.config: Error: \$SCRATCH_DEV should be unset when
\$SCRATCH_DEV_POOL is set"
+        exit 1
+    fi
+    SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | awk ''{print $1}''`
+    SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | awk ''{ for (i = 2; i
<= NF; i++) print $i}''`
+fi
+
 echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a
"$?" != "0" ]; then
     echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a
block device or a NFS filesystem"
diff --git a/common.rc b/common.rc
index 37379a5..e3c4e67 100644
--- a/common.rc
+++ b/common.rc
@@ -1499,7 +1499,7 @@ _nfiles()
                 file=f$f
                 echo > $file
                 if [ $size -gt 0 ]; then
-                    if [ $2 == false ]; then
+                    if [ "$2" == "false" ]; then
                         dd if=/dev/zero of=$file bs=1024 count=$size
2>&1 | _filter_dd
                     else
                         dd if=/dev/urandom of=$file bs=1024 count=$size
2>&1 | _filter_dd
@@ -1598,6 +1598,80 @@ _test_inode_extsz()
     echo $blocks
 }
 
+# scratch_dev_pool should contain the disks pool for the btrfs raid
+_require_scratch_dev_pool()
+{
+	local i
+	if [ -z "$SCRATCH_DEV_POOL" ]; then
+		_notrun "this test requires a valid \$SCRATCH_DEV_POOL"
+	fi
+
+	# btrfs test case needs 2 or more scratch_dev_pool; other FS not sure
+	# so fail it
+	case $FSTYP in
+	btrfs)
+		if [ "`echo $SCRATCH_DEV_POOL|wc -w`" -lt 2 ]; then
+			_notrun "btrfs and this test needs 2 or more disks in
SCRATCH_DEV_POOL"
+		fi
+	;;
+	*)
+		_notrun "dev_pool is not supported by fstype \"$FSTYP\""
+	;;
+	esac
+
+	for i in $SCRATCH_DEV_POOL; do
+		if [ "`_is_block_dev $i`" = "" ]; then
+			_notrun "this test requires valid block disk $i"
+		fi
+		if [ "`_is_block_dev $i`" = "`_is_block_dev $TEST_DEV`"
]; then
+			_notrun "$i is part of TEST_DEV, this test requires unique disks"
+		fi
+		if _mount | grep -q $i; then
+			if ! $UMOUNT_PROG $i; then
+		            echo "failed to unmount $i - aborting"
+		            exit 1
+		        fi
+		fi
+		# to help better debug when something fails, we remove
+		# traces of previous btrfs FS on the dev.
+		dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
+	done
+}
+
+# We will check if the device is virtual (eg: loop device) since it does not
+# have the delete entry-point. Otherwise SCSI and USB devices are fine. 
+_require_deletable_scratch_dev_pool()
+{
+	local i
+	local x
+	for i in $SCRATCH_DEV_POOL; do
+		x=`echo $i | cut -d"/" -f 3`
+		ls -l /sys/class/block/${x} | grep -q "virtual" 
+		if [ $? == "0" ]; then
+			_notrun "$i is a virtual device which is not deletable"
+		fi
+	done
+}
+
+# arg 1 is dev to remove and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_remove()
+{
+	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail "Remove
disk failed"
+}
+
+# arg 1 is dev to add and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_add()
+{
+	local h
+	local tdl
+	# arg 1 will be in h:t:d:l format now in the h and "t d l" format
+	h=`echo ${1} | cut -d":" -f 1`
+	tdl=`echo ${1} | cut -d":" -f 2-|sed ''s/:/ /g''`
+
+	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add disk
failed"
+}
 
 ################################################################################
 
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-20 15:41 UTC

head link

[PATCH 3/5] 264: Functional test case for the btrfs snapshot

Create snapshots in various ways, modify the data around the block and
file boundaries and verify the data integrity.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 264     |  193 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out |    2 +
 group   |    1 +
 3 files changed, 196 insertions(+), 0 deletions(-)
 create mode 100755 264
 create mode 100644 264.out

diff --git a/264 b/264
new file mode 100755
index 0000000..d655d1e
--- /dev/null
+++ b/264
@@ -0,0 +1,193 @@
+#!/bin/bash
+# FS QA Test No. 264
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot
create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null
&
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer
2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer
2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer
2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts
here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume
create $firstvol failed"
+dirp=`mktemp -duq $firstvol/dir.XXXXXX`
+_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10 -x
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit
diff --git a/264.out b/264.out
new file mode 100644
index 0000000..eca790b
--- /dev/null
+++ b/264.out
@@ -0,0 +1,2 @@
+QA output created by 264
+Silence is golden
diff --git a/group b/group
index 2a8970c..d2abe42 100644
--- a/group
+++ b/group
@@ -377,3 +377,4 @@ deprecated
 261 auto quick quota
 262 auto quick quota
 263 rw auto quick
+264 auto
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-20 15:41 UTC

head link

[PATCH 4/5] 265: Functional test case for the btrfs raid operations

This will verify the various raid features in btrfs and device
replacement functionality.

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 265     |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out |    2 +
 group   |    1 +
 3 files changed, 190 insertions(+), 0 deletions(-)
 create mode 100755 265
 create mode 100644 265.out

diff --git a/265 b/265
new file mode 100755
index 0000000..10f214f
--- /dev/null
+++ b/265
@@ -0,0 +1,187 @@
+#! /bin/bash
+# FS QA Test No. 265
+#
+# btrfs vol tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
+owner=anand.jain@oracle.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+dev_removed=0
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+    if [ $dev_removed == 1 ]; then
+        umount $SCRATCH_MNT
+        _devmgt_add "${DEVHTL}"
+    fi
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool
+_require_deletable_scratch_dev_pool
+
+# Test cases related to raid in btrfs
+_test_raid0()
+{
+	export MKFS_OPTIONS="-m raid0 -d raid0"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_raid1()
+{
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_raid10()
+{
+	export MKFS_OPTIONS="-m raid10 -d raid10"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_single()
+{
+	export MKFS_OPTIONS="-m single -d single"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs
failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_add()
+{
+	local i
+	local devs[]="( $SCRATCH_DEV_POOL )"
+	local n=${#devs[@]}
+
+	n=$(($n-1))
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+	for i in `seq 1 $n`; do
+		btrfs device add ${devs[$i]} $SCRATCH_MNT > /dev/null 2>&1 || _fail
"device add failed"
+	done
+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	umount $SCRATCH_MNT
+}
+
+_test_replace()
+{
+	local i
+	local devs=( $SCRATCH_DEV_POOL )
+	local n=${#devs[@]}
+	local ds
+	local d
+	local DEVHTL=""
+
+	# exclude the last disk in the disk pool
+	n=$(($n-1))
+	ds=${devs[@]:0:$n}
+
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs "$ds" > /dev/null 2>&1 || _fail "tr:
mkfs failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+
+	#pick the 2nd last disk 
+	ds=${devs[@]:$(($n-1)):1}
+
+	# retrive the HTL for this scsi disk
+	d=`echo $ds|cut -d"/" -f3`
+	DEVHTL=`ls -l /sys/class/block/${d} | rev | cut -d "/" -f 3 | rev`
+
+	#fail disk
+	_devmgt_remove ${DEVHTL}
+	dev_removed=1
+
+	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" >
/dev/null || _fail \
+							"btrfs did not report device missing"
+
+	# add a new disk to btrfs
+	ds=${devs[@]:$(($n)):1}
+	btrfs device add ${ds} $SCRATCH_MNT > /dev/null 2>&1 || _fail
"dev add failed"
+	# in some system balance fails if there is no delay (a bug)
+	# putting sleep 10 to work around as of now
+	# sleep 10
+	btrfs fi balance $SCRATCH_MNT || _fail "dev balance failed"
+
+	# cleaup. add the removed disk
+	umount $SCRATCH_MNT
+	_devmgt_add "${DEVHTL}"
+	dev_removed=0
+}
+
+_test_remove()
+{
+	_scratch_mkfs "$SCRATCH_DEV_POOL" > /dev/null 2>&1 ||
_fail "mkfs failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+
+	# pick last dev in the list
+	dev_del=`echo ${SCRATCH_DEV_POOL} | awk ''{print $NF}''`
+	btrfs device delete $dev_del $SCRATCH_MNT || _fail "btrfs device delete
failed"
+	btrfs fi show $SCRATCH_DEV 2>&1 | grep $dev_del > /dev/null
&& _fail "btrfs still shows the deleted dev"
+	umount $SCRATCH_MNT
+}
+
+_test_raid0
+_test_raid1
+_test_raid10
+_test_single
+_test_add
+_test_replace
+_test_remove
+
+echo "Silence is golden"
+status=0; exit
diff --git a/265.out b/265.out
new file mode 100644
index 0000000..9fa4eb0
--- /dev/null
+++ b/265.out
@@ -0,0 +1,2 @@
+QA output created by 265
+Silence is golden
diff --git a/group b/group
index d2abe42..94ddef6 100644
--- a/group
+++ b/group
@@ -378,3 +378,4 @@ deprecated
 262 auto quick quota
 263 rw auto quick
 264 auto
+265 auto
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anand Jain

2011-Oct-20 15:41 UTC

head link

[PATCH 5/5] _populate_fs should use OPTIND when getopts is used

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
---
 common.rc |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/common.rc b/common.rc
index e3c4e67..cab0b64 100644
--- a/common.rc
+++ b/common.rc
@@ -1547,6 +1547,7 @@ _populate_fs()
     root=root       # path of initial root of directory tree
     randomdata=false # -x data type urandom or zero
 
+    OPTIND=1
     while getopts "d:f:n:r:s:v:x" c
     do
         case $c in
-- 
1.7.7

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Oct-25 11:35 UTC

head link

Re: [PATCH 1/5] fill files with random data

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Oct-25 11:36 UTC

head link

Re: [PATCH 2/5] Added SCRATCH_DEV_POOL to specify multiple disks for the btrfs RAID

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Oct-25 11:36 UTC

head link

Re: [PATCH 3/5] 264: Functional test case for the btrfs snapshot

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Oct-25 11:37 UTC

head link

Re: [PATCH 4/5] 265: Functional test case for the btrfs raid operations

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-Oct-25 11:37 UTC

head link

Re: [PATCH 5/5] _populate_fs should use OPTIND when getopts is used

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>
--
To unsubscribe from this list: send the line "unsubscribe
linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Btrfs devel - Aug 2011 - [PATCH] snapshot, defragment and raid test cases for btrfs

[PATCH] snapshot, defragment and raid test cases for btrfs

Re: [PATCH] snapshot, defragment and raid test cases for btrfs

Re: [PATCH] snapshot, defragment and raid test cases for btrfs

Re: [PATCH] snapshot, defragment and raid test cases for btrfs

Re: [PATCH] snapshot, defragment and raid test cases for btrfs

Re: [PATCH] snapshot, defragment and raid test cases for btrfs

[PATCH 1/3] Added test case 257 for btrfs extended snapshot tests

[PATCH 2/3] Added test case 258 for btrfs defragmentation

[PATCH 3/3] Added test case 259 for the btrfs raid features

[PATCH v2 1/3] Added test case 257 for btrfs extended snapshot tests

Re: [PATCH 1/3] Added test case 257 for btrfs extended snapshot tests

Re: [PATCH 3/3] Added test case 259 for the btrfs raid features

Re: [PATCH 3/3] Added test case 259 for the btrfs raid features

[PATCH] Changes to received review comments

Re: [PATCH] Changes to received review comments

Re: [PATCH] Changes to received review comments

[PATCH 1/3] 263: Functional test case for the btrfs snapshot

[PATCH 2/3] 264: Functional test case for the btrfs de-fragmentation

[PATCH 3/3] 265: Functional test case for the btrfs raid operations

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

[PATCH 0/3] xfstest patch

[PATCH 1/3] 263: Functional test case for the btrfs snapshot

[PATCH 2/3] 264: Functional test case for the btrfs de-fragmentation

[PATCH 3/3] 265: Functional test case for the btrfs raid operations

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

[PATCH 0/3] xfstests patches

[PATCH 1/3] 264: Functional test case for the btrfs snapshot

[PATCH 2/3] 265: Functional test case for the btrfs de-fragmentation

[PATCH 3/3] 266: Functional test case for the btrfs raid operations

Re: [PATCH 1/3] 264: Functional test case for the btrfs snapshot

Re: [PATCH 2/3] 265: Functional test case for the btrfs de-fragmentation

Re: [PATCH 3/3] 266: Functional test case for the btrfs raid operations

Re: [PATCH 1/3] 264: Functional test case for the btrfs snapshot

Re: [PATCH 2/3] 265: Functional test case for the btrfs de-fragmentation

Re: [PATCH 3/3] 266: Functional test case for the btrfs raid operations

[PATCH 0/5] xfstests enhancement and bug fix

[PATCH 1/5] fill files with random data

[PATCH 2/5] Added SCRATCH_DEV_POOL to specify multiple disks for the btrfs RAID

[PATCH 3/5] 264: Functional test case for the btrfs snapshot

[PATCH 4/5] 265: Functional test case for the btrfs raid operations

[PATCH 5/5] _populate_fs should use OPTIND when getopts is used

Re: [PATCH 1/5] fill files with random data

Re: [PATCH 2/5] Added SCRATCH_DEV_POOL to specify multiple disks for the btrfs RAID

Re: [PATCH 3/5] 264: Functional test case for the btrfs snapshot

Re: [PATCH 4/5] 265: Functional test case for the btrfs raid operations

Re: [PATCH 5/5] _populate_fs should use OPTIND when getopts is used