thr3ads.net - Gluster users - [Gluster-users] Unify over AFR with pre-existing data [Jul 2008]

If this information is useful, please help other people find it:
Share via:

baggio liu

2008-Jul-23 02:11 UTC

[Gluster-users] Unify over AFR with pre-existing data

Hi guys,
   I have configured a cluster with unify over afr. one of server in afr has
some pre-existing  data.  And I want to make these data can be seen by
client.
   Some  config have been done according to
      "*Setting up AFR on two servers with pre-existing data*"

http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data
   But namespace can not aware with those data.
   After afr completing,* ls* at client , pre-existing  data can not be
seen. *cat  *them,* Input/output error* can be found.


   What should I fix them? Thank you for any help.

Regards
Baggio
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20080723/2ccc79ce/attachment.html>

Keith Freedman

2008-Jul-23 06:04 UTC

head link

[Gluster-users] Unify over AFR with pre-existing data

I had 2 situations with pre-existing data and I solved the both 2 
different ways.

1), I followed the instructions in the link provided, however, I was 
running xfs on the underlying filesystem and so the command to add 
attributes is different.
in this case, it's attr instead of setfattr to set the extended 
attribute.  Also, I *think* the actual attribute name is different.
so, what I recommend is to create one file through gluster, view it's 
extended attributes through the base filesystem then apply those to 
the remaining files.

2) This is the less friendly but more surefire solution:
make the gluster source point a sub directory in the source filesystem.
then "move" the files into the gluster filesystem.
This is less handy for "live" data, but it does insure the existing 
data is properly "glustered"

here's what I did:
CURRENT SOURCE: /home
GLUSTER SOURCE /home/home
GLUSTER MOUNT POINT /gluster/home

(this may not be syntatically correct):
cd /home
for i in *
do
if [ $i != 'glusterhome' ]
   mv $i glusterhome
   ln -s gluster/home/$i $i
fi

then unmount /gluster/home, unmount /home   remount /home on /gluster
now you have /gluster/home with your data which has the gluster 
attributes attached.
mount glusterfs from /gluster/home  to /home

I actually used this second procedure  to get the data onto my new 
AFR mirror.  It slowed the process down since it was replicating 50GB 
of data while moving it form disk to disk, but I'm not sure it slowed 
it down much.  it seems to really mostly just reduce disk thrashing 
and my copy was going at near network speed.

Keith
At 07:11 PM 7/22/2008, baggio liu wrote:>Hi guys,
>    I have configured a cluster with unify over afr. one of server 
> in afr has some pre-existing  data.  And I want to make these data 
> can be seen by client.
>    Some  config have been done according to
>       "Setting up AFR on two servers with pre-existing data"
> 
><http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data>http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data
>    But namespace can not aware with those data.
>    After afr completing, ls at client , pre-existing  data can not 
> be seen. cat  them, Input/output error can be found.
>
>
>    What should I fix them? Thank you for any help.
>
>Regards
>Baggio
>_______________________________________________
>Gluster-users mailing list
>Gluster-users at gluster.org
>http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users

Keith Freedman

2008-Jul-23 06:04 UTC

head link

[Gluster-users] Unify over AFR with pre-existing data

I had 2 situations with pre-existing data and I solved the both 2 
different ways.

1), I followed the instructions in the link provided, however, I was 
running xfs on the underlying filesystem and so the command to add 
attributes is different.
in this case, it''s attr instead of setfattr to set the extended 
attribute.  Also, I *think* the actual attribute name is different.
so, what I recommend is to create one file through gluster, view it''s 
extended attributes through the base filesystem then apply those to 
the remaining files.

2) This is the less friendly but more surefire solution:
make the gluster source point a sub directory in the source filesystem.
then "move" the files into the gluster filesystem.
This is less handy for "live" data, but it does insure the existing 
data is properly "glustered"

here''s what I did:
CURRENT SOURCE: /home
GLUSTER SOURCE /home/home
GLUSTER MOUNT POINT /gluster/home

(this may not be syntatically correct):
cd /home
for i in *
do
if [ $i != ''glusterhome'' ]
   mv $i glusterhome
   ln -s gluster/home/$i $i
fi

then unmount /gluster/home, unmount /home   remount /home on /gluster
now you have /gluster/home with your data which has the gluster 
attributes attached.
mount glusterfs from /gluster/home  to /home

I actually used this second procedure  to get the data onto my new 
AFR mirror.  It slowed the process down since it was replicating 50GB 
of data while moving it form disk to disk, but I''m not sure it slowed 
it down much.  it seems to really mostly just reduce disk thrashing 
and my copy was going at near network speed.

Keith
At 07:11 PM 7/22/2008, baggio liu wrote:>Hi guys,
>    I have configured a cluster with unify over afr. one of server 
> in afr has some pre-existing  data.  And I want to make these data 
> can be seen by client.
>    Some  config have been done according to
>       "Setting up AFR on two servers with pre-existing data"
> 
><http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data>http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data
>    But namespace can not aware with those data.
>    After afr completing, ls at client , pre-existing  data can not 
> be seen. cat  them, Input/output error can be found.
>
>
>    What should I fix them? Thank you for any help.
>
>Regards
>Baggio
>_______________________________________________
>Gluster-users mailing list
>Gluster-users at gluster.org
>http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users

baggio liu

2008-Jul-23 06:47 UTC

head link

[Gluster-users] Unify over AFR with pre-existing data

Hi ,
    thanks for reply. And pls see my comment inline. And  question is still
pending. has any more grace way to fix this issue??

Regards
Baggio




2008/7/23 Keith Freedman <freedman at freeformit.com>:
> I had 2 situations with pre-existing data and I solved the both 2 different
> ways.
>
> 1), I followed the instructions in the link provided, however, I was
> running xfs on the underlying filesystem and so the command to add
> attributes is different.
> in this case, it's attr instead of setfattr to set the extended
attribute.
>  Also, I *think* the actual attribute name is different.

   The extension attr names are hard-code , so it should not change
according to underlying fs.

> so, what I recommend is to create one file through gluster, view it's
> extended attributes through the base filesystem then apply those to the
> remaining files.
>
> 2) This is the less friendly but more surefire solution:
> make the gluster source point a sub directory in the source filesystem.
> then "move" the files into the gluster filesystem.
> This is less handy for "live" data, but it does insure the
existing data is
> properly "glustered"

      This way can make data right, but too many extra cost has been
involved. In our plan, hundreds of TB should be stored in this fs , This
loop may cost several days . So it can not be tolerated.

>
>
> here's what I did:
> CURRENT SOURCE: /home
> GLUSTER SOURCE /home/home
> GLUSTER MOUNT POINT /gluster/home
>
> (this may not be syntatically correct):
> cd /home
> for i in *
> do
> if [ $i != 'glusterhome' ]
>  mv $i glusterhome
>  ln -s gluster/home/$i $i
> fi
>
> then unmount /gluster/home, unmount /home   remount /home on /gluster
> now you have /gluster/home with your data which has the gluster attributes
> attached.
> mount glusterfs from /gluster/home  to /home
>
> I actually used this second procedure  to get the data onto my new AFR
> mirror.  It slowed the process down since it was replicating 50GB of data
> while moving it form disk to disk, but I'm not sure it slowed it down
much.
>


> it seems to really mostly just reduce disk thrashing and my copy was going
> at near network speed.

        BTW, I'm not very clear of it, Could you explain in detail? or list
some example, test result, and so on. thx
>
>
> Keith
> At 07:11 PM 7/22/2008, baggio liu wrote:
>
>> Hi guys,
>>   I have configured a cluster with unify over afr. one of server in afr
>> has some pre-existing  data.  And I want to make these data can be seen
by
>> client.
>>   Some  config have been done according to
>>      "Setting up AFR on two servers with pre-existing data"
>>
>> <
>>
http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data
>> >
>>
http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data
>>   But namespace can not aware with those data.
>>   After afr completing, ls at client , pre-existing  data can not be
seen.
>> cat  them, Input/output error can be found.
>>
>>
>>   What should I fix them? Thank you for any help.
>>
>> Regards
>> Baggio
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>>
>
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20080723/d90e7471/attachment.html>

chawkins at bplinux.com

2008-Jul-23 06:55 UTC

head link

[Gluster-users] Unify over AFR with pre-existing data

Hi - I wrote a script to set attributes on pre-existing data, but I have done
very little testing with it so far. If you want to try it out it might save you
some manual labor. After some more testing I was going to post it to the list
and the wiki but it looks like you might be able to use it now... It works and
it seems safe enough as it doesn't do anything but add gluster attributes to
files which don't have any. Let me know if it works for you, and if anyone
has any suggestions for improvement.

Thanks, 
Chris 

#!/bin/bash 

DIR=$1 

if [ -z $DIR ] 
then 
echo 
echo "Error - no path was given." 
echo 
echo "USAGE: set-attrs /path/to/check" 
echo 
echo "Run set-attrs to check all files and directories in the" 
echo "given path for glusterfs attributes. Any files without" 
echo "a glusterfs version will be given a version attr of 2" 
echo "and a createtime attr equal to the files last modified time." 
echo 
echo "Output information is logged to syslog (usually
/var/log/messages)"
echo 
exit 0 
fi 

set_attrs() { 

get_ver="getfattr --absolute-names -n trusted.glusterfs.version" 
set_ver="setfattr -n trusted.glusterfs.version" 
set_ct="setfattr -n trusted.glusterfs.createtime" 

if test "`$get_ver $i 2>/dev/null | fgrep -c
trusted.glusterfs.version=`" = "0"
then 
mods=$((mods+1)) 
# comment out next line to turn off logging 
initlog -n set-attrs -s "Setting attrs on $i" 
CT=`stat -c %Y $i` 
$set_ct -v $CT $i 
$set_ver -v 2 $i 
fi 
} 

mods=0 
checks=0 

initlog -n watch-dir -s "Starting a set-attrs run on path: $DIR" 

for i in `find $DIR -print` 
do 
set_attrs $i 
checks=$((checks+1)) 
done 

initlog -n watch-dir -s "Completed. $checks checks and $mods mods in this
run"

exit 0 

----- Original Message ----- 
From: "baggio liu" <baggioss at gmail.com> 
To: "Keith Freedman" <freedman at freeformit.com>, gluster-users
at gluster.org
Sent: Wednesday, July 23, 2008 2:47:01 AM GMT -05:00 US/Canada Eastern 
Subject: Re: [Gluster-users] Unify over AFR with pre-existing data 


Hi , 
thanks for reply. And pls see my comment inline. And question is still pending.
has any more grace way to fix this issue??

Regards 
Baggio 





2008/7/23 Keith Freedman < freedman at freeformit.com >: 


I had 2 situations with pre-existing data and I solved the both 2 different
ways.

1), I followed the instructions in the link provided, however, I was running xfs
on the underlying filesystem and so the command to add attributes is different.
in this case, it's attr instead of setfattr to set the extended attribute.
Also, I *think* the actual attribute name is different.

The extension attr names are hard-code , so it should not change according to
underlying fs.




so, what I recommend is to create one file through gluster, view it's
extended attributes through the base filesystem then apply those to the
remaining files.

2) This is the less friendly but more surefire solution: 
make the gluster source point a sub directory in the source filesystem. 
then "move" the files into the gluster filesystem. 
This is less handy for "live" data, but it does insure the existing
data is properly "glustered"

This way can make data right, but too many extra cost has been involved. In our
plan, hundreds of TB should be stored in this fs , This loop may cost several
days . So it can not be tolerated.




here's what I did: 
CURRENT SOURCE: /home 
GLUSTER SOURCE /home/home 
GLUSTER MOUNT POINT /gluster/home 

(this may not be syntatically correct): 
cd /home 
for i in * 
do 
if [ $i != 'glusterhome' ] 
mv $i glusterhome 
ln -s gluster/home/$i $i 
fi 

then unmount /gluster/home, unmount /home remount /home on /gluster 
now you have /gluster/home with your data which has the gluster attributes
attached.
mount glusterfs from /gluster/home to /home 

I actually used this second procedure to get the data onto my new AFR mirror. It
slowed the process down since it was replicating 50GB of data while moving it
form disk to disk, but I'm not sure it slowed it down much.



it seems to really mostly just reduce disk thrashing and my copy was going at
near network speed.

BTW, I'm not very clear of it, Could you explain in detail? or list some
example, test result, and so on. thx




Keith 

At 07:11 PM 7/22/2008, baggio liu wrote: 



Hi guys, 
I have configured a cluster with unify over afr. one of server in afr has some
pre-existing data. And I want to make these data can be seen by client.
Some config have been done according to 
"Setting up AFR on two servers with pre-existing data" 

<
http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data
>
http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data

But namespace can not aware with those data. 
After afr completing, ls at client , pre-existing data can not be seen. cat
them, Input/output error can be found.


What should I fix them? Thank you for any help. 

Regards 
Baggio 
_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users 



_______________________________________________ Gluster-users mailing list
Gluster-users at gluster.org
http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20080723/e22d7d99/attachment.html>

Gluster users - Jul 2008 - Unify over AFR with pre-existing data

[Gluster-users] Unify over AFR with pre-existing data

[Gluster-users] Unify over AFR with pre-existing data

[Gluster-users] Unify over AFR with pre-existing data

[Gluster-users] Unify over AFR with pre-existing data

[Gluster-users] Unify over AFR with pre-existing data