Harry Putnam
2010-Feb-21 18:47 UTC
[zfs-discuss] More performance questions [on zfs over nfs]
Working from a remote linux machine on a zfs fs that is an nfs mounted share (set for nfs availability on zfs server, mounted nfs on linux); I''ve been noticing a certain kind of sloth when messing with files. What I see: After writing a file it seems to take the fs too long to be able to display the size correctly (with du). I''m wondering if I should be using some special flags either on the osol end or the linux end. Here is an example: rm -f file1; sleep 2;awk ''NR< 115000{print}'' debug.log.6 >file1;du -sh file1;sleep 3;du -sh file1;sleep 3; du -sh file1; ls -l file1; sleep 10; du -sh file1 512 file1 <tested right away with du -sh> 512 file1 <checked after 3 seconds> 512 file1 <checked after 6 seconds> <long `ls -l'' just to show the size du is failing to see> -rw-r--r-- 1 nobody nobody 10831062 Feb 21 12:33 file 512 file1 <final check after 16 seconds> Some time later the correct size becomes available... not sure how much. ------- --------- ---=--- --------- -------- If that line above gets too wrapped to understand easily it is: ## clear any file rm -f file ## brief sleep sleep 2 ## write a lot of lines to a file from some log file awk ''NR< 115000{print}'' debug.log.6 >file ## check the size right away du -sh file ## sleep 3 seconds sleep 3 ## check size again du -sh file ## sleep 3 more seconds sleep 3 ## check size again du -sh file ## sleep a final 5 seconds sleep 10 ## Check on last time du -sh file
Henrik Johansson
2010-Feb-22 02:25 UTC
[zfs-discuss] More performance questions [on zfs over nfs]
On Feb 21, 2010, at 7:47 PM, Harry Putnam wrote:> > Working from a remote linux machine on a zfs fs that is an nfs mounted > share (set for nfs availability on zfs server, mounted nfs on linux); > I''ve been noticing a certain kind of sloth when messing with files. > > What I see: After writing a file it seems to take the fs too long to > be able to display the size correctly (with du).You will not see the on disk size of the file with du before the transaction group have been committed which can take up to 30 seconds. ZFS does not even know how much space it will consume before writing out the data to disks since compression might be enabled. You can test this by executing sync(1M) on your file server, when it returns you should have the final size of the file. Regards Henrik http://sparcv9.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100222/a0f9b729/attachment.html>
Bob Friesenhahn
2010-Feb-22 05:38 UTC
[zfs-discuss] More performance questions [on zfs over nfs]
On Mon, 22 Feb 2010, Henrik Johansson wrote:> You will not see the on disk size of the file with du before the transaction group have been committed > which can take up to 30 seconds. ZFS does not even know how much space it will consume before writing out > the data to disks since compression might be enabled. You can test this by executing sync(1M) on your file > server, when it returns you should have the final size of the file.Even more interesting, if the file grows to 1GB and is re-written 5 times and is then truncated to a smaller size (using ftruncate()) before the next transaction group is written, then all of that transient activity at the larger size is as if it never happened. Eventually this is seen as a blessing. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Harry Putnam
2010-Feb-23 01:06 UTC
[zfs-discuss] More performance questions [on zfs over nfs]
[Note: This is a repost of question posted about 1.5 days ago that has never appeared on the group.. at least not on my server (gmane). Sorry if it ends up being a double whammy] Working from a remote linux machine on a zfs fs that is an nfs mounted share (set for nfs availability on zfs server, mounted nfs on linux); I''ve been noticing a certain kind of sloth when messing with files. What I see: After writing a file it seems to take the fs too long to be able to display the size correctly (with du). I''m wondering if I should be using some special flags either on the osol end or the linux end. Here is an example: rm -f file1; sleep 2;awk ''NR< 115000{print}'' debug.log.6 >file1;du -sh file1;sleep 3;du -sh file1;sleep 3; du -sh file1; ls -l file1; sleep 10; du -sh file1 512 file1 <tested right away with du -sh> 512 file1 <checked after 3 seconds> 512 file1 <checked after 6 seconds> <long `ls -l'' just to show the size du is failing to see> -rw-r--r-- 1 nobody nobody 10831062 Feb 21 12:33 file 512 file1 <final check after 16 seconds> Some time later the correct size becomes available... not sure how much. ------- --------- ---=--- --------- -------- If that line above gets too wrapped to understand easily it is: ## clear any file rm -f file ## brief sleep sleep 2 ## write a lot of lines to a file from some log file awk ''NR< 115000{print}'' debug.log.6 >file ## check the size right away du -sh file ## sleep 3 seconds sleep 3 ## check size again du -sh file ## sleep 3 more seconds sleep 3 ## check size again du -sh file ## sleep a final 5 seconds sleep 10 ## Check on last time du -sh file
Hi all, According to what i have been reading the opensolaris 2010.03 should be released around March this year, but with all the process of the Oracle/Sun deal i was wondering if anyone knows if this schedule still makes sense, and if not does snv_132/133 look very similar to future 2010.03. In other words, without waiting for the opensolaris 2010.03 would anyone risk to put in production any box with snv_132/133 ? Thanks, Bruno -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3656 bytes Desc: S/MIME Cryptographic Signature URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100223/5b30d3c2/attachment.bin>
Harry Putnam
2010-Feb-23 13:11 UTC
[zfs-discuss] More performance questions [on zfs over nfs]
Harry Putnam <reader at newsguy.com> writes:> [Note: This is a repost of question posted about 1.5 days ago that > has never appeared on the group.. at least not on my server (gmane). > Sorry if it ends up being a double whammy]Apparently I missed two informative answers by: Henrik J. Bob F. Thanks for the input... and technical summation, I still don''t see my original post but no matter, my second post threaded with the two authors answers and let me see them.
There is no clustering package for it and available source seems very old also the de-dup bug is there iirc. So if you don''t need HA cluster and dedup.. BR, Jeffry> -----Original Message----- > From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Bruno Sousa > Sent: dinsdag 23 februari 2010 8:37 > To: zfs-discuss at opensolaris.org > Subject: [zfs-discuss] Opensolaris 2010.03 / snv releases > > Hi all, > > According to what i have been reading the opensolaris 2010.03 should be > released around March this year, but with all the process of the Oracle/Sun > deal i was wondering if anyone knows if this schedule still makes sense, > and if not does snv_132/133 look very similar to future 2010.03. > In other words, without waiting for the opensolaris 2010.03 would anyone > risk to put in production any box with snv_132/133 ? > > Thanks, > Bruno
Isn''t the dedupe bug fixed in svn133? -marc On Tue, Feb 23, 2010 at 9:21 AM, Jeffry Molanus <Jeffry.Molanus at proact.nl>wrote:> There is no clustering package for it and available source seems very old > also the de-dup bug is there iirc. So if you don''t need HA cluster and > dedup.. > > BR, Jeffry > > > -----Original Message----- > > From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > > bounces at opensolaris.org] On Behalf Of Bruno Sousa > > Sent: dinsdag 23 februari 2010 8:37 > > To: zfs-discuss at opensolaris.org > > Subject: [zfs-discuss] Opensolaris 2010.03 / snv releases > > > > Hi all, > > > > According to what i have been reading the opensolaris 2010.03 should be > > released around March this year, but with all the process of the > Oracle/Sun > > deal i was wondering if anyone knows if this schedule still makes sense, > > and if not does snv_132/133 look very similar to future 2010.03. > > In other words, without waiting for the opensolaris 2010.03 would anyone > > risk to put in production any box with snv_132/133 ? > > > > Thanks, > > Bruno > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100223/13164cac/attachment.html>