I am running the following test. I have 10 clients and 5 OSTs. I have set up striping so that pairs of clients are writing to an OST. That is, clients 0 + 4 write to ost-zero, clients 1 + 5 write to ost-one, etc. Each cilent is creating a large file. What I see is that one client creates its file much faster than the other client that goes to the same OST. So, for example, in the following directory listing, the files zero/n100 and zero/n122 are both going to OST-zero. Each is created from a client, and the two clients are identical. The file zero/n100 is about 3.3G, while the file zero/n122 is about 30G. Similarly for files two/n102 and two/n150, which are 4.5G and 28.9G respectively (they got to OST-two). Is there any way to make this fairer? I am running Lustre 1.4.8 on AMD-64 bit machines. -Roger _________________________________________________________________ Mortgage rates as low as 4.625% - Refinance $150,000 loan for $579 a month. Intro*Terms https://www2.nextag.com/goto.jsp?product=100000035&url=%2fst.jsp&tm=y&search=mortgage_text_links_88_h27f6&disc=y&vers=743&s=4056&p=5117
On Mar 02, 2007 13:40 -0500, RS RS wrote:> I am running the following test. I have 10 clients and 5 OSTs. I have set > up striping so that pairs of clients are writing to an OST. That is, > clients 0 + 4 write to ost-zero, clients 1 + 5 write to ost-one, etc. > > Each cilent is creating a large file. > > What I see is that one client creates its file much faster than the other > client that goes to the same OST. So, for example, in the following > directory listing, the files zero/n100 and zero/n122 are both going to > OST-zero. Each is created from a client, and the two clients are > identical. The file zero/n100 is about 3.3G, while the file zero/n122 is > about 30G. Similarly for files two/n102 and two/n150, which are 4.5G and > 28.9G respectively (they got to OST-two). > > Is there any way to make this fairer?What do you mean by "fair"? Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
>On Mar 02, 2007 13:40 -0500, RS RS wrote: > > I am running the following test. I have 10 clients and 5 OSTs. I have >set > > up striping so that pairs of clients are writing to an OST. That is, > > clients 0 + 4 write to ost-zero, clients 1 + 5 write to ost-one, etc. > > > > Each cilent is creating a large file. > > > > What I see is that one client creates its file much faster than the >other > > client that goes to the same OST. So, for example, in the following > > directory listing, the files zero/n100 and zero/n122 are both going to > > OST-zero. Each is created from a client, and the two clients are > > identical. The file zero/n100 is about 3.3G, while the file zero/n122 is > > about 30G. Similarly for files two/n102 and two/n150, which are 4.5G >and > > 28.9G respectively (they got to OST-two). > > > > Is there any way to make this fairer? > >What do you mean by "fair"? > >Cheers, Andreas >--"Perfectly fair: would me that if an OST can do X MB/s of I/O, and if there are Y clients issuing I/O requests to that OST in any given second, then each client sees ( X / Y ) MB/s of IO during that second. Of course it is not reasonable to be "perfectly fair." So, it would be fine with me if each client got at least (X / (Y*2)) MB/s in any given second. -Roger _________________________________________________________________ Don’t miss your chance to WIN 10 hours of private jet travel from Microsoft® Office Live http://clk.atdmt.com/MRT/go/mcrssaub0540002499mrt/direct/01/
In my mind, this goes hand-in-hand with QOS. I haven''t had a chance to review Eric''s draft proposal that Peter posted on Mar 1. I''m still not certain that QOS at the network layer will handle all of the issues related to QOS. I''m wondering if instead it should be handled at the RPC layer. --Shane -----Original Message----- From: lustre-devel-bounces@clusterfs.com [mailto:lustre-devel-bounces@clusterfs.com] On Behalf Of RS RS Sent: Monday, March 05, 2007 3:57 PM To: adilger@clusterfs.com Cc: lustre-devel@clusterfs.com Subject: Re: [Lustre-devel] Fairness to clients>On Mar 02, 2007 13:40 -0500, RS RS wrote: > > I am running the following test. I have 10 clients and 5 OSTs. Ihave>set > > up striping so that pairs of clients are writing to an OST. Thatis,> > clients 0 + 4 write to ost-zero, clients 1 + 5 write to ost-one,etc.> > > > Each cilent is creating a large file. > > > > What I see is that one client creates its file much faster than the >other > > client that goes to the same OST. So, for example, in the following > > directory listing, the files zero/n100 and zero/n122 are both goingto> > OST-zero. Each is created from a client, and the two clients are > > identical. The file zero/n100 is about 3.3G, while the filezero/n122 is> > about 30G. Similarly for files two/n102 and two/n150, which are4.5G>and > > 28.9G respectively (they got to OST-two). > > > > Is there any way to make this fairer? > >What do you mean by "fair"? > >Cheers, Andreas >--"Perfectly fair: would me that if an OST can do X MB/s of I/O, and if there are Y clients issuing I/O requests to that OST in any given second, then each client sees ( X / Y ) MB/s of IO during that second. Of course it is not reasonable to be "perfectly fair." So, it would be fine with me if each client got at least (X / (Y*2)) MB/s in any given second. -Roger _________________________________________________________________ Don''t miss your chance to WIN 10 hours of private jet travel from Microsoft(r) Office Live http://clk.atdmt.com/MRT/go/mcrssaub0540002499mrt/direct/01/ _______________________________________________ Lustre-devel mailing list Lustre-devel@clusterfs.com https://mail.clusterfs.com/mailman/listinfo/lustre-devel