I have an application that iterates through snapshots sending them to a remote host. With a Solaris 10 receiver, empty snapshots are received in under a second, but with a Solaris 11 Express receiver, empty snapshots are received in 2 to three seconds. This is becoming a real nuisance where I have a large number of snapshots in a filesystem that''s unchanged. For example: receiving incremental stream of export/vbox at 20110927_1805 into backup/vbox at 20110927_1805 received 312B stream in 3 seconds (104B/sec) receiving incremental stream of export/vbox at 20110927_2205 into backup/vbox at 20110927_2205 received 312B stream in 2 seconds (156B/sec) The change looks to be increased latency, bigger snapshots still appear to be received at the same speed as before. Does anyone know what has changed to cause this slowdown? -- Ian.
On 09/30/11 05:14 AM, erik wrote:> > On Thu, 29 Sep 2011 21:13:56 +1300, Ian Collins wrote: > >> I have an application that iterates through snapshots sending them to >> a remote host. With a Solaris 10 receiver, empty snapshots are received >> in under a second, but with a Solaris 11 Express receiver, empty >> snapshots are received in 2 to three seconds. This is becoming a real >> nuisance where I have a large number of snapshots in a filesystem that''s >> unchanged. >> >> For example: >> >> receiving incremental stream of export/vbox at 20110927_1805 into >> backup/vbox at 20110927_1805 >> received 312B stream in 3 seconds (104B/sec) >> receiving incremental stream of export/vbox at 20110927_2205 into >> backup/vbox at 20110927_2205 >> received 312B stream in 2 seconds (156B/sec) >> >> The change looks to be increased latency, bigger snapshots still appear >> to be received at the same speed as before. >> >> Does anyone know what has changed to cause this slowdown? > > I think that''s pretty much the baseline overhead required for > validating the consistency of the snapshot and it''s applicability on > the destination pool. I have similar numbers on a little NAS dumping > to a set of external USB disks that behave in a similar manner: >That does appear to be the case, but I was wondering why it has become so much worse? I am in the process of copying some large data sets to a new server and the whole process it taking way longer than I expected (there are thousands of small snapshots). Slowing down replication is not a good move! -- Ian.
On 09/30/11 08:03 AM, Bob Friesenhahn wrote:> On Fri, 30 Sep 2011, Ian Collins wrote: >> Slowing down replication is not a good move! > Do you prefer pool corruption? ;-) > > Probably they fixed a dire bug and this is the cost of the fix. >Could be. I think I''ll raise a support case to find out why. This is making it difficult for me to meet a replication guarantee. -- Ian.
On 09/30/11 08:12 AM, Ian Collins wrote:> On 09/30/11 08:03 AM, Bob Friesenhahn wrote: >> On Fri, 30 Sep 2011, Ian Collins wrote: >>> Slowing down replication is not a good move! >> Do you prefer pool corruption? ;-) >> >> Probably they fixed a dire bug and this is the cost of the fix. >> > Could be. I think I''ll raise a support case to find out why. This is > making it difficult for me to meet a replication guarantee. >It turns out this was a problem with e1000g interfaces. When we swapped over to an igb port, the problem went away. -- Ian.
On 11/14/11 04:00 AM, Jeff Savit wrote:> On 11/12/2011 03:04 PM, Ian Collins wrote: >> >> It turns out this was a problem with e1000g interfaces. When we >> swapped over to an igb port, the problem went away. >> > Ian, could you summarize what the e1000g problem was? It might be > interesting or useful for the list. If you don''t want to do that, but > are willing to tell me off-list that would be appreciated. (Just out > of curiosity). >I was seeing high latency (2-4 seconds each) when sending large number of small snapshots, say a series of incremental snapshots for a filesystem that hadn''t changed. -- Ian.