freebsd stable - Jan 2017 - HASt and write amplification on the seconday

I have a pair of machines which I have been runnig HAST on
for a number of years. It works well, it does what Ineed it
to do, and I havent considered the details until recently.

As I udnesratnd it though, the minimum size of block copied is
set by default to something quite arge (2 meg). I see whay, but
as I am using ZFS on top I am generating lots of writes which are
scattered across the disc (ZFS not known for keeping writes all in
the same place!).

So, is each 512 byte write being turned into a 2 meg write on the
secondary ? If so this worries me, as I am about to replcae these
drives with SSD's, which I do not want to wearn out with excessive
writes.

Am I worrying over nothing ? Should I try and tune the HAST sizes and the
zpool ashift to match, or will that trash performances 9some limited
experiments suggest it will).

-pete.

On Mon, Jan 30, 2017 at 06:41:21PM +0000, Pete French
wrote:
> I have a pair of machines which I have been runnig HAST on
> for a number of years. It works well, it does what Ineed it
> to do, and I havent considered the details until recently.
> 
> As I udnesratnd it though, the minimum size of block copied is
> set by default to something quite arge (2 meg). I see whay, but
> as I am using ZFS on top I am generating lots of writes which are
> scattered across the disc (ZFS not known for keeping writes all in
> the same place!).
> 
> So, is each 512 byte write being turned into a 2 meg write on the
> secondary ? If so this worries me, as I am about to replcae these
> drives with SSD's, which I do not want to wearn out with excessive
> writes.
Extents (2 meg blocks) are used for synchronization after the
secondary is reconnected. Normally (the secondary is connected) writes
to the primary and the secondary are the same.

Taking your example with 512 byte write:

1) The secondary is connected -- a 512 byte write to HAST device
results in 512 byte write to the primary and 512 byte write to the
secondary.

2) The secondary is disconnected -- a 512 byte write to HAST device
results in 512 byte write to the primary and marking 2Mb extent as
dirty; when the secondary is reconnected, the whole extent (2Mb) is
sent.

-- 
Mykola Golub