>> Hello >> I''m offering a solution based on our disks where replication and >> storage management should be made using only ZFS... >> The test change few bytes on one file ( 10bytes ) and check how many >> bytes the source sends to target. >> The customer tried the replication between 2 volume...They compared >> ZFS replica with true copy replica and they realized the following >> considerations: >> >> 1. ZFS uses a block bigger than HDS true copy >> 2. true copy sends 32Kbytes and ZFS 100K and more changing only 10 >> file bytes >> >> Can we configure ZFS to improve replication efficiencies ? >> >> The solution should consider 5 remote site replicating on one central >> data-center. Considering the zfs block overhead the customer is >> thinking to buy a solution based on traditional storage arrays like >> HDS entry level arrays ( our 2530/2540 ). If so ..with the ZFS the >> network traffic, storage space become big problems for the customer >> infrastructures. >> >> Are there any documentation explaining internal ZFS replication >> mechanism to face the customer doubts ? ThanksDo we need of AVS in our solution to solve the problem ?>> Thanks
Enrico,> >>> Hello >>> I''m offering a solution based on our disks where replication and >>> storage management should be made using only ZFS... >>> The test change few bytes on one file ( 10bytes ) and check how many >>> bytes the source sends to target. >>> The customer tried the replication between 2 volume...They compared >>> ZFS replica with true copy replica and they realized the following >>> considerations: >>> >>> 1. ZFS uses a block bigger than HDS true copy >>> 2. true copy sends 32Kbytes and ZFS 100K and more changing only 10 >>> file bytes >>> >>> Can we configure ZFS to improve replication efficiencies ? >>> >>> The solution should consider 5 remote site replicating on one >>> central >>> data-center. Considering the zfs block overhead the customer is >>> thinking to buy a solution based on traditional storage arrays like >>> HDS entry level arrays ( our 2530/2540 ). If so ..with the ZFS the >>> network traffic, storage space become big problems for the customer >>> infrastructures. >>> >>> Are there any documentation explaining internal ZFS replication >>> mechanism to face the customer doubts ? Thanks > Do we need of AVS in our solution to solve the problem ?AVS, not unlike HDS, does block-based replication based on actual write I/Os to configured devices. Therefore if the means for ZFS to change 10 bytes, results in ZFS writing 100KB or more, AVS will be essentially be no different than HDS in this specific area. Of course this begs to question, is the measure of a 10 byte change to a given file, a viable metric for choosing one form of replication over another? I think, or would hope not. What is need is a characterization of the application(s) write-rate to one or more ZFS filesystems, over the customers requirements for data replication. A good place to start is: http://www.sun.com/storagetek/white-papers/data_replication_strategies.pdf http://www.sun.com/storagetek/white-papers/enterprise_continuity.pdf> >>> Thanks > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discussJim Dunham Storage Platform Software Group Sun Microsystems, Inc. wk: 781.442.4042 http://blogs.sun.com/avs http://www.opensolaris.org/os/project/avs/ http://www.opensolaris.org/os/project/iscsitgt/ http://www.opensolaris.org/os/community/storage/
Enrico Rampazzo wrote:>>> Hello >>> I''m offering a solution based on our disks where replication and >>> storage management should be made using only ZFS... >>> The test change few bytes on one file ( 10bytes ) and check how many >>> bytes the source sends to target. >>> The customer tried the replication between 2 volume...They compared >>> ZFS replica with true copy replica and they realized the following >>> considerations: >>> >>> 1. ZFS uses a block bigger than HDS true copy >>>ZFS uses dynamic block sizes. Depending on the configuration and workload, just a few disk blocks will change, or a bunch of redundant metadata might change. In either case, changing the ZFS recordsize will make little, if any, change.>>> 2. true copy sends 32Kbytes and ZFS 100K and more changing only 10 >>> file bytes >>> >>> Can we configure ZFS to improve replication efficiencies ? >>>By default, ZFS writes two copies of metadata. I would not recommend reducing this because it will increase your exposure to faults. What may be happening here is that a 10 byte write may cause a metadata change resulting in a minimum of three 512 byte physical blocks being changed. The metadata copies are on spatially diverse, so you may see these three blocks starting at non-contiguous boundaries. If Truecopy sends only 32kByte blocks (speculation), then the remote transfer will be 96kBytes for 3 local, physical block writes. OTOH, ZFS will coalesce writes. So you may be able to update a number of files yet still only replicate 96kBytes through Truecopy. YMMV. Since the customer is performing replication, I''ll assume they are very interested in data protection, so keeping the redundant metadata is a good idea. The customer should also be aware that replication at the application level is *always* more efficient than replicating somewhere down the software stack where you lose data context. -- richard>>> The solution should consider 5 remote site replicating on one central >>> data-center. Considering the zfs block overhead the customer is >>> thinking to buy a solution based on traditional storage arrays like >>> HDS entry level arrays ( our 2530/2540 ). If so ..with the ZFS the >>> network traffic, storage space become big problems for the customer >>> infrastructures. >>> >>> Are there any documentation explaining internal ZFS replication >>> mechanism to face the customer doubts ? Thanks >>> > Do we need of AVS in our solution to solve the problem ? > >>> Thanks >>> > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Is there any forecast to improve the efficiency of the replication mechanisms of ZFS ? Fishwork - new NAS release .... Considering the solution we are offering to our customer ( 5 remote sites replicating in one central data-center ) with ZFS ( cheapest solution ) I should consider 3 times the network load of a solution based on SNDR-AVS and 3 times the storage space too..correct ? I there any documentation on that ? Thanks Richard Elling ha scritto:> Enrico Rampazzo wrote: >>>> Hello >>>> I''m offering a solution based on our disks where replication and >>>> storage management should be made using only ZFS... >>>> The test change few bytes on one file ( 10bytes ) and check how >>>> many bytes the source sends to target. >>>> The customer tried the replication between 2 volume...They compared >>>> ZFS replica with true copy replica and they realized the following >>>> considerations: >>>> >>>> 1. ZFS uses a block bigger than HDS true copy >>>> > > ZFS uses dynamic block sizes. Depending on the configuration and > workload, just a few disk blocks will change, or a bunch of redundant > metadata might change. In either case, changing the ZFS recordsize > will make little, if any, change. > >>>> 2. true copy sends 32Kbytes and ZFS 100K and more changing only 10 >>>> file bytes >>>> >>>> Can we configure ZFS to improve replication efficiencies ? >>>> > > By default, ZFS writes two copies of metadata. I would not recommend > reducing this because it will increase your exposure to faults. What may > be happening here is that a 10 byte write may cause a metadata change > resulting in a minimum of three 512 byte physical blocks being > changed. The metadata copies are on spatially diverse, so you may see > these three > blocks starting at non-contiguous boundaries. If Truecopy sends only > 32kByte blocks (speculation), then the remote transfer will be 96kBytes > for 3 local, physical block writes. > > OTOH, ZFS will coalesce writes. So you may be able to update a > number of files yet still only replicate 96kBytes through Truecopy. > YMMV. > > Since the customer is performing replication, I''ll assume they are very > interested in data protection, so keeping the redundant metadata is a > good idea. The customer should also be aware that replication at the > application level is *always* more efficient than replicating somewhere > down the software stack where you lose data context. > -- richard > >>>> The solution should consider 5 remote site replicating on one >>>> central data-center. Considering the zfs block overhead the >>>> customer is thinking to buy a solution based on traditional storage >>>> arrays like HDS entry level arrays ( our 2530/2540 ). If so ..with >>>> the ZFS the network traffic, storage space become big problems for >>>> the customer infrastructures. >>>> >>>> Are there any documentation explaining internal ZFS replication >>>> mechanism to face the customer doubts ? Thanks >>>> >> Do we need of AVS in our solution to solve the problem ? >> >>>> Thanks >>>> >> >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> >
Enrico,> Is there any forecast to improve the efficiency of the replication > mechanisms of ZFS ? Fishwork - new NAS release ....I would take some time to talk with and understand exactly what the customer''s expectation are for replication. i would not base my decision on the cost of replicating 10 bytes, regardless of how inefficient it may be. These two documents should help: http://www.sun.com/storagetek/white-papers/data_replication_strategies.pdf http://www.sun.com/storagetek/white-papers/enterprise_continuity.pdf Two key metrics of replication are: Recovery Point Objective (RPO), is the amount of data lost (or less), measured as a unit time. Once a day backups yield a 24 hour RPO, once an hour snapshots yields ~1 hour RPO, asynchronous replication yields zero seconds to a few minutes RPO, and synchronous replication means zero seconds RPO. Recovery Time Objective (RTO), is the amount of time after a failure, until normal operations are restored. Tapes backups could be minutes to hours, local snapshots could be nearly instantaneous, assuming the local site survived the failure. Remote snapshots or replicas could be minutes, hours or days, depending on the amount of data to resynchronize, impacted by network bandwidth and latency. Availability Suite has a unique feature in this last area, called on- demand pull. Assuming that the primary site''s volumes are lost, after they have been re-provisioned, a reverse update can be initiated. Besides the background resilvering in the reverse direction being active, eventually restoring all lost data, on-demand pull performs synchronous replication of data blocks on demand, as needed by the filesystem, database or application. Although the performance will be less then synchronous replication, the RTO is quite low. This type of recovery is analogous to loosing one''s entire email account, having recovery initiated, but also selected email can be open as needed before the entire volume is restored, using on demand requests to satisfy data blocks for relevant email requests. Jim> > > Considering the solution we are offering to our customer ( 5 remote > sites replicating in one central data-center ) with ZFS ( cheapest > solution ) I should consider > 3 times the network load of a solution based on SNDR-AVS and 3 times > the > storage space too..correct ? > > I there any documentation on that ? > Thanks > > Richard Elling ha scritto: >> Enrico Rampazzo wrote: >>>>> Hello >>>>> I''m offering a solution based on our disks where replication and >>>>> storage management should be made using only ZFS... >>>>> The test change few bytes on one file ( 10bytes ) and check how >>>>> many bytes the source sends to target. >>>>> The customer tried the replication between 2 volume...They >>>>> compared >>>>> ZFS replica with true copy replica and they realized the following >>>>> considerations: >>>>> >>>>> 1. ZFS uses a block bigger than HDS true copy >>>>> >> >> ZFS uses dynamic block sizes. Depending on the configuration and >> workload, just a few disk blocks will change, or a bunch of redundant >> metadata might change. In either case, changing the ZFS recordsize >> will make little, if any, change. >> >>>>> 2. true copy sends 32Kbytes and ZFS 100K and more changing only >>>>> 10 >>>>> file bytes >>>>> >>>>> Can we configure ZFS to improve replication efficiencies ? >>>>> >> >> By default, ZFS writes two copies of metadata. I would not recommend >> reducing this because it will increase your exposure to faults. >> What may >> be happening here is that a 10 byte write may cause a metadata change >> resulting in a minimum of three 512 byte physical blocks being >> changed. The metadata copies are on spatially diverse, so you may see >> these three >> blocks starting at non-contiguous boundaries. If Truecopy sends only >> 32kByte blocks (speculation), then the remote transfer will be >> 96kBytes >> for 3 local, physical block writes. >> >> OTOH, ZFS will coalesce writes. So you may be able to update a >> number of files yet still only replicate 96kBytes through Truecopy. >> YMMV. >> >> Since the customer is performing replication, I''ll assume they are >> very >> interested in data protection, so keeping the redundant metadata is a >> good idea. The customer should also be aware that replication at the >> application level is *always* more efficient than replicating >> somewhere >> down the software stack where you lose data context. >> -- richard >> >>>>> The solution should consider 5 remote site replicating on one >>>>> central data-center. Considering the zfs block overhead the >>>>> customer is thinking to buy a solution based on traditional >>>>> storage >>>>> arrays like HDS entry level arrays ( our 2530/2540 ). If so ..with >>>>> the ZFS the network traffic, storage space become big problems for >>>>> the customer infrastructures. >>>>> >>>>> Are there any documentation explaining internal ZFS replication >>>>> mechanism to face the customer doubts ? Thanks >>>>> >>> Do we need of AVS in our solution to solve the problem ? >>> >>>>> Thanks >>>>> >>> >>> _______________________________________________ >>> zfs-discuss mailing list >>> zfs-discuss at opensolaris.org >>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >>> >> > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080211/17015437/attachment.html>