John R Pierce wrote:> On 9/9/2017 9:47 AM, hw wrote: >> >> Isn?t it easier for SSDs to write small chunks of data at a time? >> The small chunk might fit into some free space more easily than >> a large one which needs to be spread out all over the place. > > > the SSD collects data blocks being written and when a full flash block worth of data is collected, often 256K to several MB, it writes them all at once to a single contiguous block on the flash array, no matter what the 'address' of the blocks being written is. think of it as a 'scatter-gather' operation. > > different drive brands and models use different strategies for this, and all this is completely opaque to the host OS so you really can't outguess or manage this process at the OS or disk controller level. > >What if the collector is full? I understand that using small chunk sizes can reduce performance because many chunks need to be dealt with. Using large chunks would involve reading and writing larger amounts of data every time, and that also could reduce performance. With a chunk size of 1MB, disk access might amount to huge amounts of data being read and written unnecessarily. So what might be a good chunk size for SSDs?
Stephen John Smoogen
2017-Sep-13 15:33 UTC
[CentOS] stripe size for SSDs? ( cyrus spool on btrfs?)
On 13 September 2017 at 09:25, hw <hw at gc-24.de> wrote:> John R Pierce wrote: >> >> On 9/9/2017 9:47 AM, hw wrote: >>> >>> >>> Isn?t it easier for SSDs to write small chunks of data at a time? >>> The small chunk might fit into some free space more easily than >>> a large one which needs to be spread out all over the place. >> >> >> >> the SSD collects data blocks being written and when a full flash block >> worth of data is collected, often 256K to several MB, it writes them all at >> once to a single contiguous block on the flash array, no matter what the >> 'address' of the blocks being written is. think of it as a 'scatter-gather' >> operation. >> >> different drive brands and models use different strategies for this, and >> all this is completely opaque to the host OS so you really can't outguess or >> manage this process at the OS or disk controller level. >> >> > > What if the collector is full? > > I understand that using small chunk sizes can reduce performance because > many chunks need to be dealt with. Using large chunks would involve > reading and writing larger amounts of data every time, and that also > could reduce performance. > > With a chunk size of 1MB, disk access might amount to huge amounts of > data being read and written unnecessarily. So what might be a good chunk > size for SSDs?It will depend on the type of SSD. Ones with large cache and various smarts (SAS Enterprise type) can take many different sizes. For SATA ones it depends on what the cache and write of the SSD is and very few of them seem to be the same. The SSD also has all kinds of logic which moves data around constantly on disk to wipe level so it makes it opaque. The people who have tested this usually have to burn through an SSD set to get an idea about a particular 'run' of a model but it doesn't go over every version of the model of SATA SSD.> _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos-- Stephen J Smoogen.
Stephen John Smoogen wrote:> On 13 September 2017 at 09:25, hw <hw at gc-24.de> wrote: >> John R Pierce wrote: >>> >>> On 9/9/2017 9:47 AM, hw wrote: >>>> >>>> >>>> Isn?t it easier for SSDs to write small chunks of data at a time? >>>> The small chunk might fit into some free space more easily than >>>> a large one which needs to be spread out all over the place. >>> >>> >>> >>> the SSD collects data blocks being written and when a full flash block >>> worth of data is collected, often 256K to several MB, it writes them all at >>> once to a single contiguous block on the flash array, no matter what the >>> 'address' of the blocks being written is. think of it as a 'scatter-gather' >>> operation. >>> >>> different drive brands and models use different strategies for this, and >>> all this is completely opaque to the host OS so you really can't outguess or >>> manage this process at the OS or disk controller level. >>> >>> >> >> What if the collector is full? >> >> I understand that using small chunk sizes can reduce performance because >> many chunks need to be dealt with. Using large chunks would involve >> reading and writing larger amounts of data every time, and that also >> could reduce performance. >> >> With a chunk size of 1MB, disk access might amount to huge amounts of >> data being read and written unnecessarily. So what might be a good chunk >> size for SSDs? > > It will depend on the type of SSD. Ones with large cache and various > smarts (SAS Enterprise type) can take many different sizes. For SATA > ones it depends on what the cache and write of the SSD is and very few > of them seem to be the same. The SSD also has all kinds of logic which > moves data around constantly on disk to wipe level so it makes it > opaque. The people who have tested this usually have to burn through > an SSD set to get an idea about a particular 'run' of a model but it > doesn't go over every version of the model of SATA SSD.Hm, so much to SSDs ... I can only hope they will be replaced with something better. I have decided against putting anything onto these SSDs other than temporary data, but even for that, I would need to make an md-RAID, which I don?t want. It may work or not, and "may work" is not enough. If the performance on the hardware RAID isn?t as good, it can not get worse than it is now, and it may be even better than with the SSDs. I have two at home with the system installed on btrfs. I?m going to change that to md-RAID1 and xfs. Is there anything special involved in copying the system to another disk? Will 'cp -ax' do, or should I use rsync to copy xattrs etc.? Using the commonly used stripe size of 128kb is something I?d expect the SSDs being able to handle.