Hello, I haven''t seen this discussed before. Any pointers would be appreciated. I''m curious, if I have a set of disks in a system, is there any benefit or disadvantage to breaking the disks into multiple pools instead of a single pool? Does multiple pools cause any additional overhead for ZFS, for example? Can it cause cache contention/starvation issues? Thanks... --joe
Am .. first of all its depends on how many hard disks you have and also on how much is the importance and the necessity of these data . for my system I used 2 hard disk ( 160 GB each ) , first one has three pools one for the system it self and one for my documents, books and all my work files . and the last one is for my songs and movies, if you see even if one these pools get corrupted or even the system is crashed in way or another I can retrieve the rest unless you have a mirror disk which is different case , you can get your stuff back .. the next hard disk contain only two main pools which is only for my virtual machines images files , I kept all the machines which I used frequently in the first pool for speed access , and even more for the first one I always keep the media pool ( songs and movies ) off line and when I need it I attach it .. I also found this which is talk about pool performance considerations http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#Storage_Pool_Performance_Considerations .... I hope this helps you a bit . -- This message posted from opensolaris.org
Joseph Mocker schrieb:> Hello, > > I haven''t seen this discussed before. Any pointers would be appreciated. > > I''m curious, if I have a set of disks in a system, is there any benefit > or disadvantage to breaking the disks into multiple pools instead of a > single pool? > > Does multiple pools cause any additional overhead for ZFS, for example? > Can it cause cache contention/starvation issues? > > Thanks... > > --joe > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discussCurrently, I''ve two pools in my system: one for live data and the other for backup. When doing large backups (i.e. tar''ing one directory hierarchy from live to backup), I''ve seen severe memory pressure on the system - as if both pools were competing for memory... Maybe with zfs boot/root becoming available, I''ll add a third pool for the OS. From what I''ve seen, zfs makes very much sense for boot/root if you are using live upgrade. I like the idea of having OS and data separated, but on a system with only two disks, I''d definitely go for a single mirrored zpool where both OS and data reside. I guess sharing one physical disk among multiple zpools could have severe negative impacts during concurrent accesses. But I really have no in-depth knowledge to say for sure. Maybe somebody else can comment on this... - Thomas
On Wed, Oct 8, 2008 at 9:29 PM, Joseph Mocker <mock+osol at fakebelieve.org<mock%2Bosol at fakebelieve.org>> wrote:> Hello, > > I haven''t seen this discussed before. Any pointers would be appreciated. > > I''m curious, if I have a set of disks in a system, is there any benefit > or disadvantage to breaking the disks into multiple pools instead of a > single pool? > > Does multiple pools cause any additional overhead for ZFS, for example? > Can it cause cache contention/starvation issues? > >Hello Joseph. Firstly, a separate pool for the OS is recommended. The pool from which you boot must be either Mirrored or else a single disk. Booting from Stripes / RaidZ is not supported. Thus if you want to use a stripe or RaidZ you pretty much MUST have a dedicated pool for that. Secondly, if you use whole disks in your pools, it becomes possible to physically "remove" a pool (using zpool export), eg to move a pool to another system. Further, it is recommended to use the same level of redundancy in all vdev''s. Eg all vdevs should be mirrored, or the same nr of columns in the stripe or raidz. This is not a restriction, just a strong recommendation. Never ever add multiple slices (partitions) from a single disk device to the same pool - this will cause performance to go down to a crawl! You can not (yet) "break up" a pool, though you can break off a mirror copy. And to stay in line with the above recommendations, you may want more than one pool. For best performance you should use whole-disks in pools, but sometimes for practical reasons you may want to spit a single disk up in slices and add those to separate pools. Hope that helps! _hartz -- Any sufficiently advanced technology is indistinguishable from magic. Arthur C. Clarke Afrikaanse Stap Website: http://www.bloukous.co.za My blog: http://initialprogramload.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081009/7ccb8649/attachment.html>
Thanks for the information... In my case, I do not have a root pool, its still UFS. The configuration is essentially that I have two arrays. The system was initially built with one array. A zfs pool was created from the whole disks on the array. The pool is more or less used for general storage. Along comes a second array, with different number of and size disks. It will be used for more or less the same thing. So the question is whether I should add to the original pool or create a second pool. One thing you mention is rather curious, "all vdevs should be mirrored, or the same nr of columns in the stripe or raidz. This is not a restriction, just a strong recommendation". If I add the second array to the pool, I could probably continue with the same number of columns in the raidz, but the size of the strips would increase. Would this effect performance somehow? --joe Johan Hartzenberg wrote:> > On Wed, Oct 8, 2008 at 9:29 PM, Joseph Mocker > <mock+osol at fakebelieve.org <mailto:mock%2Bosol at fakebelieve.org>> wrote: > > Hello, > > I haven''t seen this discussed before. Any pointers would be > appreciated. > > I''m curious, if I have a set of disks in a system, is there any > benefit > or disadvantage to breaking the disks into multiple pools instead of a > single pool? > > Does multiple pools cause any additional overhead for ZFS, for > example? > Can it cause cache contention/starvation issues? > > > Hello Joseph. > > Firstly, a separate pool for the OS is recommended. The pool from > which you boot must be either Mirrored or else a single disk. Booting > from Stripes / RaidZ is not supported. Thus if you want to use a > stripe or RaidZ you pretty much MUST have a dedicated pool for that. > > Secondly, if you use whole disks in your pools, it becomes possible to > physically "remove" a pool (using zpool export), eg to move a pool to > another system. > > Further, it is recommended to use the same level of redundancy in all > vdev''s. Eg all vdevs should be mirrored, or the same nr of columns in > the stripe or raidz. This is not a restriction, just a strong > recommendation. > > Never ever add multiple slices (partitions) from a single disk device > to the same pool - this will cause performance to go down to a crawl! > > You can not (yet) "break up" a pool, though you can break off a mirror > copy. And to stay in line with the above recommendations, you may > want more than one pool. For best performance you should use > whole-disks in pools, but sometimes for practical reasons you may want > to spit a single disk up in slices and add those to separate pools. > > Hope that helps! > _hartz > > -- > Any sufficiently advanced technology is indistinguishable from magic. > Arthur C. Clarke > > Afrikaanse Stap Website: http://www.bloukous.co.za > > My blog: http://initialprogramload.blogspot.com >
On Thu, Oct 9, 2008 at 10:28 PM, Joseph Mocker <mock+osol at fakebelieve.org<mock%2Bosol at fakebelieve.org>> wrote:> > If I add the second array to the pool, I could probably continue with the > same number of columns in the raidz, but the size of the strips would > increase. Would this effect performance somehow? >I hate the word performance because it doesn''t have a meaning. If you spread the load over more disks, IO will become less of a bottleneck, regardless of how you configure it. Whether this means anything at all depends on whether or not IO is currently your bottleneck or not. In addition, larger capacity disks store more bits per cylinder, thus more data pass the read/write head per revolution, thus the disks "perform" differently, regardless of other factors such as raid levels and stripe widths. Other factors include: Bus speed to the disks, type of work-load (small random reads, larges sequential writes, etc), whether you have enough free CPU and Memory to drive the disks to their full capacity, etc. Not knowing any of these things about your "general storage" data I would hazard to say it will perform better than it is currently performing, regardless of whether you add the new disks to the same or a new zpool, and if the existing pool, even if you use a different column size in the new vdev, it will still be true most likely. My suggestion: Add the disks in a way that makes sense to you holistically. Think about what you want to achieve - consider everything, like the required redundancy, performance, and required capacity. Go by your gut-feeling. Exception to the rule: If you have a serious performance problem and you know the system is currently disk (IO) bound. In this case, test it properly: Have base-line benchmarks and know what your testing objectives are before you start. Document your scenarios and run load tests on each scenario, monitoring all resources so you know which is the bottleneck in each case. For most people the correct answer is simply add the disks to the existing pool, using "sensible" raidz column sizes. If this breaks the rule of "keep the raidz column sizes the same", then so be it. My understanding for the reason behind this rule has more to do with ensuring that you (and your boss/customers) understand the amount of protection your data has rather than with performance, but I may be wrong. -- Any sufficiently advanced technology is indistinguishable from magic. Arthur C. Clarke My blog: http://initialprogramload.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081011/57eb9723/attachment.html>