Given storage provisioned off a SAN (I know, but sometimes that''s what you have to work with), what''s the best way to expand a pool? Specifically, I can either grow existing LUNs, a]or add new LUNs. As an example, If I have 24x 2TB LUNs, and wish to double the size of the pool, is it better to add 24 additional 2TB LUNs, or get each of the existing LUNs expanded to 4TB each? -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
On 2012-11-13 17:42, Peter Tribble wrote:> Given storageprovisioned off a SAN (I know, but sometimes that''s> what you have towork with), what''s the best way to expand a pool?> > Specifically, Ican either grow existing LUNs, a]or add new LUNs.> > As an example, IfI have 24x 2TB LUNs, and wish to double the> size of the pool, is itbetter to add 24 additional 2TB LUNs, or> get each of the existing LUNsexpanded to 4TB each? This is only my opinion, but I would say you''d be better off expanding your current LUNs. The reason for this is balance. Currently, your data should be spread fairly evenly over the LUNs. If you add more, those will be empty, which will affect how data is written (data will try to go to those first). If you just expand your current LUNs, the data will remain balanced, and ZFS will just use the additional space. I _think _that''s how it would work. Others here will be able to give a more definitive answer. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20121113/ce35cbbe/attachment.html>
Not sure if this will make it to the list, but I''ll try... On 11/13/12, Peter Tribble wrote:> Given storage provisioned off a SAN (I know, but sometimes that''s > what you have to work with), what''s the best way to expand a pool? > > Specifically, I can either grow existing LUNs, a]or add new LUNs. > > As an example, If I have 24x 2TB LUNs, and wish to double the > size of the pool, is it better to add 24 additional 2TB LUNs, or > get each of the existing LUNs expanded to 4TB each?The thing I''ve found about growing the LUN is to evaluate if it saves you time and effort in your setup to make it work. So, first I figure out if it''s possible. Then I figure out how big a PITA it is compared to the PITA of allocating new LUNs. For example I''ve got one SAN backend here where it''s easy as pie (lun resize blah and then OS steps) and another backend where it requires a major undertaking (create temporary horcm files, destroy mirrors, run special command to resize mirror, wait, run special command to resize source, wait, recreate mirrors, delete temporary horcms, etc etc and *then* the OS steps). What I love about ZFS is that it can handle either approach. So it depends on your setup. In your case if it''s at all painful to grow the LUNs, what I''d probably do is allocate new 4TB LUNs - and replace your 2TB LUNs with them one at a time with zpool replace, and wait for the resliver to finish each time. With autoexpansion on, you should get the additional capacity as soon as the resliver for each one is done, and the old 2TB LUNs should be reclaimable as soon as it''s reslivered out. That being said, I''m not aware of what any deeper implications of doing that are - from what Karl said about balancing the data out as one example. Cheers, Brian> > -- > -Peter Tribble > http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-- --? ----------------------------------------------------------------------------------- Brian Wilson, Solaris SE, UW-Madison DoIT Room 3114 CS&S 608-263-8047 brian.wilson(a)doit.wisc.edu ''I try to save a life a day. Usually it''s my own.'' - John Crichton -----------------------------------------------------------------------------------
On Wed, Nov 14, 2012 at 1:35 AM, Brian Wilson <brian.wilson at doit.wisc.edu> wrote:> So it depends on your setup. In your case if it''s at all painful to grow the LUNs, what I''d probably do is allocate new 4TB LUNs - and replace your 2TB LUNs with them one at a time with zpool replace, and wait for the resliver to finish each time. With autoexpansion on,Yup, that''s the gotcha. AFAIK autoexpand is off by default. You should be able to use "zpool online -e" to force the expansion though.> you should get the additional capacity as soon as the resliver for each one is done, and the old 2TB LUNs should be reclaimable as soon as it''s reslivered out.Minor correction: the additional capacity is only usable after a top level vdev is completely replaced. In case of stripe-of-mirrors, this is as soon as all vdev in one mirror is replaced. In the case of raidzX, this is when all vdev is replaced. -- Fajar
On Tue, Nov 13, 2012 at 6:16 PM, Karl Wagner <karl at mouse-hole.com> wrote:> On 2012-11-13 17:42, Peter Tribble wrote: > > > Given storage provisioned off a SAN (I know, but sometimes that''s > > what you have to work with), what''s the best way to expand a pool? > > > > Specifically, I can either grow existing LUNs, a]or add new LUNs. > > > > As an example, If I have 24x 2TB LUNs, and wish to double the > > size of the pool, is it better to add 24 additional 2TB LUNs, or > > get each of the existing LUNs expanded to 4TB each? > > This is only my opinion, but I would say you''d be better off expanding your > current LUNs. > > The reason for this is balance. Currently, your data should be spread fairly > evenly over the LUNs. If you add more, those will be empty, which will > affect how data is written (data will try to go to those first). > > If you just expand your current LUNs, the data will remain balanced, and ZFS > will just use the additional space.Maybe, or maybe not. If you think in terms of metaslabs, then there isn''t any difference between creating extra metaslabs by growing an existing LUN and creating new LUNs. With pooled storage on the SAN back-end, there''s no difference in I/O placement either. Peripherally, this note by Adam Leventhal may be of interest http://dtrace.org/blogs/ahl/2012/11/08/zfs-trivia-metaslabs/ -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/