CentOS - Jun 2015 - LVM hatred, was Re: /boot on a separate partition?

On Tue, 23 Jun 2015 11:15:30 -0500
Jason Warr <jason at warr.net> wrote:
> 
> I'm curious what has made some people hate LVM so much.  I have been 
> using it for years on thousands of production systems with no issues 
> that could not be easily explained as myself or someone else doing 
> something stupid.  And even those issues were pretty few and far
> between.
> 
> /opens can of worms
Well, I can only tell you my own story, I wouldn't know about other
people. Basically, it boils down to the following:

(1) I have no valid usecase for it. I don't remember when was the last
time I needed to resize partitions (probably back when I was trying to
install Windows 95). Disk space is very cheap, and if I really need to
have *that* much data on a single partition, another drive and a few
intelligently placed symlinks are usually enough. Cases where a symlink
cannot do the job are indicative of a bad data structure design, and
LVM is often not a solution, but a patch over a deeper problem
elsewhere. Though I do admit there are some valid usecases for LVM.

(2) It is fragile. If you have data on top of LVM spread over an array
of disks, and one disk dies, the data on the whole array goes away. I
don't know why such a design of LVM was preferred over something more
robust (I guess there are reasons), but it doesn't feel right. A bunch
of flawless drives containing corrupt data is Just Wrong(tm). I know,
one should always have backups, but still...

(3) It's being pushed as default on everyday ordinary users, who have
absolutely no need for it. I would understand it as an opt-in feature
that some people might need in datacenters, drive farms, clouds, etc.,
but an ordinary user installing a single OS on their everyday laptop
just doesn't need it. Jumping through hoops during installation to
opt-in LVM by a small number of experts outweighs similar jumping to
opt-out of it by a large number of noobs.

Also, related to (3), there was that famous Fedora upgrade fiasco a few
Fedora releases back. It went like this:

* A default installation included LVM for all partitions, except
  for /boot, since grub couldn't read inside LVM.
* Six months later, the upgrade process to the next release of Fedora
  happened to require a lot of space in /boot, more than the default
  settings.
* The /boot partition, being the only one outside LVM, was the only one
  that couldn't be resized on-the-fly.
* People who opted-out of LVM usually didn't have a reason to create a
  separate /boot partition, but bundled it under /, circumventing the
  size issue in advance without even knowing it.

So the story ended up with lots of people in upgrading griefs purely
because they couldn't resize the separate /boot partition, and it was
separate because LVM was present, and LVM was present with the goal of
making partition resizing easy! A textbook example of a catch-22,
unbelievable!!

Of course, I know what you'll say --- it wasn't just LVM, but an
unfortunate combination of LVM, limitations of grub, bad defaults and a
lousy upgrade mechanism. And yes, you'd be right, I agree. But the
bottomline was that people with LVM couldn't upgrade (without bending
backwards), while people without LVM didn't even notice that there is a
problem. And since hatred is an irrational thing, you need not look any
further than that. ;-)

Best, :-)
Marko

Marko Vojinovic wrote:
> On Tue, 23 Jun 2015 11:15:30 -0500
> Jason Warr <jason at warr.net> wrote:
>>
>> I'm curious what has made some people hate LVM so much.  I have
been
>> using it for years on thousands of production systems with no issues
>> that could not be easily explained as myself or someone else doing
>> something stupid.  And even those issues were pretty few and far
>> between.
>>
>> /opens can of worms
>
> Well, I can only tell you my own story, I wouldn't know about other
> people. Basically, it boils down to the following:
>
> (1) I have no valid usecase for it. I don't remember when was the last
> time I needed to resize partitions (probably back when I was trying to
> install Windows 95). Disk space is very cheap, and if I really need to
> have *that* much data on a single partition, another drive and a few
> intelligently placed symlinks are usually enough. Cases where a symlink
> cannot do the job are indicative of a bad data structure design, and
> LVM is often not a solution, but a patch over a deeper problem
> elsewhere. Though I do admit there are some valid usecases for LVM.
>
> (2) It is fragile. If you have data on top of LVM spread over an array
> of disks, and one disk dies, the data on the whole array goes away. I
> don't know why such a design of LVM was preferred over something more
> robust (I guess there are reasons), but it doesn't feel right. A bunch
> of flawless drives containing corrupt data is Just Wrong(tm). I know,
> one should always have backups, but still...
<snip>
I thought it was interesting years ago, having seen and worked with it in
Tru64. These days, if I needed more space, I'd go with plain RAID.

In general, the less complex the better, and the easier to recover when
something fails.


      mark

On Tue, Jun 23, 2015 at 1:54 PM, Marko Vojinovic <vvmarko at gmail.com>
wrote:
> On Tue, 23 Jun 2015 11:15:30 -0500
> Jason Warr <jason at warr.net> wrote:
>>
>> I'm curious what has made some people hate LVM so much.  I have
been
>> using it for years on thousands of production systems with no issues
>> that could not be easily explained as myself or someone else doing
>> something stupid.  And even those issues were pretty few and far
>> between.
>>
>> /opens can of worms
>
> Well, I can only tell you my own story, I wouldn't know about other
> people. Basically, it boils down to the following:
>
> (1) I have no valid usecase for it. I don't remember when was the last
> time I needed to resize partitions (probably back when I was trying to
> install Windows 95). Disk space is very cheap, and if I really need to
> have *that* much data on a single partition, another drive and a few
> intelligently placed symlinks are usually enough. Cases where a symlink
> cannot do the job are indicative of a bad data structure design, and
> LVM is often not a solution, but a patch over a deeper problem
> elsewhere. Though I do admit there are some valid usecases for LVM.
>
      AIX does use lvm a lot. Main difference is their filesystem
allows live shrinking. Kinda nice to dynamically size a partition
depending on needs, as opposite to the so often suggested approach of
formatting the entire drive as one single partition. Symlinking is
great until whatever the destination is does not mount. I myself use
lvm as disks for my vm clients, which xenserver does too (not my fault
I promise!). It is faster than an image file.
> (2) It is fragile. If you have data on top of LVM spread over an array
> of disks, and one disk dies, the data on the whole array goes away. I
> don't know why such a design of LVM was preferred over something more
> robust (I guess there are reasons), but it doesn't feel right. A bunch
> of flawless drives containing corrupt data is Just Wrong(tm). I know,
> one should always have backups, but still...
>
      Building a raid0, which is what your example is, and hoping data
will survive in case of a drive failure is wishful thinking. You can
build VLM on the top of a proper raid, or do raid inside lvm
nowadayas... just like zfs,
> (3) It's being pushed as default on everyday ordinary users, who have
> absolutely no need for it. I would understand it as an opt-in feature
> that some people might need in datacenters, drive farms, clouds, etc.,
> but an ordinary user installing a single OS on their everyday laptop
> just doesn't need it. Jumping through hoops during installation to
> opt-in LVM by a small number of experts outweighs similar jumping to
> opt-out of it by a large number of noobs.
>
      That is not lvm's fault, but the distro's decision.
> Also, related to (3), there was that famous Fedora upgrade fiasco a few
> Fedora releases back. It went like this:
>
> * A default installation included LVM for all partitions, except
>   for /boot, since grub couldn't read inside LVM.
> * Six months later, the upgrade process to the next release of Fedora
>   happened to require a lot of space in /boot, more than the default
>   settings.
> * The /boot partition, being the only one outside LVM, was the only one
>   that couldn't be resized on-the-fly.
> * People who opted-out of LVM usually didn't have a reason to create a
>   separate /boot partition, but bundled it under /, circumventing the
>   size issue in advance without even knowing it.
>
      Fedora != lvm unless I have been lied to all these years.
> So the story ended up with lots of people in upgrading griefs purely
> because they couldn't resize the separate /boot partition, and it was
> separate because LVM was present, and LVM was present with the goal of
> making partition resizing easy! A textbook example of a catch-22,
> unbelievable!!
>
> Of course, I know what you'll say --- it wasn't just LVM, but an
> unfortunate combination of LVM, limitations of grub, bad defaults and a
> lousy upgrade mechanism. And yes, you'd be right, I agree. But the
> bottomline was that people with LVM couldn't upgrade (without bending
> backwards), while people without LVM didn't even notice that there is a
> problem. And since hatred is an irrational thing, you need not look any
> further than that. ;-)
>
> Best, :-)
> Marko
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos

On 6/23/2015 11:23 AM, Mauricio Tavares wrote:
>        AIX does use lvm a lot. Main difference is their filesystem
> allows live shrinking. Kinda nice to dynamically size a partition
> depending on needs, as opposite to the so often suggested approach of
> formatting the entire drive as one single partition. Symlinking is
> great until whatever the destination is does not mount. I myself use
> lvm as disks for my vm clients, which xenserver does too (not my fault
> I promise!). It is faster than an image file.
While it has the same concepts, physical volumes, volume groups, logical 
volumes, the LVM in AIX shares only the initials with Linux.  I've heard 
that Linux's LVM was based on HP-UX's design.

in AIX, the LVM is tightly integrated with file system management, so 
you issue the command to grow a file system, and it automatically grows 
the underlying logical volume.   the OS itself can automatically grow 
file systems when its installing software. Also, in AIX, the volume 
manager is the raid manager, you say 'copies = 2' as an attribute of a 
LV, and data is mirrored.



-- 
john r pierce, recycling bits in santa cruz

On Tue, 23 Jun 2015 14:23:52 -0400
Mauricio Tavares <raubvogel at gmail.com> wrote:
> On Tue, Jun 23, 2015 at 1:54 PM, Marko Vojinovic <vvmarko at
gmail.com>
> wrote:
> > On Tue, 23 Jun 2015 11:15:30 -0500
> > Jason Warr <jason at warr.net> wrote:
> >>
> >> I'm curious what has made some people hate LVM so much.
> > 
> > (3) It's being pushed as default on everyday ordinary users, who
> > have absolutely no need for it.
> >
>       That is not lvm's fault, but the distro's decision.
Agreed, but remember that hatred is not a rational thing. When one sees
LVM being pushed onto them by their favorite distro, they are not going
to blame the distro (because it's their favorite distro, you know...),
but rather the LVM itself. Psychology is a curious thing. ;-)
 
> > Also, related to (3), there was that famous Fedora upgrade fiasco a
> > few Fedora releases back. It went like this:
> >
>       Fedora != lvm unless I have been lied to all these years.
That Fedora stunt was just one real-world example of how things can get
drastically wrong, and for a sizable number of people. I wasn't
criticizing LVM, I was answering why some people hate it. :-)

As far as an ordinary noob user thinks, this is how it goes. Things
that participated in the problem were:

 - upgrade software,
 - boot partition,
 - grub bootloader,
 - LVM.

A typical noob user knows they need the first three components for
day-to-day work, and that they don't need the fourth. Also, people who
didn't have the fourth component didn't have the problem. Guess which
of the four will catch the blame? Moreover, the fourth component failed
to help with the problem, despite it being there precisely for
partition resizing. There's nothing more to discuss, it's clear as
day... :-D

Remember, I'm not justifying this "reasoning", just reporting what
I've
seen happen out in the wild, and why some people hate LVM. ;-)

Best, :-)
Marko

On 06/23/2015 10:54 AM, Marko Vojinovic wrote:
> (1) I have no valid usecase for it. I don't remember when was the last
> time I needed to resize partitions (probably back when I was trying to
> install Windows 95). Disk space is very cheap, and if I really need to
> have *that* much data on a single partition, another drive and a few
> intelligently placed symlinks are usually enough. Cases where a symlink
> cannot do the job are indicative of a bad data structure design, and
> LVM is often not a solution, but a patch over a deeper problem
> elsewhere. Though I do admit there are some valid usecases for LVM.
Such as:
1) LVM makes MBR and GPT systems more consistent with each other, 
reducing the probability of a bug that affects only one.
2) LVM also makes RAID and non-RAID systems more consistent with each 
other, reducing the probability of a bug that affects only one.
3) MBR has silly limits on the number of partitions, that don't affect 
LVM.  Sure, GPT is better, but so long as both are supported, the best 
solution is the one that works in both cases.
4) There are lots of situations where you might want to expand a 
disk/filesystem on a server or virtual machine.  Desktops might do so 
less often, but there's no specific reason to put more engineering 
effort into making the two different.  The best solution is the one that 
works in both cases.
5) Snapshots are the only practical way to get consistent backups, and 
you should be using them.
6) If you use virtualization, LV-backed VMs are dramatically faster than 
file-backed VMs.

LVM has virtually zero cost, so there's no practical benefit to not 
using it.

When btrfs comes along and supports flexible volumes, snapshots, and 
reliable storage, then it'll make sense to ditch LVM.  Until then, LVM 
shouldn't even be a question; the answer is yes.

The point of view that LVM isn't needed when a symlink will do is no 
more valid than the opposite point of view: that there's no reason to 
play stupid games with symlinks when you have the ability to manage volumes.
> (2) It is fragile. If you have data on top of LVM spread over an array
> of disks, and one disk dies, the data on the whole array goes away.
That's true of every filesystem that doesn't use RAID or something like 
it.  It's hardly a valid criticism of LVM.
> And since hatred is an irrational thing, you need not look any
> further than that. ;-)
Well, let's not forget that you are the one who said that you despise 
LVM.  As long as you recognize that you aren't rational, I suppose we 
agree on at least one thing. :)

On Tue, 23 Jun 2015 19:08:24 -0700
Gordon Messmer <gordon.messmer at gmail.com>
wrote:
> 
> Such as:
> 1) LVM makes MBR and GPT systems more consistent with each other, 
> reducing the probability of a bug that affects only one.
> 2) LVM also makes RAID and non-RAID systems more consistent with each 
> other, reducing the probability of a bug that affects only one.
OTOH, it increases the probability of a bug that affects LVM itself.

But really, these arguments sound like a strawman. It reduces the
probability of a bug that affects one of the setups --- I have a hard
time imagining a real-world usecase where something like that can be
even observable, let alone relevant.
> 3) MBR has silly limits on the number of partitions, that don't
> affect LVM.  Sure, GPT is better, but so long as both are supported,
> the best solution is the one that works in both cases.
That only makes sense if I need a lot of partitions on a system that
doesn't support GPT. Sure, that can happen (ever more rarely on modern
hardware), but I wouldn't know how common it is. I rarely needed many
partitions in my setups.
> 4) There are lots of situations where you might want to expand a 
> disk/filesystem on a server or virtual machine.  Desktops might do so 
> less often, but there's no specific reason to put more engineering 
> effort into making the two different.  The best solution is the one
> that works in both cases.
What do you mean by engineering effort? When I'm setting up a data
storage farm, I'll use LVM. When I'm setting up my laptop, I won't.
What effort is there? I just see it as an annoyance of having to
customize my partition layout on the laptop, during the OS installation
(customizing a storage farm setup is pretty mandatory either way, so
it doesn't make a big difference).
> 5) Snapshots are the only practical way to get consistent backups,
> and you should be using them.
That depends on what kind of data you're backing up. If you're backing
up the whole filesystem, than I agree. But if you are backing up only
certain critical data, I'd say that a targeted rsync can be waaay more
efficient.
> 6) If you use virtualization, LV-backed VMs are dramatically faster
> than file-backed VMs.
I asked for an explanation of this in the other e-mail. Let's keep it
there.
> LVM has virtually zero cost, so there's no practical benefit to not 
> using it.
If you need it. If you don't need it, there is no practical benefit of
having it, either. It's just another potential point of failure, waiting
to happen.
> The point of view that LVM isn't needed when a symlink will do is no 
> more valid than the opposite point of view: that there's no reason to 
> play stupid games with symlinks when you have the ability to manage
> volumes.
I would agree with this, up to the point of fragility/robustness (see
below).
> > (2) It is fragile. If you have data on top of LVM spread over an
> > array of disks, and one disk dies, the data on the whole array goes
> > away.
> 
> That's true of every filesystem that doesn't use RAID or something
> like it.  It's hardly a valid criticism of LVM.
If you have a sequence of plain ext4 harddrives with several symlinks,
and one drive dies, you can still read the data sitting on the other
drives. With LVM, you cannot. It's as simple as that.

In some cases it makes sense to maintain access to reduced amount of
data, despite the fact that a chunk went missing. A webserver, for
example, can keep serving the data that's still there on the healthy
drives, and survive the failure of the faulty drive without downtime.
OTOH, with LVM, once a single drive fails, the server looses access to
all data, which then necessitates some downtime while switching to the
backup, etc. LVM isn't always an optimal solution.
> > And since hatred is an irrational thing, you need not look any
> > further than that. ;-)
> 
> Well, let's not forget that you are the one who said that you despise 
> LVM.  As long as you recognize that you aren't rational, I suppose we 
> agree on at least one thing. :)
Oh, of course! :-) The ability to be irrational is what makes us human.
Otherwise life would be very boring. ;-)

Best, :-)
Marko