On Wed, Dec 31, 2014 at 11:03 AM, Warren Young <wyml at etr-usa.com> wrote:> On Dec 29, 2014, at 10:07 PM, Les Mikesell <lesmikesell at gmail.com> wrote: > >> it's not necessary for either code interfaces or data structures >> to change in backward-incompatible ways. > > You keep talking about the cost of coping with change, but apparently you believe maintaining legacy interfaces is cost-free. > > Take it from a software developer: it isn?t.OK, but should one developer make an extra effort or the bazillion people affected by it?> People praise Microsoft for maintaining ancient interfaces, and attribute their success to it, but it?s really the other way around: their success pays for the army of software developers it takes to keep a handle on the complexity that results from piling 20-30 years of change on top of the same base.That's what it takes to build and keep a user base.> Most organizations cannot afford to create the equivalents of WOW64, which basically emulates Win32 on top of Win64. (Or *its* predecessor, WOW, which emulates Win16 on top of Win32.) That isn?t trivial to do, especially at the level Microsoft does it, where a whole lot of clever low-level code is employed to allow WOW64 code to run nearly as fast as native Win64 code.It's hard to the extent that you made bad choices in interfaces in the first place. Microsoft's job was hard. But Unix SysV which Linux basically emulates wasn't so bad. Maybe a few size definitions could have been better.> Result? We cannot afford to maintain every interface created during the quarter century of Linux?s existence. Every now and then, we have to throw some ballast overboard.And the user base that depended on them.>>> If your software is DBMS-backed and a new feature changes the schema, you can use one of the many available systems for managing schema versions. Or, roll your own; it isn?t hard. >> >> Are you offering to do it for free? > > This is one of the things my employer pays me to do. This is what I?m telling you: the job description is, ?Cope with change.?So either it "isn't hard", or "you need a trained, experienced, professional staff to do it". Big difference. Which is it?>> I'm asking if computer science has advanced to the point where adding >> up a total needs new functionality, or if you would like the same >> total for the same numbers that you would have gotten last year. > > Mathematics doesn?t change. The business and technology worlds do. Your example is a non sequitur.If you are embedding business logic in your library interfaces, something is wrong. I'm talking about things that are shipped in the distribution and the commands to manage them. The underlying jobs they do were pretty well established long ago.> How many single computers have to be up 24/7? I mean really.All of our customer-facing services - a nd most internal infrastructure. Admittedly, not individual boxes - but who wants to have systems running concurrently with major differences in code base and operations/maintenance procedures?> If you have any form of cluster ? from old-school shared-everything style to new-style shared-nothing style ? you can partition it and upgrade individual nodes.Yes, everything is redundant. But when changes are not backwards compatible it makes piecemeal updates way harder than they should be. Take something simple like the dhcp server in the disto. It allows for redundant servers - but the versions are not compatible. How do you manage that by individual node upgrades when they won't fail over to each other?> If your system isn?t in use across the world, you must have windows of low or zero usage where upgrades can happen. If your system *is* in use across the world, you likely have it partitioned across continents anyway.How nice for you...>>> This means we?re still building binaries for EL3. >> >> I have a few of those, but I don't believe that is a sane thing to recommend. > > It depends on the market. A lot of Linux boxes are basically appliances. When was the last time you upgraded the OS on your home router? I don?t mean flashing new firmware ? which is rare enough already ? I mean upgrading it to a truly different OS. > > Okay, so that?s embedded Linux, it doesn?t seem remarkable that such systems never change, once deployed.Which sort of points out that the wild and crazy changes in the mainstream distributions weren't all that necessary either...>>> This also means our software must *remain* broadly portable. >> >> You'd probably be better off in java if you aren't already. > > If you actually had a basis for making such a sweeping prescription like that, 90% of software written would be written in Java.I do. We have a broad mix of languages, some with requirements that force it, some just for historical reasons and the team that maintains it. The java stuff has been much less problematic in porting across systems - or running the same code concurrently under different OS's/versions at once. I don't think the C++ guys have even figured out a sane way to use a standard boost version on 2 different Linux's, even doing separate builds for them.> There?s a pile of good reasons why software continues to be written in other languages, either on top of other runtimes or on the bare metal.Maybe. I think there's a bigger pile of not-so-good reasons that things aren't done portably. Java isn't the only way to be portable, but you don't see much on the scale of elasticsearch, jenkins or opennms done cross-platform in other languages.> No, don?t argue. I don?t want to start a Java flame war here. Just take it from a software developer, Java is not a universal, unalloyed good.The syntax is cumbersome - but there are things like groovy or jruby that run on top of it. And there's a lot of start-up overhead, but that doesn't matter much to long-running servers.> Take systemd. You can go two ways here: > > 1. sysvinit should also be supported as a first-class citizen in EL7. If that?s your point, then just because the sysvinit code was already written doesn?t mean there isn?t a cost to continuing to maintain and package it. > > 2. sysvinit should never have been replaced. If that?s your position, you?re free to switch to a sysvinit based OS, or fork EL6. What, sounds like work? Too costly? That must be because it isn?t free to keep maintaining old code.Yes, I'm forced to deal with #1. That doesn't keep me from wishing that whatever code change had been done had kept backwards compatibility in the user interface commands and init scripts department.>>> I?ve never done much with Windows Server, but my sense is that they have plenty of churn over in their world, too. >> >> Yes, there are changes - and sometimes mysterious breakage. But an >> outright abandonment of an existing interface that breaks previously >> working code s pretty rare > > Yes, well, that?s one of the things you can do when you?ve got a near-monopoly on PC OSes, which allows you to employ 128,000 people. [1]And you only get that with code that keeps users instead of driving them away.>> And conversely, they felt is was worth _not_ doing for a very very >> long time. So can the rest of us wait until we have google's >> resources? > > You?re never going to have Google?s resources. Therefore, you will never have the *option* to roll your own custom OS. > > So, cope with change.What google does points out how unsuitable the distro really is. I just don't see why it has to stay that way. -- Les Mikesell lesmikesell at gmail.com
On Dec 31, 2014, at 4:41 PM, Les Mikesell <lesmikesell at gmail.com> wrote:> On Wed, Dec 31, 2014 at 11:03 AM, Warren Young <wyml at etr-usa.com> wrote: >> You keep talking about the cost of coping with change, but apparently you believe maintaining legacy interfaces is cost-free. >> >> Take it from a software developer: it isn?t. > > OK, but should one developer make an extra effort or the bazillion > people affected by it?That developer is either being paid by a company with their own motivations or is scratching his own itch. You have no claim on his time. Open source is a do-ocracy: those who do the work, rule. People throw all kinds of hate at Poettering, but he is *doing things* and getting those things into consequential Linux distributions. The haters are just crying about it, not out there trying to do things differently, not trying to win an audience. I don?t just mean an audience standing around your soapbox. Crazies on the street corner can manage that. I mean an audience of people who are willing to roll your new Linux distro out over their infrastructure. I repeat my call to action: If you think EL6 was better, fork it. If enough people agree with you that sitting still is better than what Red Hat is doing, you *will* get Red Hat?s attention. Even if the end the result is an EGCS-like divergence and re-merging, you will cause something to happen.>> Result? We cannot afford to maintain every interface created during the quarter century of Linux?s existence. Every now and then, we have to throw some ballast overboard. > > And the user base that depended on them.Nonsense. This is open source. EL6 is still there, if that?s what you want. You?re free to maintain it as long as you like.>>> Are you offering to do it for free? >> >> This is one of the things my employer pays me to do. This is what I?m telling you: the job description is, ?Cope with change.? > > So either it "isn't hard", or "you need a trained, experienced, > professional staff to do it". Big difference. Which is it?You?re trying to shove a crowbar in where there is no gap: It isn?t hard for an experienced staff. It is one of the reasons our various organizations employ us. Why do you believe this is a stringent requirement? I thought CentOS was the distro targeted at organizations staffed by competent technical professionals. That?s me. Isn?t that you, too?>>> I'm asking if computer science has advanced to the point where adding >>> up a total needs new functionality, or if you would like the same >>> total for the same numbers that you would have gotten last year. >> >> Mathematics doesn?t change. The business and technology worlds do. Your example is a non sequitur. > > If you are embedding business logic in your library interfaces, > something is wrong.Once again you?re making non sequiturs. Your example was that arithmetic doesn?t change, then you go off and try to use that to explain why EL7 is wrong. So, where is the part of EL7 that doesn?t add columns of numbers correctly? When the rest of the technology world changes, Red Hat has two options: they can react to it, or they can keep churning out the same thing they always have done. The latter is a path toward irrelevance. OS/2 is a fine OS for solving 1993?s problems. Given a choice between disruptive change and a future where RHEL/CentOS is the next OS/2, I?ll take the change.> Take something simple like the dhcp server in the disto. It allows > for redundant servers - but the versions are not compatible. How do > you manage that by individual node upgrades when they won't fail over > to each other?Is that hypothetical, or do you actually know of a pair of dhcpd versions where the new one would fail to take over from the older one when run as its backup? Especially a pair actually shipped by Red Hat? I?m not interested in the reverse case, where an old server could not take over from a newer one, because there?s no good reason to manage the upgrade that way. You drop the new one in as a backup, take the old one offline, upgrade it, and start it back up, whereupon it can take over as primary again.>> Okay, so that?s embedded Linux, it doesn?t seem remarkable that such systems never change, once deployed. > > Which sort of points out that the wild and crazy changes in the > mainstream distributions weren't all that necessary either?No. The nature of embedded systems is that you design them for a specific task, with a fixed scope. You deploy them, and that?s what they do from that point forward. (Routers, print servers, media streamers?) New general purpose OSes do more things than their predecessor did. Their scope is always changing. (Widening, usually, but occasionally old functionality does get chopped off.) If all you need is a print server, go buy a Lantronix box and be done with it. If you need the power of CentOS, you?re probably not actually doing the same thing with it year after year. Just to keep with the print server example, I know there?s been a lot of change in *my* world with printing in the past 10-20 years. The change from parallel to USB wasn?t trivial. Plug & play device support is one of those things the hated systemd does for us. In the past 5 years or so, I?ve pretty much retired all my Samba print servers, because every printer I?ve bought during that time has a competent print server built in. It?s no surprise then that CentOS 7 isn?t anything like Red Hat Linux 7.>>> You'd probably be better off in java if you aren't already. >> >> If you actually had a basis for making such a sweeping prescription like that, 90% of software written would be written in Java. > > I do. ...The java stuff has been much less problematic in porting across > systems - or running the same code concurrently under different > OS's/versions at once.And yet, 90% of new software continues to *not* be developed in Java. Could be there are more downsides to that plan than upsides.> I don't think the C++ guys have even figured > out a sane way to use a standard boost version on 2 different Linux's, > even doing separate builds for them.This method works for me: # scp -r stableserver:/usr/local/boost-1.x.y /usr/local # cd /usr/local # ln -s boost-1.x.y boost Then build with CXXFLAGS=-I/usr/local/boost/include.>>>> I?ve never done much with Windows Server, but my sense is that they have plenty of churn over in their world, too. >>> >>> Yes, there are changes - and sometimes mysterious breakage. But an >>> outright abandonment of an existing interface that breaks previously >>> working code s pretty rare >> >> Yes, well, that?s one of the things you can do when you?ve got a near-monopoly on PC OSes, which allows you to employ 128,000 people. [1] > > And you only get that with code that keeps users instead of driving them away.Seriously? I mean, you actually believe that if RHEL sat still, right where it is now, never changing any ABIs, that it would finally usher in the Glorious Year of Linux? That?s all we have to do?
On Fri, Jan 2, 2015 at 5:52 PM, Warren Young <wyml at etr-usa.com> wrote:>> >> OK, but should one developer make an extra effort or the bazillion >> people affected by it? > > That developer is either being paid by a company with their own motivations or is scratching his own itch. You have no claim on his time.Agreed - but I'm not going to say I like his breakage.> > Why do you believe this is a stringent requirement? I thought CentOS was the distro targeted at organizations staffed by competent technical professionals. That?s me. Isn?t that you, too?Yes, but I'd rather be building things on top of a solid foundation than just using planned obsolescence as job security since the same thing needs to be done over and over. And I'll admit I can't do it the right way with the approach google uses of just tossing the distribution and its tools.>>> Mathematics doesn?t change. The business and technology worlds do. Your example is a non sequitur. >> >> If you are embedding business logic in your library interfaces, >> something is wrong. > > Once again you?re making non sequiturs. > > Your example was that arithmetic doesn?t change, then you go off and try to use that to explain why EL7 is wrong. So, where is the part of EL7 that doesn?t add columns of numbers correctly?If the program won't start or the distribution libraries are incompatible (which is very, very likely) then it isn't going to add anything.>> Take something simple like the dhcp server in the disto. It allows >> for redundant servers - but the versions are not compatible. How do >> you manage that by individual node upgrades when they won't fail over >> to each other? > > Is that hypothetical, or do you actually know of a pair of dhcpd versions where the new one would fail to take over from the older one when run as its backup? Especially a pair actually shipped by Red Hat?It's my experience or I wouldn't have mentioned it. I built a CentOS7 to match my old CentOS5 pair. It can do the same thing, but there is no way to make them actively cluster together so the new one is aware of the outstanding leases at cutover or to have the ability to revert if the new one introduces problems.> I?m not interested in the reverse case, where an old server could not take over from a newer one, because there?s no good reason to manage the upgrade that way. You drop the new one in as a backup, take the old one offline, upgrade it, and start it back up, whereupon it can take over as primary again.The ability to fail back is important, unless you think new software is always perfect. Look through some changelogs if you think that...>> Which sort of points out that the wild and crazy changes in the >> mainstream distributions weren't all that necessary either? > > No. The nature of embedded systems is that you design them for a specific task, with a fixed scope. You deploy them, and that?s what they do from that point forward. (Routers, print servers, media streamers?)Well yes. We have computers doing specific things. And those things span much longer that 10 years. If you are very young you might not understand that.>>>> You'd probably be better off in java if you aren't already. >>> >>> If you actually had a basis for making such a sweeping prescription like that, 90% of software written would be written in Java. >> >> I do. ...The java stuff has been much less problematic in porting across >> systems - or running the same code concurrently under different >> OS's/versions at once. > > And yet, 90% of new software continues to *not* be developed in Java.Lots of people do lots of stupid things that I can't explain. But if numbers impress you, if you count android/Dalvik which is close enough to be the stuff of lawsuits, there's probably more instances of running programs than anything else.> Could be there are more downsides to that plan than upsides.You didn't come up with portable non-java counterexamples to elasticsearch, jenkins, opennms, etc. I'd add eclipse, jasper reports and the Pentaho tools to the list too - all used here.>> I don't think the C++ guys have even figured >> out a sane way to use a standard boost version on 2 different Linux's, >> even doing separate builds for them. > > This method works for me: > > # scp -r stableserver:/usr/local/boost-1.x.y /usr/local > # cd /usr/local > # ln -s boost-1.x.y boost > > Then build with CXXFLAGS=-I/usr/local/boost/include.I don't think CMake is happy with that since in knows where the stock version should be and will find duplicates, And then you have to work out how to distribute your binaries. Are you really advocating copying unmanaged.unpackaged libraries around to random places.>>>>> I?ve never done much with Windows Server, but my sense is that they have plenty of churn over in their world, too. >>>> >>>> Yes, there are changes - and sometimes mysterious breakage. But an >>>> outright abandonment of an existing interface that breaks previously >>>> working code s pretty rare >>> >>> Yes, well, that?s one of the things you can do when you?ve got a near-monopoly on PC OSes, which allows you to employ 128,000 people. [1] >> >> And you only get that with code that keeps users instead of driving them away. > > Seriously? I mean, you actually believe that if RHEL sat still, right where it is now, never changing any ABIs, that it would finally usher in the Glorious Year of Linux? That?s all we have to do?Yes, they can add without changing/breaking interfaces that people use or commands they already know. The reason people use RHEL at all is because they do a pretty good job of that within the life of a major version. How can you possibly think that the people attracted to that stability only want it for a short length of time relative to the life of their businesses. -- Les Mikesell lesmikesell at gmail.com
> On Jan 2, 2015, at 4:52 PM, Warren Young <wyml at etr-usa.com> wrote: > > I?m not interested in the reverse case, where an old server could not take over from a newer one, because there?s no good reason to manage the upgrade that way. You drop the new one in as a backup, take the old one offline, upgrade it, and start it back up, whereupon it can take over as primary again.Most professional upgrades require a written and tested roll-back plan ? and a published set of criteria (usually down-time) where the roll-back out of the failed ?upgrade? MUST occur. Just sayin?? that?s a professional upgrade. What you described left out a lot of steps. An upgrade where the admin is ?not interested? in going backward, is not what most of our employers will allow these days. Most of the ways to manage that on most OS?s without utilizing virtual machines, or entire filesystem snapshots? is truly awful if you try to do without either of those. The package managers sure aren?t written for it. It?s virtually impossible to find package management systems that can handle a properly done professional upgrade/test/revert cycle on any OS? (well, not without buying them, anyway? and they?re still slim pickin?s on *nix.) The other area most OS package managers suck badly is in creating IDENTICAL systems. It takes a LOT of sysadmin work and coddling of the package management tools currently en vogue, to make them do what businesses really want: e.g. If the QA machine has these exact 1000 packages installed on it, by god, that?s what I want on the Pre-Production server, and later on the Production server when they eventually get updates. I don?t want ?yum update? grabbing whatever?s current, on those three different dates. Most of us are leveraging things like Ansible and Puppet to force such things? Things a good package management system would know how to do? since every business I?ve ever worked at needs both the ability to roll back, and the ability to make identical systems/package sets. Ah well, rant off? ? Nate