Linus Torvalds schools Lennart Poettering on the importance of users

Janis@feddit.debanned from sitebanned from site to Linux@lemmy.ml – 176 points –
Linus Torvalds schools Lennart Poettering on the importance of users
m.youtube.com
38

Unfortunate title, but it's a good video and some good thoughts from both Linus and AC.

Interestingly, this video is just 2 years after Linus and Alan Cox had a bit of a blowup that caused AC to resign from the TTY subsystem. And even more interestingly, the blowup was specifically about the very topic they're discussing: not breaking userspace and keeping a consistent user experience. Linus felt AC had broken userspace unnecessarily, was too proud/stubborn to revert the change and save the user experience. AC felt Linus was trivializing how easy "just fix it" was going to be and threw up his hands and resigned.

I was curious if they were still on good terms after that, and it's nice to see that they were. For newcomers to Linux, Alan Cox used to be (in the 1990s) the undisputed Riker to Linus' Picard, the #2 in command, ready to take over all of Linux at a moment's notice. As we got into the 2000s, that changed, and this video (2011) was from the middle of a chaotic time for him. In 2009 he quit Red Hat, then joined Intel 2 years later, then quit shortly after that and has just a few years ago stopped kernel development permanently.

Should probably also mention that his wife, Telsa Gwynne, was diagnosed with cancer around the time he retired and she sadly passed away in 2015.

Lennart has always been a joke. Forced systems on us and wrote the disaster of pulseaudio.

Both pulseaudio and systemd brought significant improvements to the Linux ecosystem.

After being a pile of crap forced on us by a corporate giant for many many years. Make no mistake, they are doing embrace extend extinguish just like Microsoft.

How are they going to extinguish anything with an open-source project that is trivial to fork?

I use Pipewire now but Pulseaudio is (and has been for years) better than both the Windows and Mac audio stack. It may have been bad once (yes, I remember the days of having to start Wine with some pulse env var so the audio doesn't crackle) but nowadays it doesn't deserve the level of hate it still gets.

Wasn't Red hat also responsible for Pipewire and Wayland?

I love to hate them for what they did recently, but those two projects kick major ass.

It would have been fine if it wasn't forced. "We are the audio stack everyone should use" but when it doesn't work then it's an ALSA bug and alsa ppl should take the blame (even when it works fine with full alsa, like my audio card). And it was designed more like a networking stack then an audio stack.

Sure it was necessary at the time (so that hdmi, and later bluetooth, would work transparently), but the "i know best" attitude hurt its execution.

SystemD on the other hand brought nothing of value. Did way more harm then good.

Quit your bullshit, nothing was ever forced on you. This is Linux, free software and all that, if you're not happy then use a systemd-less distro and stop complaining about meaningless points. SystemD works very well for me (and the vast majority of the Linux community) and is very easy to use and understand

This. If you want to go back to the days without systemd and writing invit scripts manually, knock yourselves out. The rest of us will continue to live in the modern world of systemd, pulse audio (and now pipe wire).

Udev was changed to depend on systemd. No good reason for it. So it practically was forced. You can lie all you want, it won't change reality. SystemD was hyped up by comparing it with the worst implementation of sysV, at a time when no major distro other then fedora even used sysV. And that is not even the tip of the pile of dishonesty.

Just by saying that it is no better then alternatives of the time will get ignorant people like you to yell. That is how strong the hype was around it. How can you even talk about free software when RH can take a core component and make it hard dependant on whatever they want. Just like bluetooth has a hard dependency on PA. I'm also free to say something sucks, just like you are free to lick their balls.

Use Devuan and quit whining.

It always was about feelings with you fanboys. Pathetic.

PS I wouldn't mind using systemD, it's the same as every other. Functionally absolutely the same.

As someone currently actively supporting two commercial products, one using OpenRC and one using systemd to meet different requirements for different projects

Functionally absolutely the same

Makes it blatantly obvious you have no idea what you’re saying

As someone who wrote an init system for fun and knows how udev and practically everything else associated with bringing a modern computer to a fully functional state (including network mounts, if that is your nitpick) works, i can not know what you are nitpicking about without you saying it. Not that someone who is actively supporting two commercial products to meet different requirements would have any idea what i am saying.

PS It's all simple really, just that it seems magic to people without curiosity.

Sure - it’s primarily the way systemd uses cgroups

For example, systemd’s use of cgroups for process monitoring makes it trivial to support setting resource limits for us

One of the major issues we’re having with systemd, and the reason we’re using OpenRC on a different project, is the way Before and After with targets still cause all the services to start at the same time, causing resource contention

An alternative we’ve used once is to create a special target for the services that had to start early, even if the entire boot took longer, and use a process to then request new targets be started by systemd

This project we found it simpler to use OpenRC, though

Calling them “functionally the same” without taking into account how process monitoring works on different init systems is disingenuous

Process monitoring, in the basic sense, is seeing if a process is running. You mean how they handle dependency trees/graphs ? From what i just read sysD targets are groups that can have other groups in them (aka inherit, aka "services", aka compose). I wonder if that is the core of the problem. Not that i care, that's the hole they dug for themselves when they insisted only pid EINS can orchestrate cgroups (didn't use to be).

Either way, in the overwhelming majority of use cases they are practically the same.

Bdw, i didn't downvote you. I reserve it only for the most irrational fans, aka parroting fanboys.

One of the big issues with process monitoring, in the general sense, is how PID 1 checks on processes

The cgroups usage lets them make use of a very powerful Linux-specific feature. Some competitors such as Upstart tried to use ptrace for this, but that causes services to run slower

“Is a process running” I think is a harder question than you realize. systemd also offers the ability to ask “is a process running correctly” through watchdogs, and “is a process using too much memory” or “is a process using too much CPU” and offer corrective action if they are

The systemd.target issues I mention are related to different design goals. Systemd tries to start as many services as possible at once, but we need some services up within 1 second, and the rest can take longer

One option I offered was a modification to systemd so that targets could handle Before/After during our design, but the maintenance of porting it over for each update versus using OpenRC was decided to be too much effort

when it doesn’t work then it’s an ALSA bug and alsa ppl should take the blame (even when it works fine with full alsa, like my audio card).

Well, yeah. PA used ALSA APIs that most applications didn't, which exposed bugs in little-used, little-tested driver code. Nothing implausible about that.

The standard AC97 and USB audio drivers worked fine—I know they did because that's what I was using with PA at the time—but the drivers for more esoteric audio hardware had yet to be debugged, and Lennart couldn't feasibly test and fix all of them by himself because he didn't have the hardware. Others in the community did, and together they fixed the bugs and eventually got PA working smoothly on everything.

And it was designed more like a networking stack then an audio stack.

Of course. PA was specifically designed to be network transparent, same as the X11 protocol it was typically used with.

Sure it was necessary at the time (so that hdmi, and later bluetooth, would work transparently), but the “i know best” attitude hurt its execution.

Ah, but he was correct. He did, in fact, know best. Lennart Poettering brought an end to the clusterf*** that was Linux audio pre-PA. No one else solved the problem until he came along and said “no more,” and I must say I'm appalled at the ingratitude of his detractors.

SystemD on the other hand brought nothing of value. Did way more harm then good.

Nonsense! Before systemd, startup took forever, shutdown took forever, and it was a crapshoot whether shutdown would succeed or hang. Systemd hasn't fully solved this problem, but it's a lot better than what I had to live with in the bad old days.

Also, systemd brings with it a logging system with integrity checking, structured data, and database-like querying. Huge improvement over BSD syslog.

Also also, systemd has proper process supervision, services can depend on devices, unit/global start/stop timeouts, networkd, user session tracking and cleanup, user services, easy-to-use sandboxing, and on and on and on. There's all kinds of useful goodies in there.

Nobody else solved the problems ? Other then hotplugging audio, init thing have been solved many times over. Wanna know the alsa bug on my audio card ? It calls master volume "master center" instead of master. Good progress on pa has been made after its creator long left. And it's done properly only now with pw. PW... where the dev asked for advice from professionals instead of knowing it all. PA is now x11, without the pedigree.

And what about making udev locked down to one init ? I should be greatful for that ? SystemD didn't make computers boot faster the, say, upstart. Logging does not have to be tied to it, as there are even established protocols for it. Etc etc etc

I should be greatful.. for the one true way

Nobody else solved the problems ?

Nope. Before PA, we had ESD, aRts, and NAS. All of them had horrible latency. They could play a ding at approximately the right time, but everything else was utterly beyond them. They were also mutually incompatible and there was no reliable way for apps to discover which one, if any, they were supposed to use.

Other then hotplugging audio

And multiplexing multiple audio streams to a single input/output device with reasonable latency, and moving an audio stream to a different device, and individually controlling audio streams' volume, and sending audio streams over the network or Bluetooth with reasonable latency, and…

Yeah, no. Linux audio sucked before PA came along. I know it did because I was there, struggling with it.

Wanna know the alsa bug on my audio card ? It calls master volume “master center” instead of master.

Okay? I didn't claim that PA never had any bugs of its own. All software has bugs.

And it’s done properly only now with pw. PW… where the dev asked for advice from professionals instead of knowing it all. PA is now x11

I also didn't claim PA is perfect, nor that some newer, better system will never come along. I claimed that PA is vastly better than what came before it, which is correct, and I have the experience to prove it.

And what about making udev locked down to one init ?

Red Hat chose to cease maintenance of the separate udev and focus its efforts on the systemd-integrated udev. Others took up the task of maintaining a separate udev, and called their fork eudev. I'm not seeing any problem with this.

SystemD didn’t make computers boot faster the, say, upstart.

Upstart expects daemons to SIGSTOP themselves to signal readiness. That is not a sound design.

I don't know if there's anything else wrong with it, as I have never used it myself, but I have yet to hear any well-reasoned explanation of how it's better than systemd.

Logging does not have to be tied to it

It isn't. You can still use a syslog daemon with systemd if you want.

as there are even established protocols for it.

Yes, and have you ever tried to implement them? I have. They suck.

  • Extremely easy to format or parse incorrectly.
  • Non-extensible, leading to a proliferation of dialects, making parsing log records a minefield.
  • No way to include line feeds or other control characters in a log record.
  • There are two different, mutually incompatible protocols, only one of which is described by IETF RFCs. The RFCs describe the protocol used for logging on another machine. There do not appear to be any formal standards for the local logging protocol used on /dev/log. There isn't even a specification saying /dev/log exists at all.
  • No structured data, outside of an obscure RFC that has, as far as I can tell, zero implementations.
  • 1024 bytes max per log record. If you want to include a stack trace with your log message, too bad, not happening.

And that's just the log record submission protocol. The storage format has all of the above problems except the one about /dev/log, and several more:

  • The only real standard for how log records are stored is that they're delimited by line feeds. That's it. Everything else is configurable and therefore unpredictable. Parsing them reliably is basically impossible.
  • If the syslog daemon has a hiccup, log records can be smushed together without a line feed delimiter, making them extra-impossible to parse. There is no checksum or length field with which to automatically detect that this has happened.
  • Log records are typically separated into numerous different log files. Don't know which one your program's log records are ending up in? Here's grep; good luck.
  • Owing to the aforementioned parsing difficulties, there is no reliable way to filter logs by program, by date, etc. Here's grep; good luck.
  • The systemd service (or equivalent) from which a log record originated is not recorded in the log. Better hope your program doesn't use a different name than the one you expect it to use. I know of at least one that has this behavior: cron.
  • There is no record of whether a log record was generated during the current boot or the previous one. Want to only see log records since the last reboot? Here's grep; good luck.
  • Did I mention there are no checksums? Because there are no checksums.

This may have been acceptable in the 1980s, but it doesn't hold a candle to the systemd journal. Good riddance.

I should be greatful… for the one true way

You should be grateful for software that is vastly better than its predecessors with no significant drawbacks, yes.

ALSA wasn't going to take the Linux desktop anywhere

PulseAudio sits on top of ALSA, so that isn't entirely true. ALSA alone isn't a modern audio stack, but it is part of one.

i dont get why youre being downvoted.i m with you.

Linux community is so inherently meritocratic that one can't meaninfully force anything upon any large group of them.

Thore particular two creations of Lennart took the world by storm precisely because they were so absurdly good that working on other stuff was a dead-end, obvious for all but such tiny fraction of people that even forming vacuous hate bubbles haven't rallied enough effort to foster and maintain alternatives.

It became trendy to hate Pulseaudio and call it bloat years after Nokia shipped a rather anemic phone where it already worked flawlessly. I need no further proof that there's no technical basis beneath the hate.

Linux community is so inherently meritocratic that one can't meaningfully force anything upon any large group of them.

Even for developers, there is a very substantial cost to any deviation from the herd and little time or money for these projects. Factually a handful of companies run the Linux userspace and a handful of people run those companies.

You can go your own way but existing market share and resources matter more than quality or merit.

As a Red Hat employee who had his all-around sensible Fedora Change to prevent it from falling too far behind RHEL (!) rejected, I think I can confidently claim that your statements smell of conspiracy theories.

Do Linux-involved companies have resources to develop the projects they like the most? Yes. Do companies dominate userspace development? I don't think so, in fact, they're all seem quite focused in their interests, and their involvement with a median package on your community distro desktop system isn't even minimal, it's none. Do the se companies at least all push for a united agenda? Absolutely not. Can they force a single random community distro like Debian to pick something over something else? No. 99% of the distros? Goes without saying.

It's not a conspiracy theory to imagine that IBM's vision for Linux compared to 2000s or 2010s era Linux is opaque, complicated, and enterprisey. It's who they are.

The grandparent comment

Linux community is so inherently meritocratic that one can’t meaningfully force anything upon any large group of them.

Is pure fantasy. Software projects are dictatorships of those willing to put in the work, not meritocracies. There is nothing immoral or wrong about this but we should be realists. The entire software ecosystem is dominated by oft shitty good enough solutions which people poured enough work into to solve problems well enough.

IBM's vision

Anybody can have a vision, but it's the work that matters. I'll be worried when they become a player.

Software projects are dictatorships of those willing to put in the work, not meritocracies.

Most linux distros are slight variations on the best components available. Yes, one can put in resources, do a great job and now everyone switches to the fruits of their labor. No, it does nothing to stop another player from one-upping them and taking the lead with their next best good enough. In political terms, dictatorships are incompatible with voting with one's feet.

Anybody can have a vision, but it’s the work that matters. I’ll be worried when they become a player.

Did you entirely miss the part where IBM bought Red Hat

It was hard to miss, I was working there. It doesn't mean that much as you're assuming.

Even for developers, there is a very substantial cost to any deviation from the herd and little time or money for these projects.

If everyone thought that way, we'd all still be using MS-DOS. This is an absurd take for a Linux user.

I didn't say we should be satisfied with "good enough" I said in practice we build our sandcastles on the beach irrespective of the approaching tide because it's what we know. In that context, it is no longer an absurd take it's just recognizing reality.