TIL You can use `systemd-analyze plot > plot.svg` to plot the service startup time to find bottlenecks

qaz@lemmy.world to Linux@lemmy.ml – 775 points –

Apparently the reason my computer has been taking 2 minutes to boot was a faulty network mount

60

I'm pretty sure the main system startup bottleneck is me typing the disk encryption passphrase.

Combine that with the 20-30 seconds my system takes to do bios memory training on the DDR5 ram and we’re practically back to the “go make some coffee while the system boots up” days 🤦

Glad I haven't built a modern chipset PC yet, didn't realize it was this bad.

As another DDR5 user, it’s not always this bad - there’s a bios setting that makes it remember the previous configuration and skips this step, but sometimes it still needs to do it, and then it can take a minute or two

Those where the good days. You always had fresh coffee when your computer was ready for work.

I wish to replace it with a yubikey, but I don't even know if it's supported.

It is supported by systemd to use FIDO2 + pin to decrypt luks partitions with many security keys, including Yubikeys. I use it every day on my laptop.

It is, I have it set up on my laptop. It's a bit finicky in how it works and it's not easy to setup, but it is possible.

Does it work by emulating the keyboard and typing in the password? Or by the encrypted protocol that works using the on device secret?

Both should be possible. I am using the psuedo 2FA method. First I type the PIN and after that I confirm with YubiKey.

You can't even use a fucking fingerprint scanner while being in the system, that package is borked for months and nobody seem to care to solve it.

I think using Yubikey at boot time is quite out of reach

My system bottleneck is the damn Bios Post

Fucking true. Does anyone know why this is so slow?

You can use systemd-analyze blame if you want raw numbers:

This command prints a list of all running units, ordered by the time they took to initialize. This information may be used to optimize boot-up times.

Good way to see if your systemd also waits 2 minutes for a network connection which already exists but it can't see it because systemd doesn't do the networking (lxc containers on proxmox in my case) lol.

Also see systemd-analyze.

Also systemd-analyze critical-chain

systemd also waits 2 minutes for a network connection which already exists but it can’t see it because systemd doesn’t do the networking

Any way to speed this up? On my system in every boot it waits for network for 30s.

In my case i masked the service because like i said, inside the lxc container there is no networking to do, it's done on the host (proxmox). Note that disabling the service in my case was not enough since it could be invoked by other services, and then you would have to wait again.

See this for further info and maybe arguments why you shouldn't do it.

My main offender is something called updatedb.service, whatever that is.

Systemd can generate SVGs? Damn thats "bloat" but also unexpectedly fancy

SVGs are just fancy text files after all

If you go far enough, everything is.

But SVGs are one of the few image types that can be human readable and editable

If you go far enough, everything is.

No, SVG are text files, it's XML. You can write an SVG file representing a square using only a text editor relatively easily.

1 more...
1 more...
1 more...

Is there a way to do this for shutdown? It'd be great to debug shutdown hangs.

No, there currently isn’t

And it's not as easy to add actually. Note that systemd only keeps units loaded as long as they are referenced by something else that is loaded, are running, have failed, or have a job queued. That means if a service is terminated at shutdown there's a very good chance it is GC'ed away pretty quickly. Now, while systemd keeps timestamping info around for services that tell us how long a service was running, took to start or took to shut down all that info is lost the instant the unit is GC'ed away...

Source

TIL: Systemd is great and despite its usefulness, it is often overlooked due to controversy and the current state of things when it comes to software development. https://tadeubento.com/2023/systemd-hidden-gems-for-a-better-linux/

Thanks for the article, I've already spotted a few utilities that can come in useful. I've heard a lot of criticism about systemd too, but never really actively used it myself until a few weeks ago. I actually quite like it from what I've seen so far.

I wrote a long-ish comment in another thread explaining why lots of people don’t like systemd.

Stuff like this is why people do like systemd.

The massive, un unixy and complex tools allow for very powerful and somewhat knowledge agnostic approaches to all sorts of problems.

One of the nicest things about systemds toolset is that it allows a person who relies on finding the problem and googling it to resolve thing much faster than their alternative, learn what’s going on and figure it out.

I don’t mean that as a pejorative, plenty of computer work is maintenance as opposed to engineering and there’s nothing wrong with that.

My bottleneck at boot is my damn Bios... I am so hyped about flashing Heads on my Thinkpad T430.

Even the old legacy Lenovo bioses where very fast at startup. The UEFI (with extremely nice secure-boot settings too) of an AMD Acer starts up in like 2 seconds. My old intel Thinkpad T430 needs like 4 seconds.

And then my Lenovo T495 bullshit UEFI comes. No secure boot configuration at all, I have no idea how to boot from USB sticks, and this thing needs nearly 10 seconds to boot! Linux compared, a full Desktop OS, needs 3 seconds to show SDDM (after the LUKS dialog)

I adore my T530. I could kill a moose with it if it ever stops working. Literally dug it out of a dumpster and saw the i7 sticker and almost shit myself. Honestly I've had it for years and never even looked at the bios cuz with an SSD even with encryption enabled on the disk it booted in 30 sec.

Until I built my latest rig I was doing ai image generation on it with 8gigs of ram.

If you have a T530, there is coreboot for it! Dont know if 1vyra.in works, check it.

Its not the question, if it works, but how it works! Its trustworthy and not extremely outdated proprietary garbage. Actually extremely important to update

Thanks friend!

You can even install Heads! This requires Hardware flashing though, not soo easy like 1vyrain but actually secure.

Heads

Dang, my T410 is just too old.

how does it run modern OSses?

Its crazy how expensive T430 etc still are. People know how great they are

I'm shocked how smoothly it runs Gnome on Debian 12. KDE on NixOS was okay but had some noticeable slowdown. Here I was thinking it would be relegated to being an Arch CLI terminal.

The good ol' Gnome on Wayland on SystemD on GNU on Linux trick

the only "bottleneck" i currently have is plymouth-quit-wait.service, which takes 3.9 seconds. i can live with that

I know you put bottleneck is quotes but just to explain... apparently this service is simply the splash screen that waits on a ready environment. It doesn't actually delay anything.

It tells me that my system boots in 7 seconds. That's pretty cool, considering that it's installed on a plain old sata SSD.

POST, however...

Dang had no idea this was a thing, but this looks very useful! I've been meaning to troubleshoot slow startup on one of my servers.

this is interesting! if i had a two minute boot time, I'd look for ways to figure out what's going on.

i remember init messages used to be printed to the console, but nowadays all i get is Manjaro branding. anyone know how to get my console messages back from systemd?

If you hit a key when the Manjaro logo is up (or maybe just ESC) it will go away and you'll get your messages back. Hit it again and you'll get the logo back. The splash screen is due to a program called Plymouth if you want to remove it permanently

Aha! Reminds me of the good old days when I tried to minimize boot up time on my puny Ivy Bridge i5 laptop. Those days were fun!

Damn I really needed systemd analyzer to debug stuff! Thanks!

How many times a day you guys reboot? 236? Mine takes like 17 seconds... Every week or so...

11s on my laptop which i boot once a day, but it is useful for diagnostics. I had something hanging once during boot and it's pinpointed it right away.

Anyone know about a Windows equivalent for my work laptop?

You can use the Windows Performance Recorder to capture the boot time and then use Windows Performance Analyzer to view the results. It should also be able to show the results as a timeline.