Why is Gnome fractional scaling 1.7518248558044434 instead of 1.75?

ugjka@lemmy.world to Linux@lemmy.ml – 104 points –
Why is Gnome fractional scaling 1.7518248558044434 instead of 1.75?
unix.stackexchange.com
32

You are viewing a single comment

TBF the error can become that big if you do a bunch of unstable operations (i.e. operations that continue to increase the relative error), though that's probably not what is happening here.

To get to 0.01 error, you'd need to add up trillions of trillions of floating point errors. It will not happen solely because of floating point unless you're doing such crazy math that you shouldn't be using primitives in the first place.

That's why I said unstable operations. Addition is considered a stable operation (for values with the same sign)