Redox OS - an OS built entirely out of Rust

ChiefSinner@lemm.ee to Linux@lemmy.ml – 244 points –
redox-os.org

This isn't Linux, but Linux-like. Its a microkernel built from the rust programming language. Its still experimental, but I think it has great potential. It has a GUI desktop, but the compiler isn't quite fully working yet.

Has anyone used this before? What was your experience with it?

Note: If this is inappropriate since this isn't technically Linux, mods please take down.

95

I don't understand the obsession with rust.

From my personal experience I can tell you 2 reasons. The first is that this is the first general purpose language that can be used for all projects. You can use it on the web browser with web assembly, it is good for backend and it also is low level enough to use it for OS development and embedded. Other languages are good only for some thing and really bad for others. The second reason is that it is designed around catching errors at compile time. The error handling and strict typing forces the developer to handle errors. I have to spend more time creating the program but considerably less time finding and fixing bugs.

That sounds pretty great. I get sick of having to switch gears for every layer. As a hobbyist it is tough to remember five or six languages well enough when only coding something a few times a year.

Since I do embedded, scripting, web front and back end this is sure tempting.

I have been hesitant to try to learn yet another language (this would make...ummm.. idk I lost count ages ago). But with all the hype I may break down and give it a whirl.

8 more...

I feel like C++ is as competent as Rust for any project and it's definitely older.

Not sure I can think of anything I'd enjoy less than trying to build a web app in cpp

Before using Rust I was using C++ for most projects and while it is a really powerful language there were some big problems:

  • no standard build system, most projects use cmake or meson and vendor dependencies with the projects. These build systems were really hard to learn (especially cmake, meson is easier). There are package managers these days such as conan and vcpkg but there is not really one standard way to build programs like in rust.
  • error messages were really hard to understand, especially when the project uses templates
  • it felt like 3 languages in one, projects written before c++11 differ greatly from c++11 and up
  • some of the new language features have really weird syntax, for example lambdas
  • some people say that rust is hard, but modern c++ is considerably harder to learn, just look at the list of modern c++ features: https://github.com/AnthonyCalandra/modern-cpp-features, you have to know the different pointer types (unique_pointer, shared_pointer etc.), templates, rvalue references and move semantic, exceptions, constexpressions and the list goes on

At least it won't complain about rigid, skolem or occurrence check.

Rust was created because c++ was so bad. Just take a look at crates they need a whole lot less maintenance because less bugs.

My point wasn't that C++ is good. My point was that C++ can and is used everywhere (desktop applications, web applications, OSs,...) and is older than Rust. So I feel that "this is the first general purpose language that can be used for all projects" is false. Probably "this is the first general purpose language that I (and many others) like to use for all projects" is true, but is a different claim.

TLDR: You said Rust was first language capable of system, app and web, it isn't.

It depends on what ā€œcan be usedā€ means. I really like C# and it ā€œcan be usedā€ for that full stack C# for example can write out native machine code, can manually and precisely lay out memory, and can directly link to assembly language routines. You can write an OS in C#. Even as a fan though, I would certainly argue that it is the wrong tool for that job.

In the same vein, while I know C++ ā€œcanā€ be used for web dev, I would argue that anybody that tries to do so for any significant project is insane.

I am not sure I would use Rust for ā€œeverythingā€ but I do think the claim that Rust is one of the first languages where it is reasonable or practical to choose it for any of these uses is valid. Rust code can be very high level and often does not much different than a scripting language. At the same time, it can go as low-level as you want. This article is about an OS in Rust ( and there are few ). Web dev in Rust is totally reasonable and there are a few popular frameworks available. Rust has one of the best WASM stories around.

It is good and rust improves on its gaping weaknesses.

Yeah I never made that claim the threads OP did.

8 more...

I know the evangelists can be somewhat overwhelming, but its popularity is not unwarranted. It's fairly easy to pick up, has an incredibly enthusiastic and welcoming community. People like it because it's incredibly performant, and its memory safe. In terms of DX it's really a joy to work with. It just has a LOT going for it, and the main drawback you'll hear about (difficulty) is really overblown and most devs can pick it up in a matter of months.

The main difficulty I have with Rust (what prevents me from using it), is that the maintainers insist on statically compiling everything. This is fine for small programs, and even large monolithic applications that are not expected to change very often.

But for the machine learning projects I work on, I might want to include a single algorithm from a fairly large library of algorithms. The amount of memory used is not trivial, I am talking about the difference between loading a single algorithm in 50 MB of compiled code for a dynamically loadable library, versus loading the entire 1.5 GB library of algorithms of statically linked code just to use that one algorithm. Then when distributing this code to a few dozen compute nodes, that 50 MB versus 1.5 GB is suddenly a very noticeable difference.

There are other problems with statically linking everything as well, for example, if you want your application to be written in a high-level language like Python, TypeScript, or Lisp, you might want to have a library of Rust code that you can dynamically load into the Python interpreter and establish foreign function bindings to the Rust APIs. But this is not possible with statically linked code.

And as I understand, it is a difficult technical problem to solve. Apparently, in order for Rust to optimize a program and guarantee type safety and performance, it needs the type information in the source code. This type information is not normally stored into the dynamically loadable libraries (the .so or .dll files), so if you dynamically load a library into a Rust program its type safety and performance guarantees go out the window. So the Rust compiler developers have chosen to make everything as statically compiled as possible.

This is why I don't see Rust replacing C any time soon. A language like Zig might have a better chance than Rust because it can produce dynamically loadable libraries that are fully ABI compatible with the libraries compiled by C compilers.

You can load Rust into Python just fine. In fact, several packages have started requiring a Rust compiler on platforms thst don't get prebuilt binaries. It's why I installed Rust on my phone.

The build files for Rust are bigger than you may expect, but they're not unreasonably big. Languages like Python and Java like to put their dependencies in system folders and cache folders outside of their project so you don't notice them as often, but I find the difference not that problematic. The binaries Rust generates are often huge but if you build in release mode rather than debug mode and strip the debug symbols, you can quickly remove hundreds of megabytes of "executable" data.

Rust can be told to export things in the C FFI, which is how Python bindings are generally accomplished (although you rarely deal with those because of all the helper crates).

Statically compiled code will also load into processes fine, they just take up more RAM than you may like. The OS normally deduplicates dynamically loaded libraries across running processes, but with statically compiled programs you only get the one blob (which itself then gets deduplicated, usually).

Rust can also load and access standard DLLs. The safety assertions do break, because these files are accessed through the C FFI which is marked unsafe automatically, but that doesn't need to be a problem.

There are downsides and upsides to static compilation, but it doesn't really affect glue languages like Python or Typescript. Early versions of Rust lacked the C FFI and there are still issues with Rust programs dynamically loading other Rust programs without going through the C FFI, but I don't think that's a common issue at all.

I don't see Rust replace all of C either, because I think Rust is a better replacement for C++ than for C. The C parts it does replace (parsers, drivers, GUIs, complex command line tools) weren't really things I would write in C in the first place. There are still cars where Rust just fails (it can't deal with running out of memory, for one) so languages like Zig will always have their place.

Is it not possible for Rust to optimize out unused functions as with C? That seems ...like a strange choice if so.

Is it not possible for Rust to optimize out unused functions as with C?

No Rust can do dead code elimination. And I just checked, Rust can do indeed do FFI bindings from other languages when you ask the compiler to produce dynamically linking libraries, but I am guessing it has the same problems as Haskell when it produces .so or .dll files. In Haskell, things like "monad transformers" depend pretty heavily on function inlining in order to achieve good performance.

So I am talking more about how Rust makes use of the type system to make decisions about when to inline functions which is pretty important when it comes to performance. You usually can't inline across module boundaries unless modules are all statically linked. So as I understand it, if you enable dynamic linking in your Rust program, you might see performance suffer a lot as compared to static linking, and this is why most Rust people (as I understand it) just make everything statically linked by default.

I am not sure that is quite right. I dont think rust support just enabling dynamic linking of its dependencies. It can talk to dynamically linked libraries - which is how FFI works. And you can compile rust crates to be dynamically linked. But when you are going down this route you are talking over the C ABI. This requires some effort on the code author to make their APIs exportable to C types and means you lose all safety when talking over the C ABI.

I also dont think that rust inlines across a crate boundary unless the function is marked as inline or LTO is enabled - inlining across crate boundaries is expensive and so only done when explicitly needed or asked for it. It is more that you lose features like generics and traits and other things that are not supported over the C API.

Do you need inlining if you just use fixed monad transformers?

Do you need inlining if you just use fixed monad transformers?

I am not sure what you mean by "fixed" monad transformers, if you mean writing your own newtype where the functor variable is the only type variable, essentially what you are doing is hand-inlining the monad transformer, and so no, if you inline by hand, then the compiler doesn't need to do it.

Haskell inlines all newtype definitions automatically, so if your monad transformer has all of the type variables bound (except for the functor variable, because that is a special case the Haskell compiler is specifically designed to handle) the compiler will usually reduce those to ordinary lambda expressions automatically, and lambda expressions usually optimize to the most efficient machine code.

The only time the compiler cannot reduce a newtype to an efficient lambda is if the non-functor variables, e.g. the state type variable or the exception type variable, are unbound. Those values could become anything at all at its call site, limited only by the constraints set by the type context. So the type context information, a lookup table of type class instances, must be associated with that lambda expression, and in order to do that, the compiler must create a closure around those values. Creating closures allocates values on the heap, and this is much, much slower than efficient lambda expressions, and no faster than allocating a data constructor as with Free Monads.

Alexis King did a presentation on it where she explains all of this extremely well, if you are interested: https://youtu.be/0jI-AlWEwYI

It is a bit long, but at 17:40 or so she starts talking about strategies for how monads and effects can be implemented in the GHC intermediate code, and compares Free Monads and effects to monad transformers. At 21:15 or so she begins to explain how newtype types can be optimized away completely, newtype constructors don't exist at all in the low-level code, they are a "zero-cost abstraction." On the other hand, data constructors (used for Free monads and effects) always allocate something on the heap which is an order of magnitude slower.

Then at around 27:45 she begins to show how newtypes with type variables cannot be inlined across module boundaries for the reason I explained above (type context tables associated with closures), and so monad transformers cannot be optimized across module boundaries.

Yep, I mean like newtype MyT m a = MyT (ReaderT MyEnv (StateT MyState m) a). But one can use ReaderT MyEnv (State MyState m) a directly as well.

I found the MTL style (tagless final) a bit problematic anyway, so I wanted to comment about this.

3 more...
3 more...

And the fucking MIT License

Yes, as much as I appreciate memory safety and rust in particular. I'm very worried by this pivot away from copyleft and GPL. Specially the rewriting in rust phenomenon of fundamental stuff. It's safer, yes, but they're all pretty much non GPL and it seems very risky to me. Make no mistake, the industry is riding this wave to move away from copyleft to permissive licenses.

I wish that people understood the importance of FSF and GNU

Well that is rather insidious. Crap. They probably understand the reasons for the GPL very well. Doesn't mean they support them.

I'm sure there's some community pull as well, because most of the rust ecosystem seems to be converged on MIT. But what despairs me is the wilful sidelining of GPL and everything GNU by some open source community members/corporate people. So yeah, you're probably right

You make it sound like a conspiracy. Just accept that some things are organically more popular, like MIT which is very easy to understand and use for normies. It's not perfect, but that's how it is

MIT is a terrible license that only got popular because of the popularity of the anti-open source movement in the last decade.

one could write books about what's wrong with the MIT license.

It could even theoretically be argued that MIT has in some ways allowed big tech companies to proliferate, by effectively allowing them to take open-source code, modify it, and then close it off in their proprietary software. What does this mean? It means that the work of countless dedicated open-source developers can be co-opted by companies that have done almost none of the work, reaping several billions of dollars, while the developers who actually did the work make no money. It's like opening your doors wide only to have someone come in, take your stuff, and sell it back to you.

In contrast, in licenses like the GPL, there's a requirement that if you use GPL-licensed code and modify it, your new code also has to be open-source under the GPL.

I love the free software ideals, but I think we've got a different understanding about what constitutes a good and a bad license. What many people seem to forget about software licenses is that there are these other countries besides America. They couldn't care less about whatever judges rule over there. A good license is a dumb simple license that anyone can enforce in court with ease. A bad license is a convoluted license that crumbles like a house of cards in court. I read the GPL. It's convoluted. It's an opaque terms of service agreement riddled with legal boilerplate disguised as software license. A poor execution of the ideals I hold. I only use the GPL as a formality to say that I support the free software ideals, but I have zero confidence in enforcing the GPL.

I'd like to correct you by saying that GPL is DEFINITELY enforceable in countries other than america. I can't say about every country (tho that will be the case with every license), but for instance it's definitely enforceable in europe. For example in Germany and France there have been a few lawsuits that the FSF helped carry out against immoral companies.

GPL Enforcement Cases - FSFE

If you're in Germany the Institute for Legal Questions on Free and Open Source Software is a law firm that literally works only on enforcing the GPL, FOSS licenses and other technological human rights that are being ignored by big tech.

If you want to be even more sure about European Enforcement you may want to checkout the EUPL v1.2 which is GPLv3 compatible.

In other countries, such as Japan, the GPL is also enforceable, so long as you treat it the same way as copyright, so you're willing to sue companies that you know are stealing from you (the FSF can help you if you can't afford it).

Russia and China don't care, but... it's Russia and China, that's not really news, is it? :)

EDIT: I will write a full article about the legal enforce-ability of FOSS licenses such as the GPL before the end of the year

Shame that we don't have a proper copyleft license tho? GPL, as nice as the intentions are, is a license so convoluted that I'm not sure whether it'd hold up in court in my country.

The idea is less bugs due to stricter rules when developing and compiling. You can understand that.

Then, also more access to build tools and high level programming without changing languages.

If you have no need for that, then just know others do and it's a great thing.

26 more...

Now imagine the new COSMIC desktop environment in Rust on Redox, that would be great

Probably inevitable considering Jeremy Soller is the lead dev on Redox while also currently contributing to COSMIC.

To be honest, I'm surprised COSMIC isn't the default, but it's likely due to display server stuff that isn't part of Redox (Wayland, etc.).

Why would that be great? It's so weird that people care this much about what language their OS is written in...

Rust is a memory-safe language. So in this case, it could result in more stable software.

I personally like rust, so I get excited when cool things are done with it because each one makes rust just that much bigger, which leads to it being made that much better.

I see projects like that as more of a statement that "rust can do it" than anything.

I want the newest, best software. Is that uncommon? Modern rewrites are often much better than their age-old counterparts since the tech got better over time, compare for example grep vs ripgrep, or find vs fd. The rewrites are much faster and user-friendlier

Memory corruption bugs account for a majority of cve's in the linux kernel

Iā€™ve used it in a VM just to mess around. Iā€™d like to install it on an old ThinkPad and try to compile some applications.

How long would it take to compile their Rust microkernel alone compared to a similar one done in C? There are many posts around the web complaining about Rust's long compile times, though thankfully rarely as slow as C++

I wouldn't say it's inappropriate as there is more and more rust making it into the native kernel. I'll definitely throw this on my Ventoy usb and see if I can get it to boot

Oh my god they rewrote Linux in Rust. Amazing.

Kinda. Redox uses a microkernel architecture and tries to keep only the most important functionality in ring 0 while they push everything else in userspace. It's great.

So they actually rewrote The Hurd in Rust.

Many lessons were learned from the Hurd that has impacted ALL microkernel kernel and userspace designs, but it is ancient by today's standards.

Its more accurate to say they were "rewriting" MINIX, but I don't like the word "rewriting" as all of these systems are unique in their own way.

Having some hardware mentioned on the site that is supported and ready for use could be helpful if someone wants to try it (say raspberry pi), There are probably people who are worried to will make their computer explode.

Slap a Firefox on top (and time travel to when ff is all rusted) and we'll be coming for ChromeOS. But will windows be completely rusted first? šŸ™ƒ

I think the main reason Firefox isn't on there is because redox os doesn't use Wayland and x11. Porting firefox would be a massive effort unfortunately.

If it weren't "written in rust" nobody would give a shit.

I would definitely give a shit even if it was whole OS written in JavaScript.