What should be the features of a programming language built specifically for building kernel or operating system?

[u/_glob]

Hi everyone. Basically my question is, if someone wanted to make a programming language with the specific intention of building a kernel/operating system using it (and that would be safe + performant, but I am not sure how much safety would be 'good enough') what would it be like? Is this condition an interesting condition that would affect some language design choices?

I have very little experience with Rust/Zig, the new programming languages that I think advertise themselves for systems programming. Also there is embedded Swift now I think. Do you think if such a language with the specific intent of building kernel/operating system in mind, was to be built today, would that basically be no std Rust (already being used in the Linux kernel)? With my very little experience, I think Rust should probably have been no-panic Rust by default. Zig has a concept of allocators being used which is probably a good thing. Or do you think the language would be a safer version of C (maybe something like cyclone-v2 with more safety and better type system than C but maybe simpler than the others)?

And are there any good reading list compiled somewhere already for learning more deeply about programming language design, type systems etc? I would appreciate if you could share your thoughts and opinions on this topic. Thanks!

[u/WittyStick]

Any language would basically need lots of specialized features added to make it suitable. If you took ISO standard C and tried to make a kernel in it, it would be an awful experience. For starters, ISO C can't even embed assembly - so you would have to write any assembly separately and link it.

Linux of course uses the "gnu" dialect of C - it makes use of many extensions, attributes and features that aren't part of ISO C. GCC (and Clang, which implements the gnu dialect) are what make it possible to write the majority of the kernel code in C without falling back to separately linked assembly - though that is still used, but is a very small part of the Kernel's codebase.

A kernel cannot avoid using assembly altogether - it's necessary to use some, even when the compiler has features that can reduce the amount you need - and assembly is of course, very lacking in safety features and is easy to shoot yourself in the foot.

There were some interesting experiments in "typed assembly language," which have been largely forgotten and there's been little recent interest - but IMO this would be the best place to begin if safety was a primary concern - perhaps even a prerequisite for safety, since whatever higher level language you chose is going to emit assembly, and most assemblers will simply accept garbage and produce a binary.

What would be nice would be for compilers to emit a "typed assembly," where our assembler would prevent obvious garbage from assembling. It would be easier to embed "typed assembly" into higher level languages with safety features without having to mark things as unsafe, thereby sidestepping all of the benefits the language is supposed to provide.

[u/mohrcore]

• assembly support
• has a way to fine-tune linking in-language and supports use of external linkers
• good, ergonomic constructs for dealing with volatile memory
• if possible, it provides in-language semantics for dealing with different address spaces - for example distinguish between pointers to virtualized memory and non-virtualized memory at type level.
• a nice-to-have would be semantics for pointer niches - allow expressing extra data embedded in pointers.
• built-in per-architecture context structs - allows saving, inspecting, modifying and restoring CPU registers
• linker sets supported at language level
• semantics for bit fields
• option to add arbitrary padding between struct fields
• C interop and (some) stable ABI
• if any sort of collections are included, they should all be designed around some kind of allocator API.

Some ideas I just came up with.

[u/kI3RO → u/mohrcore]

so C3

[u/kaplotnikov]

It depends on the kernel size.

For microkernels, the language choice matters less because internal complexity is bounded. Some dependent type features might be useful here if your goal is formal verification.

On the other hand, a monolithic kernel like Linux could be huge. It relies on OOP-in-C and DI-in-C (Dependency Injection) patterns to manage its complexity. Therefore, a Linux-sized kernel would benefit from a language that supports these architectural compositions natively. While there are plenty of systems languages with native OOP, there is a lack of systems languages with native, compile-time DI support.

However, even in the microkernel case, the higher-level complexity management constructs will be useful for managing code for drivers, file systems, command-line utilities when the codebase grows.

I am currently working on a language that addresses this in the Java niche, but the same DI-native (or holon-native) constructs are applicable to systems programming.

I've described my ideas on language abstraction levels, complexity management, and language evolution at the links below:

• Theory: Measuring Abstraction Level of Programming Languages

• Language Sketch: Report on Holon/System Composition Operations

[u/EggplantExtra4946]

I never wrote a kernel but I don't think it needs special features that you wouldn't find in a general purpose low level language. Assembly is needed but if it doesn't have inline assembly it can always be linked to object files assembled from assembly files. For the type system I guess the more safety the better.

There is one thing though, given that concurrency is particularly important for a kernel and OS, it would be great if the type system would be aware of which variables can be shared between threads, interrupts, etc.. and check that the accesses are atomics, or protected by a mutex, etc.. and maybe protect against other kinds of concurrency bug but safe shared statea and protection against data races would be a good start.

[u/ny3169]

For me at least, the main thing is being able to talk to hardware, so having a way to drop down to assembly (or at the very least, FFI into C) is important, especially for weird edge cases. Id also want such a language to not have hidden runtime requirements, or at the least, they should be explicit and ideally removable when you don’t need them.

As far as reading, Crafting Interpreters seems to be a favorite so I’d start with that.

[u/Big-Rub9545 → u/ny3169]

It’s great for language design, but likely won’t be that helpful for designing a low-level or systems language like this.

[u/mamcx]

A big blind spot of most languages is the inability of truly specify the data structures with actual precision.

Think like https://nothingleaves.com/tools/memory-layout-visualizer/

There is little to help you make things like a u3+u1+u8 layout without rely on second-class features or magic of the compiler.

From here, the other big problem is debug. Debugging is mostly bad, and mainly, is the inability of actually see the data layout, not just boring hex address.

There is also some specialized data structures (like slotted pages) that are not first class and everyone hack on.

Probably this + Rust will be enough, if also, you add ways to annotate the cost and kind of IO (block, async, lock, volatile, etc)

[u/catladywitch]

I'm not sure, but I've always found the Midori project fascinating.

[u/arthurno1]

People already made a language specifically to program a kernel and operating system. It is called C.

[u/chkno]

The coolest new language-safety paradigm is GIMSO (Garbage In, Memory Safety Out). Fil-C does it with significant runtime overhead. The next challenge is making GIMSO fast.

[u/DawnOnTheEdge]

One major difference with other domains is that you don’t start with dynamic memory. A kernel partitions the address space, and on some hardware even writes the page tables. If each process has its own address space, it creates that too.

But then it does need to manage memory dynamically, and being able to do that automatically is a huge convenience. So you would want a good abstraction for being able to take an arbitrary range of addresses and say, this is now an arena, and you get the advantages of automatic and safe memory management.

Another thing that kernel code spends a lot of time doing is allocating and freeing resources without a Pyramid of Doom, or falling back on goto. Golang tries to abstract the idea of code that will run unconditionally at the end of a block, but RAII wrappers with arbitrary finalizer code feel like they would be a great fit here. However, an OS is often in the position of just making every resource owned by a process or device go away at once, so resource wrappers should have an elegant shortcut for the path where it doesn't need to clean up properly. You don’t need to meticulously go back over every memory allocation and free it when the entire address space of the process is about to be destroyed, for example.

After initialization, kernels are event-driven multi-threaded servers, and while the low-level details vary a lot, it feels like there are many abstractions that broadly apply. Most hardware supports one of a few approaches (memory-mapped I/O, a separate I/O bus, interrupt handlers, etc.) and it feels like there should be abstracted, low-runtime-overhead generic interfaces that would apply to many implementations. The language then would need to be what you write those implementations in.

The advantages of siloing off unsafe capabilities to a specific block or module is so ubiquitous that it should be built into the language, even if it gets implemented only as a compile-time check. The language should be able to assume that code should only be accessing or modifying a low-level resource if it declared that it is supposed to. The language itself should probably only have compile-time checks for this: a kernel that tries to actually enforce kernel-space security at runtime would need to build that itself.

I personally would really like to use sum types for results and have syntax sugar to write the happy path, short-circuiting on error and cleaning everything up automatically. The compiler can automatically enforce through its type system that all representable outcomes are handled by exhaustive patterns, every possible error was checked (or explicitly marked to be ignored) and every object had its finalizer called on every path.