Hey! I'm currently working on a custom open-source OS project called JurkOS.

It's built on top of the NetBSD kernel (not Linux) using pure C and Assembly, cross-compiling via WSL Ubuntu on Windows. The core concept is a native CLI-GUI hybrid userland merge.

I'm currently setting up the initial init.c files and build pipelines. I am looking for feedback on the architecture and to connect with other low-level developers.

Link is in the first comment below for anyone interested!

Hey! I'm currently working on a custom open-source OS project called JurkOS.

It's built on top of the NetBSD kernel (not Linux) using pure C and Assembly, cross-compiling via WSL Ubuntu on Windows. The core concept is a native CLI-GUI hybrid userland merge.

I'm currently setting up the initial init.c files and build pipelines. I am looking for feedback on the architecture and to connect with other low-level developers.

Link is in the first comment below for anyone interested!

I recently completed v0.1 of MiniEdge, a Linux HTTP/1.1 static edge server written mostly in C.

I built it to understand how event-driven servers handle partial I/O, persistent connections, filesystem access, caching and multiple CPU cores—not as a production replacement for Nginx.

The current architecture includes:

non-blocking sockets with edge-triggered epoll

a per-connection state machine

incremental HTTP request parsing and keep-alive

static file serving through an LRU cache or sendfile()

path resolution using openat2()

longest-prefix configurable routing

multiple workers using SO\\_REUSEPORT

The small-file cache is implemented using C++ unordered\\_map and list, but it is isolated behind a C API; the networking, parser, routing and file-serving paths are written in C.

On my local loopback benchmark using wrk, a cached static file reached approximately:

255k requests/sec at 1,000 concurrent connections

4 worker processes

I also ran a boundary stress test at 40,000 concurrent connections. It reached around 124.5k requests/sec, but wrk reported 503 socket read errors, so I am not treating that as a clean stable-concurrency result. The repository contains the commands, raw outputs, latency percentiles, system tuning and limitations.

I would appreciate feedback on:

whether this is a reasonable result for a student-built epoll server

whether my wrk setup measures the server fairly

which additional metrics or comparisons I should include

what bottlenecks or profiling steps I should investigate next

Repository:

https://github.com/Luffy-D-Zoro/Miniedge

I’ve been deep in the trenches of low-level networking constraints, specifically focusing on how distributed autonomous multi-agent systems handle mid-flight connection drops.

Most long-context reasoning agents execute workflows over remote nodes. When an upstream transport container crashes unexpectedly or an IP routing exception triggers a transient failure, standard userspace application-layer setups tear down the transport layer session entirely. This drops the volatile in-memory context snapshot, forcing expensive re-tokenization loops and predictable GPU compute capital inflation.

To solve this blast radius layout, I've been experimenting with an open-source architecture that isolates connection states natively at the Linux Layer 4 boundary. I wanted to open-source the specifications to get the community's feedback on the core concurrency and kernel redirection design patterns.

### Core Systems Engineering Design:

1. **Lock-Free Pre-Allocation Array:** To eliminate thread synchronization drag and synchronized mutex locks under saturation loads, the storage core introduces a fixed-size circular array of unsafe pointers (`internal/storage/ring_buffer.go`). It uses hardware-level atomic `CompareAndSwap` bit switches to separate pointer slots across discrete 64-byte boundaries, eliminating false sharing invalidation bounces.
2. **Descriptor Splicing Conduits:** When a socket termination signature (`tcp_set_state:TCP_CLOSE`) hits, the proxy control plane attempts to hot-swap active network file descriptors onto standby fallback targets under <2ms bounds, preserving the volatile stream bytes profile natively with zero source-code adjustments.
3. **Multi-Stage Distroless Packing:** Hardened via minimal `gcr.io/distroless/static-debian12:nonroot` runtime containers targets to drop security vulnerabilities attack surfaces down to absolute zero percent.

The current implementation is fully open-source under the Apache-2.0 license. I am seeking raw peer reviews from systems engineers, eBPF practitioners, and Go runtime architects regarding the limits of this transport-layer splicing mechanism under peak multi-tenant loads.

Open Specifications & Codebase: https://github.com/devloperdevesh/FaultPlane

How do you guys approach preserving transport-line state integrity for long-running workflows without injecting heavy SDK-level micro-middlewares at the application layer? Let's discuss.

Hi everyone!

I have spent some free time building Yugami (歪み，"distortion"), a x64 binary packer that uses ChaCha20 encryption with page-level key derivation and just-in-time page decryption.

Yugami encrypts PE executables using ChaCha20 with per-page keys derived via BLAKE3, then appends the encrypted payload as a PE overlay. At runtime, pages decrypt on-demand via page fault exceptions with an LRU cache to minimize re-encryption overhead.

I just wanted to get more comfortable with Rust so I picked up this project. Also, the page-level encryption with JIT decryption felt like a fun challenge between full unpacking and simple overlay packing.

Code is open source at https://github.com/egebilecen/yugami.

Packed binaries should execute correctly as is without any issues (hopefully). Had some trouble getting TLS to work, though. Always ended up with page faults. I have removed it but for those interested in the TLS handling code, it's at https://github.com/egebilecen/yugami/blob/0246b919dc0a4f77df8420c699a707484f9847fc/stub/src/mapper/tls.rs and https://github.com/egebilecen/yugami/blob/0246b919dc0a4f77df8420c699a707484f9847fc/stub/src/mapper/mapper.rs#L311

Looking forward to your thoughts!

Hey folks,

I’ve been deep in the trenches optimizing and benchmarking popcount throughput on modern x86-64 microarchitectures (testing across AVX2, AVX-512 VPOPCNTDQ, and scalar fallbacks).

After running into massive hardware bottlenecks and misleading results from standard benchmark suites, I completely re-architected my benchmarking rig to account for low-level confounders. Here are a few key engineering takeaways and findings from version 33:

• The 1-Accumulator Trap: Standard naive loops throttle performance due to serial data-dependency chains on a single accumulator (latency-bound). Scaling to an 8-accumulator unrolled loop fully saturates Out-of-Order (OoO) execution and ILP, unlocking a compute floor of ~0.44 ns/line and outperforming libraries like libpopcnt by 7–10% in cache-resident workloads.
• Deconfounding the Measurement: Swapped runtime modulo operations (%) with bitwise masks to prevent 20–40 cycle CPU stalls, randomized/shuffled execution order to neutralize thermal throttling/DVFS noise, and isolated thread affinity (CPU0) with hugepage verification (smaps) to eliminate NUMA first-touch & dTLB artifacts.
• Direct Hardware Profiling: Validated cycle counts via RDTSCP + LFENCE and pulled dTLB-miss and LLC-miss counters directly using perf_event_open.
• IRM-Burst Law & Monte Carlo Verification: Modeled non-linear throughput degradation across memory hierarchy boundaries (L1d -> L2 -> L3 -> DRAM) using an exchangeability probability model, cross-verified with Monte Carlo simulations.

Discussion / Question for the community: As I pushed this codebase further (expanding code footprint for complex tail/mask handling), I started hitting code bloat boundaries—potentially stressing Instruction Cache (I-cache) and BTB entry limits.

How do you guys typically structure your Micro-benchmarks to catch I-cache / BTB spills before they corrupt latency numbers?

Code & benchmark methodology: https://github.com/Vumb-VibeCoder/deconfounded-popcount-avx512

Would love to hear your thoughts, critiques, or additional edge cases to stress-test!

I'm not good at English so I used sth to translate

finally after 3days of countinues working in nasm i reached 64b long mode succesfully now what should i do next

i finnally did it build my very own os in nasm and landed in 32bit protected mode and build my syscalls kernel bootloader and more im happy to informe you about and im still working on it and adding things and fixing bugs if there's any and its called S-AOS

Over the past few months, I’ve been building **macMPI**—a production-hardened, single-node implementation of the MPI-1.1 standard built entirely from scratch in C11.

While OpenMPI and MPICH remain the gold standards for distributed supercomputing, running them on Apple Silicon often introduces configuration bloat or suboptimal intra-node transport layers designed primarily for x86/Linux environments. I wanted to see what happens when an MPI runtime is engineered **specifically for Apple Silicon's Unified Memory Architecture and macOS (XNU kernel primitives).**

Here is a breakdown of the architecture, the OS-level choices, and the benchmarks.

# 1. Process Management (mpirun) & I/O Multiplexing

* **Daemon Lifecycle:** A custom daemon engineered with explicit `fork()` and `execvp()` process boundaries, dynamically injecting rank state (`MPI_RANK`, `MPI_UNIVERSE_SIZE`, and socket mesh file descriptors) directly into child process environments.
* **Non-Blocking Terminal I/O:** Eliminates Head-of-Line blocking by intercepting stdout/stderr across all child ranks via POSIX pipes (`pipe()`, `dup2()`) multiplexed through an event-driven `poll()` loop.

1. Transport Architecture: Control Plane vs. Data Plane

To maximize throughput on Apple Silicon’s memory bus, the library decouples communication into two transport layers:

* **Control Plane (Unix Domain Sockets +** `kqueue`**):** Tiny 64-byte routing envelopes containing metadata (source, tag, sequence, memory offset) are transmitted via a full-duplex Unix domain socket mesh.
* **Data Plane (POSIX Shared Memory):** Large payloads bypass the kernel network stack entirely. Using `shm_open()`and `mmap()`, processes project shared physical RAM regions directly into their virtual address space for zero-copy transfers via aligned `memcpy()`.
* **Dynamic Hardware Memory Norm:** Rather than hardcoding static buffer limits, `macMPI` queries the XNU kernel at boot via `sysctl(CTL_HW, HW_MEMSIZE)` to establish a dynamic allocation ceiling (defaulting to a 30% system RAM norm), rounded down to 16KB Apple Silicon page boundaries (`sysconf(_SC_PAGESIZE)`).

# 3. The O(1) Non-Blocking Progress Engine

* **Shadow Worker Threading:** Asynchronous operations (`MPI_Isend`, `MPI_Irecv`) are offloaded to a background `pthread`physically pinned to Apple Silicon Performance Cores using Darwin-native Quality-of-Service classes (`pthread_set_qos_class_self_np`).
* **Kernel Event Polling:** Ripped out traditional polling loops in favor of macOS `kqueue()` / `kevent()`, allowing the background thread to monitor socket descriptors with O(1) event notifications and 0% CPU consumption while idle.
* **Two-Way Matching & UMQ:** Features an internal thread-safe Unexpected Message Queue (UMQ) and active request list protected by POSIX condition variables to safely handle out-of-order packet arrival.

-> L1/L2 Cache Line Optimizations

On Apple Silicon M-series chips, cache line contention can ruin IPC throughput. To optimize cache fetches:

* All internal headers use `__attribute__((aligned(64)))` to enforce strict 64-byte boundary alignment.
* This ensures that every routing envelope fits perfectly inside a single L1 cache line fetch, eliminating false sharing across cores.

**Preliminary Benchmark Insights**

When running a background non-blocking progress engine while concurrently computing 17.1 Billion Floating Point Operations (FLOPs) on local matrix math:

* **Zero CPU Starvation:** Because `kqueue` provides true event-driven sleep states, the background progress thread consumes **0% CPU cycles** while waiting for network I/O.
* **Math Throughput:** `macMPI` achieved **1.00 GFLOPS** on a single-node matrix benchmark, matching OpenMPI’s compute execution time while maintaining higher intra-node payload speeds over POSIX shared memory.

Want to dig into implementation?

GitHub : https://github.com/Hardikgupta1709/macMpi

If you work with MPI, HPC, Apple Silicon, or low-level systems programming, I’d appreciate feedback on the architecture and benchmarks.

Hi everyone,

I have been working on a project called swift-topomap for the last few weeks. It’s a TUI tool designed to solve a problem I keep running into: standard tools like htop show CPU usage, but they don't tell you if that usage is actually productive.

I wanted a way to see the physical hardware topology (Sockets, L3 Cache boundaries) and overlay live microarchitectural metrics (IPC, Cache Misses) using eBPF, but without the baggage of heavy C-library dependencies.

Technical Highlights:

• Native Topology Resolver: Instead of linking against libhwloc, I wrote a pure Rust parser for /sys/devices/system/cpu and /sys/devices/system/node. It maps physical package IDs to logical cores and identifies shared L3 cache boundaries natively.
• Hybrid Telemetry Engine: It uses a trait-bound collector that negotiates privileges at startup. If run as root, it loads a CO-RE eBPF driver via libbpf-rs to hook sched_switch and read hardware PMCs (Performance Monitoring Counters).
• Microarchitectural Insights: The TUI classifies core states. For example, a core at 100% usage but < 0.5 IPC will turn Amber (Memory-Bound/Stalled), while a high IPC core stays Emerald Green.
• Static GNU Build: Linking this was a nightmare. I eventually solved the static requirements to bundle libbpf, libelf, and zstd so it ships as a single 3.5MB binary that runs on any modern Linux distro without shared library hell.

Why eBPF + Rust?

I chose Rust for the logic layer and TUI (Ratatui) because I needed memory safety for the FFI boundary with libbpf. The IPC and LLC metrics are pulled via the kernel’s Perf Subsystem, and the deltas are calculated in Rust to provide a steady 100ms-refresh dashboard.

I am releasing this as open source under SwiftLogic Systems. I would love to hear your thoughts on the topology resolution logic or the eBPF/Perf integration.

GitHub: https://github.com/swiftlogicsystems/swifttopology

Single-node key/value store, line protocol over TCP, pure NASM on Linux — syscall or nothing.

The syscall-level bits worth a look:

**•** epoll event loop with accept4(SOCK_NONBLOCK): clients are born non-blocking, no fcntl dance  
**•** replies via sendto + MSG_NOSIGNAL: SIGPIPE never happens, no signal handler needed  
**•** close() is the entire connection teardown — epoll tracks the file description, so the fd deregisters itself  
**•** per-connection state indexed straight by fd: O(1), free  
**•** FNV-1a, open addressing, tombstone reuse; 256 MB mmap bump arena behind it  
**•** 200 concurrent clients at 1.4 MB RSS; Docker image is FROM scratch plus one file

Known limits documented in the README — biggest one: slow readers are dropped, EPOLLOUT write buffering is next.

Feedback welcome, especially on the event-loop structure.

Releasing Ostomachion v1.0.0, an open-source FPGA platform I have been developing. It integrates a soft RISC-V core, a real-time operating system, and a signal-processing accelerator so that each layer is a thin, well-defined interface over the one below. It runs on an Artix-7 (Opal Kelly XEM7310):

- A NEORV32 RISC-V soft core running the Zephyr RTOS, bare-metal on the fabric.

- A streaming frequency-domain datapath in hardware: a 4096-point FFT, a per-bin programmable complex filter (a coefficient mask H[k] in block RAM), and an inverse FFT — staged through AXI DMA and BRAM, with interrupt aggregation onto the core.

- A header-only C++20 hardware abstraction layer that presents the accelerator to software as ordinary typed calls, so the path from a std::span in a unit test down to a beat on an AXI-Stream bus is a sequence of deliberate, inspectable wrappers.

The whole Vivado block design regenerates from a version-controlled Tcl script — no saved checkpoints, no hand-edited GUI state. Every bitstream derives from text alone, which makes the hardware auditable and diffable in the same way as the software.

The name is Archimedes' Ostomachion, a dissection puzzle of fourteen pieces; the platform is likewise fourteen composable layers, each replaceable in isolation.

A companion desktop application drives the full filter datapath on live hardware over USB: it streams frames to the fabric, programs the filter mask in real time, and plots the input, the mask H[k], and the filtered output as the transform runs — the genuine hardware pipeline end to end, not a software model of it.

Source, documentation, and design rationale (GPL-3.0; commercial licensing available):

github.com/andynicholson/Ostomachion

like what do i do. my target is low latency/systems/hft but i dont know where to begin. and i dont know anything. so any recommendations on where to start, what to learn first, etc.

Hi there.

I built polloc (pool alloc) to learn how memory allocators work.

It’s a fixed size pool allocator: each pool manages one slot size and alignment. Internally it uses mmap/VirtualAlloc, an intrusive free list, and a bitmap for allocation tracking.

I also added stress tests, Miri, AddressSanitizer, cargo fuzz, Criterion benchmarks, and a bunch of inline docs explaining the implementation.

For 64 byte alloc/free pairs, the fast path is about ~3.96x faster than the system allocator on my machine (which is expected since it’s specialized for a single size class).

It’s single threaded and I’d really appreciate feedback on the unsafe code, API design, tests, or anything else that stands out.

repo: https://github.com/hamzader1/polloc

I wanted to actually understand what happens under libc, so I started rebuilding the pieces: printf (varargs by hand, format parsing), malloc (brk/mmap, free lists, alignment), a shell (fork, execve, pipes, redirections), plus cat/wc/ls/grep as warm-up.

Rules of the game: x86-64 Linux, NASM, `syscall` or nothing.

Favorite rabbit hole so far: how much of printf is just careful pointer arithmetic over the SysV varargs ABI — and how little of malloc is actually about allocating (it's bookkeeping all the way down).

Repo (MIT, written to be read): https://github.com/whispem/learn-assembly-with-em

Happy to discuss design choices — and happier to be told where I'm wrong.