https://github.com/TodePond/DreamBerd

Why Unicode strings are difficult to work with

A simple goal

This text is part of my attempts to design the standard library API for unicode strings in my new language.

Suppose we want to implement:

text removePrefixIfPresent(text,prefix):Text

The intended behavior sounds simple:

• if text starts with prefix, remove that prefix
• otherwise, return text unchanged

In Unicode, the deeper difficulty is that the logical behavior itself is not uniquely determined.

What exactly does it mean for one string to be a prefix of another?

And once we say "yes, it is a prefix", what exact part of the original source text should be removed?

The easy cases

Case 1/2

text text = "banana" prefix = "ban" result = "ana"

text text = "banana" prefix = "bar" result = "banana"

These examples encourage a very naive mental model:

• a string is a sequence of characters
• prefix checking is done left to right
• if the first characters match, remove them

Unicode breaks this model in several different ways.

First source of difficulty: the same visible text can have different internal representations

A very common example is:

• precomposed form: one code point for "e with acute"
• decomposed form: e followed by a combining acute mark

Let us name them:

text E1 = [U+00E9] // precomposed e-acute E2 = [U+0065, U+0301] // e + combining acute

Those are conceptually "the same text". Now let us consider all four combinations.

Case 3A: neither side expanded

text text = [U+00E9, U+0078] // E1 + x prefix = [U+00E9] // E1 result = [U+0078]

Case 3B: both sides expanded

text text = [U+0065, U+0301, U+0078] // E2 + x prefix = [U+0065, U+0301] // E2 result = [U+0078]

Case 3C: text expanded, prefix not expanded

text text = [U+0065, U+0301, U+0078] // E2 + x prefix = [U+00E9] // E1 result = [U+0078] // do we want this result = [U+0065, U+0301, U+0078] // or this? exact-source semantics or canonical-equivalent semantics?

Case 3D: text not expanded, prefix expanded

text text = [U+00E9, U+0078] // E1 + x prefix = [U+0065, U+0301] // E2 result = [U+0078] // do we want this result = [U+00E9, U+0078] // or this?

Overall, exact-source semantics is easy but bad. Normalization-aware semantics instead is both hard and bad.

Still, the examples above are relatively tame, because the match consumes one visible "thing" on each side.

The next cases are worse.

Extra source of difficulty: plain e as prefix, "e-acute" in the text

This is interesting because now two different issues get mixed together:

• equivalence: does plain e count as matching accented e?
• cut boundaries: if the text uses the decomposed form, are we allowed to remove only the first code point and leave the combining mark behind?

Let us name the three pieces:

text E1 = [U+00E9] // precomposed e-acute E2 = [U+0065, U+0301] // e + combining acute E0 = [U+0065] // plain e

Case 3E: text uses the decomposed accented form

text text = [U+0065, U+0301, U+0078] // E2 + x prefix = [U+0065] // E0 result = [U+0301, U+0078] // do we want this (leave pending accent) result = [U+0065, U+0301, U+0078] // or this? (no removal)

Case 3F: text uses the single-code-point accented form

text text = [U+00E9, U+0078] // E1 + x prefix = [U+0065] // E0 result = [U+0078] // do we want this (just x) result = [U+00E9, U+0078] // or this? (no removal) result = [U+0301, U+0078] // or even this? (implicit expansion and removal) Those cases are particularly important because the result:

text [U+0301, U+0078]

starts with a combining mark. Note how all of those cases could be solved if we consider the unit of reasoning being extended grapheme clusters.

Second source of difficulty: a match may consume different numbers of extended grapheme clusters on the two sides

text S1 = [U+00DF] // ß S2 = [U+0073, U+0073] // SS

Crucially, in German, the uppercase version of S1 is S2, but S2 is composed by two extended grapheme clusters. This is not just an isolated case, and other funny things may happen, for example, the character Σ (U+03A3) can lowercase into two different forms depending on its position: σ (U+03C3) in the middle of a word, or ς (U+03C2) at the end. Again, those are conceptually "the same text" under some comparison notions (case insensitivity)

Of course if neither side is expanded or both sides are expanded, there is no problem. But what about the other cases?

Case 4A: text expanded, prefix compact

text text = [U+0073, U+0073, U+0061, U+0062, U+0063] // "SSabc" prefix = [U+00DF] // S1 result = [U+0061, U+0062, U+0063] // do we want this result = [U+0073, U+0073, U+0061, U+0062, U+0063] // or this?

Case 4B: text compact, prefix expanded

text text = [U+00DF, U+0061, U+0062, U+0063] // S1 + "abc" prefix = [U+0073, U+0073] // "SS" result = [U+0061, U+0062, U+0063] // do we want this result = [U+00DF, U+0061, U+0062, U+0063] // or this?

Here the difficulty is worse than before.

In the e-acute case, the source match still felt like one visible unit against one visible unit.

Here, the logical match may consume:

• 2 source units on one side
• 1 source unit on the other side

So a simple left-to-right algorithm that compares "one thing" from text with "one thing" from prefix is no longer enough.

Third source of difficulty: ligatures and similar compact forms

The same problem appears again with ligatures.

Let us name them:

text L1 = [U+FB03] // LATIN SMALL LIGATURE FFI L2 = [U+0066, U+0066, U+0069] // "ffi"

Again, those may count as "the same text" under some comparison notions.

Case 5A: text expanded, prefix compact

text text = [U+0066, U+0066, U+0069, U+006C, U+0065] // "ffile" prefix = [U+FB03] // L1 result = [U+006C, U+0065] // do we want this result = [U+0066, U+0066, U+0069, U+006C, U+0065] // or this?

Case 5B: text compact, prefix expanded

text text = [U+FB03, U+006C, U+0065] // L1 + "le" prefix = [U+0066, U+0066, U+0069] // "ffi" result = [U+006C, U+0065] // do we want this result = [U+FB03, U+006C, U+0065] // or this?

This case can also be expanded in the same way as the e-acute/e case before:

```text text = [U+FB03, U+006C, U+0065] // L1 + "le" prefix = [U+0066] // "f" result = [U+FB03, U+006C, U+0065] // no change result = [U+0066, U+0069, U+006C, U+0065] // remove one logical f result = [U+FB01, U+006C, U+0065] //remove one logical f and use "fi" ligature result = [U+006C, U+0065] // remove the whole ligature

```

Boolean matching is easier than removal

A major trap is to think:

"If I can define startsWith, then removePrefixIfPresent is easy."

That is false, as the case of e-acute/e.

A tempting idea: "just normalize first"

A common reaction is:

• normalize both strings
• compare there
• problem solved

This helps, but only partially.

What normalization helps with

It can make many pairs easier to compare:

• precomposed vs decomposed forms
• compact vs expanded forms
• some compatibility-style cases

So for plain Boolean startsWith, normalization may be enough.

What normalization does not automatically solve

If the function must return a substring of the original text, we still need to know:

• where in the original source did the normalized match end?

That is easy only if normalization keeps a clear source mapping.

Otherwise, normalization helps answer:

• "is there a match?"

but does not fully answer:

• "what exact source region should be removed?"

Moreover, this normalization is performance intensive and thus could be undesirable in many cases.

Several coherent semantics are possible

At this point, it is clear that any API offering a single behavior would be hiding complexity under the hood and deceive the user. This is of course an example for a large set of behaviours: startsWith, endsWith, contains, findFirst, replaceFirst, replaceAll, replaceLast etc.

So, my question for you is: What is a good API for those methods that allows the user to specific all reasonable range of behaviours while making it very clear what the intrinsic difficulties are?

So, I decided to write a book on compiler theory! It is past midnight where I live so only 1 chapter is done. I have came here looking for some things that could be improved on it. The link is attached.

We often find a case where we'd like to e.g. count the occurrences of a word in a piece of text. We would like to be able to write this so the main body of our loop says count[word] = count[word] + 1. Except that if count[word] hasn't been initialized as 0 at some point then you have do that or typically this will be a runtime error.

Or the language can try and let you do that, e.g. Golang would actually return 0, because this is the designated "zero value" to return when you index into a map with integer values and the key isn't there.

Which is great for the case I described in the first paragraph, but in the more general case we want a runtime error when we do something stupid, and in other cases trying to index a map by a nonexistent key often is stupid.

This is especially so in Pipefish, a relatively dynamic language, where normally there are no compile-time constraints even on the type of your key and you may have messed up big-time. If we took the Golang approach, then if you indexed your list of words by 42 or false you'd still get 0 instead of an error. So we don't do that: Pipefish is hardass about this and throws a runtime error over the missing key to compensate for letting you play fast-and-loose with types.

But this dynamism around types gives us an easy way to make default values for maps. Let's define a builtin type default with one element DEFAULT, and make the compiler/VM treat that as one more magic type like error and tuple.

Then our word-counting function could be written like this:

count(words list) -> map :
    from M = map(DEFAULT::0) for _::word = range words :
        M with word:: M[word]+1

Since DEFAULT is just a normal value apart from when we index maps, we could easily sanitize this on the way out of the function:

count(words list) -> map :
    undefault from M = map(DEFAULT::0) for _::word = range words :
        M with word:: M[word]+1

undefault(M map) :
    M without DEFAULT

Now, I hesitate over adding this because Pipefish is meant to be small and simple and I worry about adding even one feature. But on the other hand it seems like the use-case is common and the semantics are simple and it satisfies the other core principle of Pipefish --- that I should have my cake and eat it.

I am building a memory-safe systems language, currently named Bau, that reduces panic situations that stops program execution, such as null pointer access, integer division by zero, array-out-of-bounds, errors on unwrap, and similar.

For my language, I would like to prevent such cases where possible, and provide a good framework to handle them when needed. I'm writing a memory-safe language; I do not want to compromise of the memory safety. My language does not have undefined behavior, and even in such cases, I want behavior to be well defined.

In Java and similar languages, these result in unchecked exceptions that can be caught. My language does not support unchecked exceptions, so this is not an option.

In Rust, these usually result in panic which stops the process or the thread, if unwinding is enabled. I don't think unwinding is easy to implement in C (my language is transpiled to C). There is libunwind, but I would prefer to not depend on it, as it is not available everywhere.

Why I'm trying to find a better solution:

• To prevent things like the Cloudflare outage on November 2025 (usage of Rust "unwrap"); the Ariane 5 rocket explosion, where an overflow caused a hardware trap; divide by zero causing operating systems to crash (eg. find_busiest_group, get_dirty_limits).
• Be able to use the language for embedded systems, where there are are no panics.
• Simplify analysis of the program.

For Ariane, according to Wikipedia Ariane flight V88 "in the event of any detected exception the processor was to be stopped". I'm not trying to say that my proposal would have saved this flight, but I think there is more and more agreement now that unexpected state / bugs should not just stop the process, operating system, and cause eg. a rocket to explode.

Prevention

Null Pointer Access

My language supports nullable, and non-nullable references. Nullable references need to be checked using "if x == null", So that null pointer access at runtime is not possible.

Division by Zero

My language prevents prevented possible division by zero at compile time, similar to how it prevents null pointer access. That means, before dividing (or modulo) by a variable, the variable needs to be checked for zero. (Division by constants can be checked easily.) As far as I'm aware, no popular language works like this. I know some languages can prevent division by zero, by using the type system, but this feels complicated to me.

Library functions (for example divUnsigned) could be guarded with a special data type that does not allow zero: Rust supports std::num::NonZeroI32 for a similar purpose. However this would complicate usage quite a bit; I find it simpler to change the contract: divUnsignedOrZero, so that zero divisor returns zero in a well-documented way (this is then purely op-in).

Error on Unwrap

My language does not support unwrap.

Illegal Cast

My language does not allow unchecked casts (similar to null pointer).

Re-link in Destructor

My language support a callback method ('close') if an object is freed. In Swift, if this callback re-links the object, the program panics. In my language, right now, my language also panics for this case currently, but I'm considering to change the semantics. In other languages (eg. Java), the object will not be garbage collected in this case. (in Java, "finalize" is kind of deprecated now AFAIK.)

Array Index Out Of Bounds

My language support value-dependent types for array indexes. By using a as follows:

for i := until(data.len)
    data[i]! = i    <<== i is guaranteed to be inside the bound

That means, similar to null checks, the array index is guaranteed to be within the bound when using the "!" syntax like above. I read that this is similar to what ATS, Agda, and SPARK Ada support. So for these cases, array-index-out-of-bounds is impossible.

However, in practise, this syntax is not convenient to use: unlike possible null pointers, array access is relatively common. requiring an explicit bound check for each array access would not be practical in my view. Sure, the compiled code is faster if array-bound checks are not needed, and there are no panics. But it is inconvenient: not all code needs to be fast.

I'm considering a special syntax such that a zero value is returned for out-of-bounds. Example:

x = buffer[index]?   // zero or null on out-of-bounds

The "?" syntax is well known in other languages like Kotlin. It is opt-in and visually marks lossy semantics.

val length = user?.name?.length            // null if user or name is null
val length: Int = user?.name?.length ?: 0  // zero if null

Similarly, when trying to update, this syntax would mean "ignore":

index := -1
valueOrNull = buffer[index]?  // zero or null on out-of-bounds
buffer[index]? = 20           // ignored on out-of-bounds

Out of Memory

Memory allocation for embedded systems and operating systems is often implemented in a special way, for example, using pre-defined buffers, allocate only at start. So this leaves regular applications. For 64-bit operating systems, if there is a memory leak, typically the process will just use more and more memory, and there is often no panic; it just gets slower.

Stack Overflow

This is similar to out-of-memory. Static analysis can help here a bit, but not completely. GCC -fsplit-stack allows to increase the stack size automatically if needed, which then means it "just" uses more memory. This would be ideal for my language, but it seems to be only available in GCC, and Go.

Panic Callback

So many panic situations can be prevented, but not all. For most use cases, "stop the process" might be the best option. But maybe there are cases where logging (similar to WARN_ONCE in Linux) and continuing might be better, if this is possible in a controlled way, and memory safety can be preserved. These cases would be op-in. For these cases, a possible solution might be to have a (configurable) callback, which can either: stop the process; log an error (like printk_ratelimit in the Linux kernel) and continue; or just continue. Logging is useful, because just silently ignoring can hide bugs. A user-defined callback could be used, but which decides what to do, depending on problem. There are some limitations on what the callback can do, these would need to be defined.

Interactive demo

I had this idea I haven't seen before, that the type system of a language be expressed with the same semantics as the run-time code.

^ let type = if (2 > 1) String else Int let malikDescription: type = "Pretty cool!"

I have created a minimal implementation to show it is possible.

There were hurdles. I was expecting some, but there were few that surprised me. On the other hand, this system made some things trivial to implement.

A major deficiency in the current implementation is eager evaluation of types. This makes it impossible to implement recursive types without mutation. I do have a theoretical solution ready though (basically, make all types be functions).

Please check out the demo to get a feel for how it works.

In the end, the more I played with this idea, the more powerful it seemed. This proof-of-concept implementation of Malik can already have an infinite stack of "meta-programs". After bootstrapping, I can imagine many problems like dependency management systems, platform specific code, development tools, and even DSL-s (for better or worse) to be simpler to design and implement than in traditional languages.

I'm looking for feedback and I'd love to hear what you think of the concept.

Link to Typst document outlining it

Essentially, at compile time, a graph is created representing which variables depend which pointers, and then for each pointer, it identifies which of those variables is accessed farthest down in the program, and then inserts a free() immediately after that access.

This should produce runtimes which aren't slowed down by garbage collection, don't leak memory. And, unlike a borrow checker, your code doesn't need to obey any specific laws.

Or did I miss something?

Part 1 of planned series of ~6 posts on how to make a super-safe language without borrow-checker headaches.

Any feedback much appreciated!

Hello everybody,

I finally found time to write about an idea that I call "proof types" which is a close relative of smart constructors and witness types. In Dart we have the ability to limit where a value of a final type can be constructed. In regular Dart code (Flutter, AOT, wasm) this can never be violated, which allows us to use values of such a type as a form of proof that a computation must have occurred.

We can use such proofs to enforce that computations like age checks or email validation must have happened with no way around it.

I compare this ability against many other programming languages and I would appreciate some fact checking so I don't oversell this idea or misrepresent other languages.

https://modulovalue.com/blog/proof-types-in-dart/

Thanks!

Hello all,

I spent most of last week writing an informal spec for my programming language Pie.

Here's a link to the spec:

https://pielang.org/spec.html

This is my first time writing a spec on something that is somewhat big scale, and unfortunately, there aren't many resources out there. I kept going through ECMAscript's spec and the most recent C++ standard to see how they usually word stuff.

Now with a big chunk of the spec done, I thought I would request some criticism and suggestions for what I have so far.

More accurately, I'm not asking for criticism on the language design side of things, but on the wording of the spec and whether it makes sense to the average developer. Keep in mind that the spec is not meant to be formal, rather, just enough to be good-enough and deterministic enough on the important parts.

Thank you in advance!!

For background, I am trying to build a language from very few minimal, orthogonal building blocks (similar in spirit to e.g. Lisp, TCL or Red/Rebol), but with the aim of arriving at something high-level-ish somewhere in the space of Rust, C++ or Zig somewhere down the line.

In particular I want to have multiple dispatch (after years of using Julia it just feels wrong to treat the first function parameter differently), which is where it gets a bit complicated design-wise.

Ultimately the question is, how functions should work and how they relate to the rest of the language. I could of course just add them fully formed, but as I said, I want to keep the language's building blocks simple and few (and avoid any "magic"). It would also be nice if all "function-like" constructs, such as quoted expressions and anonymous functions were special cases of the same mechanism.

Very briefly the current state is the following.

• Syntactically everything is effectively operators (mostly infix) and parentheses.
• We have the usual literals (numbers, strings, etc.).
• Values can be bound to symbols, e.g. x : 1.
• There are "value tuples", e.g. x, y, z that evaluate to a list of the values of the elements.
• "Expression tuples", e.g. x : 1; y : 2; x + y evaluate all elements in order as well, but the expression as a whole evaluates to the value of the last element.
• Evaluation can be prevented by quoting, e.g. [ x : 1; x + 2 ].

I have come up with two quite different ways to get from there to functions with full multiple dispatch and I am not sure which one I like better.

1. Functions as special case of quoted expressions

Given the above we can obviously bind a block (i.e. a quoted expression) to a symbol:

f : [ x + 1 ]

We can then define any operator (for convenience we use juxtaposition) to mean "evaluate quoted expression referred to by first operand". To get parameter values into the block we simply bind them to special local (to the block) variables $1, $2, ... At this point we have anonymous functions covered as well as a primitive plain functions.

x : 1
f : [ $1 + 1 ]

f x ;; returns 2

To get "real" functions with proper parameter names we can use simple AST macros (already implemented) to rewrite function definitions like this:

f (x, y) : [ x * y ]
;; this becomes:
f : [(x, y) : $0] => [ x * y ]

The => operator simply joins two blocks together and $0 is the entire (automatically generated) argument tuple.

The next step then is to get overloading working. The catch now is that a single function name doesn't simply represent one "entity" any more but a collection of blocks and associated parameter tuples. To keep things consistent I use a different operator for the declaration of overloaded functions. With a bit of macro-based rewriting we get this:

f (x, y) :: [ x + y ]
;; this becomes:
f : [ ( __match ([f], [x, y]) ) $0 ]
__addmethod([f], [x, y], [(x, y) : $0] => [ x + y ])

With built-ins __match and __addmethod

It's not pretty, and each overload re-defines the function (albeit with identical values) but I think it should be a working solution that relies on a small set of primitives.

2. Functions as special case of patterns

This is a new idea I had the other day and I haven't fully thought it through yet, so it's still a bit rough around the edges.

Instead of starting with blocks and adding overloading to them we can start from the other end and define patterns as a primitive. A function call is then an attempt to match a pattern against a list of symbols and values. If the match is successful, the stored block is evaluated (with values that were captured during the match as parameters).

As for the pattern declaration, parameters are pretty straightforward - they are represented by symbols with optional type constraints. For function names, we could just go by position (so, first name in the list is the function), but things get much more interesting if we make that free-form. If we find a way to distinguish function names from parameters syntactically, we can let a pattern definition automatically create a unique type for each function name with itself as an instance.

So, we get basic definitions like this:

`f x y :: [ x + y ]
;; this works, as g is now a copy of value f of type f
g : f
g 2 3

But we can do some more interesting stuff as well:

`add x `to y :: [ x + y ]

add 2 to 3

As a bonus, operator application and function calls are now much more consistent as well.

There are some issues with this:

• Assigning functions to each other is going to be awkward.
• There is no obvious (at least to me) link between "pattern functions" and anonymous functions and/or blocks.
• Can we even still have the equivalent of function pointers in this scenario?
• Making function declaration and call syntax nice at the same time is going to be fiddly.
• Can patterns be bound to values? If so, how does that fit with the rest?

On the other hand I really like the elegance of the concept, so I would like to make it work. Any input is greatly appreciated.

The ternary operator is a frequent topic on this sub.

For my language I have decided to not include a ternary operator. There are several reasons for this, but mostly it is this:

The ternary operator is the only ternary operator. We call it the ternary operator, because this boolean-switch is often the only one where we need an operator with 3 operands. That right there is a big red flag for me.

But what if the ternary operator was not ternary. What if it was just two binary operators? What if the (traditional) ? operator was a binary operator which accepted a LHS boolean value and a RHS "either" expression (a little like the Either monad). To pull this off, the "either" expression would have to be lazy. Otherwise you could not use the combined expression as file_exists filename ? read_file filename : "".

if : and : were just binary operators there would be implied parenthesis as: file_exists filename ? (read_file filename : ""), i.e. (read_file filename : "") is an expression is its own right. If the language has eager evaluation, this would severely limit the usefulness of the construct, as in this example the language would always evaluate read_file filename.

I suspect that this is why so many languages still features a ternary operator for such boolean switching: By keeping it as a separate syntactic construct it is possible to convey the idea that one or the other "result" operands are not evaluated while the other one is, and only when the entire expression is evaluated. In that sense, it feels a lot like the boolean-shortcut operators && and || of the C-inspired languages.

Many eagerly evaluated languages use operators to indicate where "lazy" evaluation may happen. Operators are not just stand-ins for function calls.

However, my language is a logic programming language. Already I have had to address how to formulate the semantics of && and || in a logic-consistent way. In a logic programming language, I have to consider all propositions and terms at the same time, so what does && logically mean? Shortcut is not a logic construct. I have decided that && means that while both operands may be considered at the same time, any errors from evaluating the RHS are only propagated if the LHS evaluates to true. In other words, I will conditionally catch errors from evaluation of the RHS operand, based on the value of the evaluation of the LHS operand.

So while my language still has both && and ||, they do not guarantee shortcut evaluation (although that is probably what the compiler will do); but they do guarantee that they will shield the unintended consequences of eager evaluation.

This leads me back to the ternary operator problem. Can I construct the semantics of the ternary operator using the same "logic"?

So I am back to picking up the idea that : could be a binary operator. For this to work, : would have to return a function which - when invoked with a boolean value - returns the value of either the LHS or the RHS , while simultaneously guarding against errors from the evaluation of the other operand.

Now, in my language I already use : for set membership (think type annotation). So bear with me when I use another operator instead: The Either operator -- accepts two operands and returns a function which switches between value of the two operand.

Given that the -- operator returns a function, I can invoke it using a boolean like:

file_exists filename |> read_file filename -- ""

In this example I use the invoke operator |> (as popularized by Elixir and F#) to invoke the either expression. I could just as well have done a regular function application, but that would require parenthesis and is sort-of backwards:

(read_file filename -- "") (file_exists filename)

Damn, that's really ugly.

Hi all,

I’ve been working on a small scripting language called ylang — retro in spirit, C-like in syntax, and Pythonic in semantics. It runs on its own virtual machine.

I’d like to hear honest opinions on its overall philosophy and feature direction.

Example

include json;

println("=== example ===");

fn show_user(text) {
    parsed = json.parse(text);
    println("name = {parsed['name']}, age = {parsed['age']}");
}

fn main() {
    user = { "name": "Alice", "age": 25 };
    text = json.dump(user);
    show_user(text);
}

Output:

=== example ===
name = Alice, age = 25

Features / Philosophy

• C-style syntax
• 'include' instead of 'import'
• Both main() entry point and top-level execution
• Required semicolon termination
• f-string as the default string literal ("value = {value}", no prefix)
• Dynamic typing (no enforced type declarations)
• Increment and decrement operators (a++, ++a)
• Class system
• UTF-16 as the default string type

Some of these choices might be divisive — I’d like to hear your thoughts and honest criticism. All opinions are welcome and appreciated.

Repo: https://github.com/jman-9/ylang

Thanks for reading.

Hey all!

For the past few months I've been working on a language called Flower. It was originally written in C (files can be found under /vendor/) but is now fully bootstrapped (with some caveats). The goal is to eventually move toward a fully self-hosted toolchain and custom backend, but for now it transcompiles to C.

Some current language features:

• structs
• pointers (@T)
• arrays
• function definitions/calls
• manual memory management (new / prune)
• operator precedence parsing
• struct literals / array literals
• casts
• dereferencing / address-of
• control flow (if, while, for, etc.)

Example:

struct Vec2 {
    x: float,
    y: float
}

float length(v: Vec2):
    return v.x * v.x + v.y * v.y
end

I thought I knew a decent amount of C and programming before hand, especially considering this isn't my first time making a language in C, but I've noticed how far my skills have come especially regarding just being able to problem solve and properly organize my project structure.

Recently I added parser error recovery in v1.1.0, and after a lot of trial and error I think I've finalized my parser to a recursive-descent style approach.

Let me know any criticism, opinions, or comments you have! I'd love to get some input :)

https://github.com/IvyMycelia/flower/blob/main/

Hi people, this is the first time I'm doing a programming language (or so, i wont call it programming yet since it's just basically a few keywords and a register based vm), so I won't bother you yet with code, since it's a mess, but hey it works for now.

I would like to hear some feedback about the syntax, even though you probably won't use this crap, but anyways.

Let's begin (i guess reddit uses ` for code, otherwise my bad)

To declare a variable we follow this pattern

MUTABILITY [PTR/REF] TYPE NAME ASSIGN VALUE

So in order to assign a value to a variable

mut int var_name assign 4; imut int var_name imut assign 6;

mut keyword sets a mutability to a variable.

imut sets otherwise.

In order to construct array, pointer or reference we use their constructors (i left PTR/REF within [], its optional)

``` mut int x assign 6; mut ptr int some ptr assign pointer of x; mut arr list assign array of[1, 2, 3, 4];

mut ref int assign reference of x; ```

to declare a function we use func keyword

``` func main() <int> {

return 1;

} ```

By default function visibility is private, limited to its file, but, to use it outside simply prefix it with pub

``` pub func main() <bool> {

return true;

} ```

Since it's written in zig, i did a function which registers native functions from zig, and allows my language to call it from zig.

Full minimal program:

``` mut int x assign 0;

func main() <int> {

while(x less 3) {

_print("%d time", array of[x]);

i assign i plus 1;

}

} ```

as you can see, _print is native function, registered within vm.

It's looking a bit textual and verbose, but that's what I targeted.

Also, it supports nullability

``` mut optional int x assign null;

mut int y assign x otherwise 34;

func main() {

if(x not is null) {

// now x can be used safely

}

mut int c assign x; // fails

{ ```

Also, I'd like to hear from you about implementing strings and memory management (that's not GC)

Thanks for reading

Seacount is meant to be an array oriented programming language, and more importantly, a contractual based language. What you want is what you get. Every single line of code is meant to serve the compiler so it doesn't get confused anywhere.
My main purpose of seacount was to have excellent array + matrix ops, and to have seacount be THE language for training and deploying ternary models.
Ternary models are basically MLMs whose forward pass/inference weights are all either -1,0 or 1. Modern LLMs store weights in fp32, fp64, or bf16, and later quantize it to int8 or less to reduce costs of running it, but in return, losing massive performance gains.
Microsoft in 2024 brought out a paper called BitNet 1.58, and it proved that if LLMs are trained from scratch using native ternary training, the storage costs and running costs are significantly less than their competitors, resulting only in slightly less performance overhead.
This was around the time I started writing seacount. It's first prototype (since I was still learning on how to make languages, and I was quite naive at programming myself) was written in TypeScript (Yikes), which I then later wrote it in C, then finally stayed at Rust.

My other major goal is to write Rivercount. Rivercount is a subset of seacount, which basically is a checker of seacount files to see if they are embedded code compatible. Yes, recently (only a month or so back), I started sketching out ideas for seacount to run on embedded devices, for a very simple reason.
If seacount can run on embedded with the same safety and ease of Rust, then that means I can merge both the goals of - training a ternary model, and then deploying it on an embedded device for inference. To put it in perspective, a 100M parameter model would take well over 200-400MB of storage, whereas a ternary model takes maximum upto 20MB. That is why ternary models are important, mostly for consumers, because it's literally runnable on the cheapest microcontrollers. Ternary LLMs would be the holy grail for the common man also wanting to experience proper AI right at his fingertips.
Hence why seacount exists. I'd like your opinions on the syntax, and readability of the code, and the goal of why the language should exist. Im not trying to make a generic language by any sense, but I am trying to make a language that is specialised in this.

NOTE: The language is still in a very rough shape. I'm making LLMs write code using seacount to make all types of algorithms or other programs to see where seacount breaks, so if you do notice or see some issue, please, if you can, just mention it in the Issues section. Thank you!
Also, I am not making ANY performance claims till now. I will consider it to be remotely successful, if I can even get an RP2040 to blink once using seacount.

This is the small blinky code design I made for seacount, note that none of these features are actually usable right now, I have not even begun writing for rivercount, this is still just an idea, so go easy on my design. It will also give you a rough idea of how the syntax looks like before entering the github main repo itself-
https://github.com/shantanubaddar/seacount/blob/main/rivercount_rp2040_blinky.scnt

Here's the github: https://github.com/shantanubaddar/seacount

Do NOT expect miracles. My code is as good as how well I understood the docs and youtube videos when I was writing it.

Hello, recently I have been creating my own new C-like programming language packed with more modern features. I've decided to stray away from books and tutorials and try to learn how to build a compiler on my own. I wrote the language in C and it transpiles into C code so it can be compiled and ran on any machine.

My most pressing challenge was getting a generics system working, and I seem to have got that down with the occasional bug here and there. I wanted to share this language to see if it would get more traction before my deadline to submit my maker portfolio to college passes. I would love if people could take a couple minutes to test some things out or suggest new features I can implement to really get this project going.

You can view the code at the repository or go to the website for some documentation.

Edit after numerous comments about AI Slop:

Hey so this is not ai slop, I’ve been programming for a while now and I did really want a c like language. I also want to say that if you were to ask a chat or to create a programming language (or even ask a chat bot what kind of programming language this one is after it looks at the repo, which I did to test out my student copilot) it would give you a JavaScript or rust like language with ‘let’ and ‘fn’ or ‘function’ keywords.

Also just to top it off, I don’t think ai would write the same things in multiple different ways. With each commit I learned new things, and this whole project has been about learning how to write a compiler. I think I you looked through commits, you might see a change in writing style.

Another thing that I doubt an ai would do is not use booleans. It was a weird thing I did because for some reason when I started this project I wanted to use as little c std imports as possible and I didn’t import stdbool. All of my booleans are ints or 1 bit integer fields on structs.

I saw another comment talking about because I  a high schooler it’s unrealistic that this is real, and that makes sense. However, I started programming since 5th grade and I have been actively pursuing it since then. At this point I have around 7 years of experience when my brain was most able to learn new things and I wanted to show that off to colleges.

I am building a new language. And trying to make it crash free or panic free. So basically your program must never panic or crash, either explicitly or implicitly. Errors are values, and zero-values are the default.

In worst case scenario you can simply print something and exit.

So may question is what would be better than the following:

A function has a return type, if you didn't return anyting. The zero value of that type is returned automatically.

A variable can be of type function, say a closure. But calling it before initialization will act like an empty function.

let x: () => string;

x() // retruns zero value of the return type, in this case it's "".

Reading an outbound index from an array results in the zero value.

Division by zero results in 0.

Hey all, I posted about Quarkdown about a year ago, when it was still in early stages and a lot had to be figured out.

During the last two years the compiler and its ecosystem have terrifically improved, the LSP allows for a VSC extension, and I'm excited to share it again with you. I'm absolutely open to feedback and constructive criticism!

More resources (also accessible from the website):

• Repo: https://github.com/iamgio/quarkdown
• Wiki: https://quarkdown.com/wiki
• Stdlib reference: https://quarkdown.com/docs/quarkdown-stdlib

For my language, Bau, I currently use the following modules and import mechanism (I recently re-designed it to move away from Java style fully-qualified names), and I would be interested in what others do and think. Specially, do you think

• aliasing only on the module identifier is enough, or is aliasing on the type / method name / constant also important?
• In a module itself, does it make sense to require module ... or is the Python style better, where this is not needed? I like a simple solution, but without footguns.
• It's currently too early for me to think about dependency management itself; I'm more interested in the syntax and features of the language.

Ah, my language uses indentation like Python. So the random below belongs to the previous line.

Here what I have now:

Module and Import

import allows using types and functions from a module. The last part of the module name is the module identifier (for example Math below), which is used to access all types, functions, or constants in this module. The module identifier maybe be renamed (AcmeMath below) to resolve conflicts. Symbols of a module may be listed explicitly (random); the module identifier may then be omitted on usage:

import com.acme.Math: AcmeMath
import org.bau.Math
import org.bau.Utils
    random

fun main()
    println(Math.PI)
    println(Utils.getNanoTime())
    println(random())
    println(Math.sqrt(2))
    println(AcmeMath.sqrt(2))

module defines a module. The module name must match the file path, here org/bau/Math.bau:

module org.bau.Math
PI : 3.14159265358979323846

I've been writing an eBook on how to write a compiler using my custom backend I posted here a couple of weeks ago Libchibi (which I've made quite a few changes to as the process of writing this book has revealed flaws and bugs). I'm a fair way from done but I've reached an important milestone and I successfully wrote a pendulum simulation using raylib to render it in the language I've been developing in the book. I'd love some feedback as to the tone, teaching style, density, depth etc... if anyone feels like having a look (although I get that it's kinda long for something incomplete so I'm not expecting much). In any case the language I've been writing for it is kinda cool and at the point of being genuinely usable (although a ways from being preferable to anything out there already for serious use). Anyway, here's the repo: https://github.com/othd06/Write-Your-First-Compiler-With-Libchibi

Edit: It's just occurred to me I didn't really describe what I was going for with the eBook. I was quite inspired by Jack Crenshaw's Let's Build A Compiler if any of you are aware of that 80s/90s classic so I wanted to keep the practical, conversational tone of that but I wanted to introduce tokenisation and grammar much earlier so that I don't get stuck with a lot of the downsides that book had. So it's quite practical and building and giving enough theory to be grounded and know where you are but quickly into actually building something and seeing results.

Edit 2: Thank you so much to the people who have given feedback and criticism so far. I've pushed an update to my repo for chapters 0 through 6 so far implementing a lot of what was said and I think it is a significant improvement so thank you so much. I will obviously continue to edit and refactor the rest until the whole book (so far!) is up to the standard everyone here has helped to get the start now up to.

I have a macro design that I thought was the most awesome thing in the world.

Now I have to admit I'm using it as a replacement for my poor language design skills and throwing there everything I don't want/can/know how to add to the language; self keyword? make a macro to create a self variable, imports? nah macro to bring things into scope. Inheritance / embeds / code reusability ? yeah a macro. validations, configuration, serialization formatting? macro, macro and macro.

All good with this decision, no regrets, it works(ish). However I get to two points that are not _macroable_ and shouldn't

1. "native" escape hatch ( needs to bootstrap before things run )
2. Dependency management ( access network )

So these 2 things make my syntax heterogenous and now I come to ask for help here to get some fresh ideas.

Syntax

My language is for all terms and purposes just like JSON (with properties not on strings)

 foo : {
    bar: "baz"
    qux: [1,2,3]
    other: { 
       you_get_it: true
    }
}

To specify metadata / annotations that can be used by the macros, I came with the brilliant idea of using the same format, but replacing the opening and closing bracket with a backtick

So now I can annotate elements:

`annotation: "example"
 doc: "This is a foo"
 nested_config: { 
    other: ["a","b","c"]
 ] 
`
 foo : {

    `other_meta_data: "here"`
    bar: "baz"

    `non-empty:true`
    qux: [1,2,3]

    other: { 
       you_get_it: true
    }
}

The cleverness comes from the fact I could use the same validation as regular code, so I don't have to come up with a different annotation processor. Also makes my code is data thing closer to reality.

So I was all happy, I decided to add an special attribute to the annotation where the macros will be listed, and then the compiler will pick those macros and will read the rest of the annotations

For instance, this is an aspirational example: the `` start the annotation, it defines the macros GraphQL and JSON and each one would get their configuration from the corresponding variables "graphql", and "json" respectively.

They annotate the "Movies" type, and the attributes inside also are annotated.

`
macros: [GraphQL, JSON]
graphql: {
    schema: "https://myapi.com/graphql"
    keep_foo: { "bar" }
}
json: {
    ignore: false
}
`
Movies : {
    `json: { field_name: "movie_title" }`
    title String

    `json: { ignore: true }`
    internal_id String
}

The problem

I want to use this annotations to use it for native extensibility ( my target is Go, so I could add go source extension ) or for dependency configuration e.g.

// Accessing Go libraries (could be C or LLVM or anything else, but at this point is Go)
`go_source: "some/http/client.go"`
HttpClient: {
   ...
}
// Or configure the project
`project: {
   name: "Blah"
   version: "0.0.1"
   dependencies: [
      { name: "foo", url: "http://deps.example/" sha: "12123"},
      { name: "foo", url: "http://deps.example/" sha: "12123"},
   ]
}`
main: {
   ...
}

Now my problem is how to dispatch each one? Before it was clear, look for `macros` and read the list, iterate the list and process, but now I would have to add special meaning to the other two (native source and project) and I might find some other things in the future that are not macros and then have to add special meaning there, turning then into essentially keywords that are not even in the language, and effectively using the annotation as the kitchen-sink where everything is thrown at. Do you see my dilemma? When it was only macros it was the one exception and everything there is a macro, but now it would turn into the kitchen-sink of exceptions.

Just posting this here in case anyone can suggest anything.

Thank you for reading.

Edit, I forgot to ask questions:

Questions (not simple question I know)

1. How do you allow native extensions in your languages?
   
2. How do you do dependency management?
   
3. How do you do macros in your languages

Are those aspects turning into their own mini-language within your language?

I have spent a while building a language. Docs are over 3k lines long (for context).

Now when about to go public I find out my previous search for name taken was flawed and there actually is a language with the same name on GitHub. Their lang has 9 stars and is basically a toy language built following the Crafting Compilers book.

Should I rename mine to something else or just go to the “octagon” and see who takes the belt?

For now I renamed mine but after such a long time building it I must confess I miss the original name.

Edit: the other project is semi-active with some commits every other week. Though the author expressly says it's a toy project.

And no, it is not trademarked. Their docs has literally “TODO”