Starting out, part 3: The Wild Parts

What does the future hold?

Unlike the other entries in the first two parts of the overview, this one is far more abstract, since it only considers future ideas I have for the language, and as such, none of it is implemented. Well, except for the ones where the idea is "I'm explicitly not wanting this".

Some of the ideas are also quite directly stolen from other languages, or C++ papers, or alternative compilers or wherever.

Function extension

One of the things I do not know any other programming language has, but I've played around in my mind (and with the second iteration of the compiler itself), and which has affected the design of the early parts of the language a bit is how I plan to (or at least try to) handle functions.

First-order calling conventions

I want to play around having calling conventions, and in more general, function call ABI be a part of the function definition.

In my mind, the language would first support writing custom calling conventions in the language itself, and then those would be usable as a part of the function definition.

It comes with some pros and cons, but it's an idea I feel fits quite well with the general idea where I want to take Flower, and I personally like the pros more than I hate the cons.

I like that calling conventions for a platform could be implemented as a library. And the fact that things such as name mangling could be implemented as one too. (Would probably make C++ or Rust interop way more realistic too!)

Heterogenous programming

With the ABI becoming part of the function definition, it offers a quite simple gateway to heterogenous programming. If part of the ABI is already part of the function definition, it doesn't take too big a leap to think that compilation target or the execution model could be that as well.

In some form, this already happens in Flower, as the "can this be executed by the Flower VM" and "can this be executed in a native code version" is already a part of the function definition. Though implicit instead of explicit at the moment, but the machinery is there and already serves a purpose, since there are pieces of code that are designed to be VM-only runnable (such as type definitions, they are functions, remember?) and code that is not allowed to be run in the VM (such as main(), or any other possible entry function).

Extending this would give a chance to for example, compile parts of the program to SPIR-V. Sean Baxter's Circle compiler actually does something like this already, so there is a real-world example.

I also think this is something I find interesting to pursue, since it would allow Flower to be good at writing simulations or GPGPU code in general, and it fits with the general design ideas and goals.

These ideas have been hatching in my head for over 6 months now, and are quite likely to get into the language in some form.

Formal Specification

Flower needs a formal specification. I think it is a overlooked requirement for a programming language. The only reason there isn't one for Flower yet, is that everything changes still. Once the syntax and features start freezing, the specification will pop up.

Custom backends and linkers/loaders

I'm not sure everyone would agree with this, but I'd say that there's a good chance Flower will break the traditional split between compiler frontend and backend.

The more I work with the "ABI is a part of function definition" - idea, the more I feel the need for a custom backend, and maybe even a custom linker/loader. I have not put considerable amount of time into this problem yet, but it might be something to tackle.

Most languages are fine dealing with object code and system linkers as they've always done. I think there is a lot of unexplored territory there, and while it certainly is a dream for more distant future, I do not want to lose sight of it.

It will need more thinking when a module system is introduced to Flower, but we're not there yet. But even just things like pushing monomorphisation to link-time, and in general the ability to provide linker with more details about the program seem like worthy avenue of exploration to me.

Definitely going back to this one in the future.

Self-hosting

Ah, the usual. Being able to self-host the language is definitely a goal at some point. But there are some things that I hae on my mind with this one.

For me to call a language self-hosting, the self-hosting chain needs to be complete. Which means no LLVM or QBE. It would definitely require a custom backend at that point. Though debatable, I think it's pretty disingenous to call a language self-hosting if you need components written in other languages to be able to create machine code. Linking it is a somewhat different story, but I find it a bit of a stretch for a language to call itself self-hosting if it is just a frontend.

In Flower's case it might make sense to include linker in addition to the backend to that chain, but the jury's still out with that one.

I'm not too much in a rush with this one, but it should happen in some point of time. With or without the linker part is a question for the later time though.

Working aside C, instead of relying on it

There is the very far-away goal of being able to use Flower without using the C standard library.

Sure, we want to be able to link with it, and handle language interop in a reasonable manner -- even if we didn't have to. But I would also slowly want to reduce the reliance on that. But let's not kid ourselves too much, for a really long time, most things would be very reliant on the system C library.

In any case, C interop is a thing that already exists in some forms, and it is not going anywhere. I am also unsure if I'm willing to go all the way and rewriting syscalls, reinventing POSIX and such with Flower. But I'm not appalled by the idea.

In any case, there needs to be a reasonable route there if it was ever to happen. After all, Flower is supposed to be practical.

Ideas about the Flower Standard Library

I've floated around some thoughts about the standard library. They've been mostly architectural or gathering of requirements and nothing too specific at this point. There's much work to do before I even can get to the point of starting standard library implementation.

But some things are pretty clear. Since Flower has such a heavy emphasis on metaprogramming, a lot of the things that are language features in other languages are probably going to be library features in Flower. Quick examples include things like namespaces and classes.

I've even played around with implementing return as a library function, which would open a whole can of worms in itself.

I would also like there to be some kind of system to get quick access to experimental stuff, since there's going to be a lot of it. Or, in other words, most of everything is going to be experimental for a good while. I've been throwing around some separation like ["core", "base", "unstable", "experimental"] or something, where different parts of the standard library have different set of requirements imposed on them.

Another thing I've played around in my head is that the standard library could standardise some interfaces instead of the implementations, and sort of work like a bridge between libraries. I don't know yet how this would handle in practice, but I think it tackles the biggest problem C++ has with its standard library. Which I think is the fact that while there's bazillion implementations of different things, such as networking, graphics, maths, etc., none of them are in the standard library. But the problem there really isn't that they aren't in the standard library. How I see it is that the problem is that those multitudes of libraries have no common way to talk to each other. Maybe tackling the problem from this perspective could work with Flower, even if it would be too late for C++?

Another thing I want would be that the standard library would be practical. It needs to cater to needs of users, not some obscurities never witnessed.

Of course, there ever being a standard library probably requires that I wouldn't be working with the language alone anymore, and I'm not yet even ready to open up the compiler source to public (I will later on though!)

So definitely something that is a future problem for now.

Things never meant to be

Well, those are some wild and less wild ideas I have for the future. Then there is the side what Flower will quite likely never be. I am not going out of my way to prevent someone from trying any of this, but they are explicitly non-goals for the language.

Garbage-collection

Garbage-collected is the first thing among those. Well, I guess technically it will be possible for someone to create a VM to run flower bytecode with GC, but I see little to no point in doing that.

First of all, I believe RAII is superior to GC in general. It is much clearer what is actually happening, it is controllable and it is deterministic in principle. That fits a systems language much better than a garbage collector.

Second point is that even though currently Flower bytecode is run in a VM, and that it is not short-term-priority to change that, the end goal still is to compile all the way to native code. And I do not want to introduce a huge runtime with adding a GC.

Memory-safety (a.k.a. not being Rust)

Being as memory-safe as something like Rust is explicitly not a goal. And there are a couple of reasons for that. And since memory-safety seems to be the trend phrase for systems programming, I feel like I need to go a bit more into detail about why.

I do not care about reinventing Rust or C++

I'm not making a "successor language to Rust", or even a "successor language to C++". (Although I do try to make the latter happen for me with Flower in the long run, but I do not intend to cater to nearly all the use-cases of C++.) I believe there is room for a niche languages, even if they were general-purpose ones, which sounds kind of weird at first. Rust didn't stop me from using C++, Flower will not stop me either, regardless of how successful or unsuccessful it will end up being.

I'm not here to replace a language, but to give more tools to solve problems with. I hope I can cut out a niche (at least for myself) with the language, but if you want memory-safety and like Rust, you should probably use Rust. Flower isn't here to change that.

I do not think Rust-like memory-safety is good for embedded

I do not like embedded Rust. And that is mostly because its memory-safety.

Sure, I would love to not make memory errors while working in embedded, but memory is not some abstract thing in embedded world, and I very much need the tools to deal with the more concrete memory myself. Memory-safety just gets in the way here, and makes a lot of the work significantly harder, and you need to deal with the memory magic in any case.

I just do not think it is worth the tradeoff in that domain. I know Rust front page (at least in 2022-10) touts about being for embedded, but from what I've used Rust in embedded, I think it absolutely sucks to use it there, compared to languages with less strict memory semantics, such as C++ or Zig.

Constraints and complexity

Like most base-level features, aiming for memory safety adds significant constraints to the language semantics. It also requires a lot of extra complexity from the compiler. Neither of which I find too desirable.

So, while memory-safety is going to be a thing at the back of my head, I'm not aiming nearly as high there as Rust is. There is a room for compromise between ergonomics and safety here, and while it definitely is worth considering, there is practically zero chance that I'll want to introduce anything nearly as strict as Rust here.

Things that may be?

There's a final category. It's about things I've thought but I'm still relatively unsure how to handle those.

How to handle coroutines

Handling of coroutines is something I've given a good amount of thought, and I still am not sure how I want to handle it.

The usual await/async - way of handling things is not something I am too fond of in a systems level language (I find it totally fine in higher-level languages though). It obscures a lot from the developers, to the point that many people I've met hold really weird misconceptions about what exactly even coroutine is (it's a function that can suspend and resume) to what hardware features it requires (no, it doesn't require threads).

I've been playing around with just adding suspend as a way to exit functions and then just calling them again to resume them later. Technically it allows implementation of the usual await/async-mechanism if that is wanted, but since it's not really battle-proven anywhere, I'm a bit hesitant with that approach.

The const problem and sane defaults

When I first started writing Flower, I was completely convinced I would want everything to be const-by-default. I am much more conflicted about that now.

In fact most of the things I thought as sane defaults have started to show cracks when I think about them more. Maybe I am overthinking, but I haven't yet come to any decision whatsoever in almost any of the defaults to set to things.

And where does that leave us?

With a lot of unimplemented stuff and a lot of future possibilities that may affect the basics.

And there are a lot of smaller features, that do not significantly alter how to think around the language, e.g. language-level-bigints (which I hope to get to a point where they are range-checked in the compiler before being coerced to fixed-size-integers), etc. But those are pretty minor things for now.

Anyways. That concludes our overview of Flower. Or at least how it was in 2022-10-07. I hope I will be more able to show the syntax in the future posts, as well as delve into details in a bit more in-depth way.