One Year to a Glorified Calculator

Current Flower codebase is now over a year old

The current codebase of Flower survived through an entire year of git history while remaining in active, albeit erratic, development.

And it works as a glorified calculator. While that is not what I was hoping for for a year-worth of development during my free time, it is certainly getting somewhere - though I may be the only person in the world to see that progress.

But how does one waste a year to create a complex version of a calculator?

Two ways to write a compiler

Following the (low-level) programming language community, I've noticed that getting your language to a working state is a bit of a balancing act.

On the other hand, it's not too complicated to roll a compiler in a weekend and many people have done it. Either following the Kaleidoscope tutorials or the Crafting Interpreters book. I've done that too. That's how Flower got started in practice. I had little previous experience writing a complete compiler, but in a week I had something that took my syntax and compiled into native code through LLVM. The only thing still remainning of that code is the lexer.

On the other hand, some people want to go very in-depth with their design before putting down a single line of compiler code. There might be year's worth of design, maybe even tooling support or LSP server going on - yet the language itself doesn't really do anything yet, or possibly ever.

I do not think either of these approaches is wrong. They are for different purposes and both have their merits and flaws, which I am not too interested in discussing. There is plenty of discussion about it available in the Internet if you care to look. But it is a good point to reflect on Flower, and myself as a language designer.

I have a vision what I want to do with my language, and I am perfectionist enough to think that if something is worth doing, it probably is also worth overdoing. I am afraid that puts me firmly in the second camp.

Flower is definitely not a jam language, I want to be able to build actual projects with it at some point. I agonise over almost every single detail I put into the language design, and try to reflect those intentions in the compiler. I rewrite great deal of what I write if I deem something unsatisfactory.

While this definitely makes me utterly subpar game programmer (unless we are thinking about some 6502-era device where you kinda have to think about those things). But I think this is not a bad thing to do when you are creating a general-purpose programming language. I enjoy taking my time and doing things as well as possible with my current knowledge.

There has been some talk about how not seeing the progress kill many hobby languages in the second camp. But I entered the world of programming language development through exessive Yak shaving. I enjoy the journey, so I think my enthusiasm for the language is better maintained by seeing how the internal parts fit together rather than being able to quickly produce end results. I almost gave up on Flower before, because I went the fast route and was not happy with the result. But now, that I get to write this in my own pace, miss my own deadlines but do it the way I want to I have much more long-lasting interest than my average project.

The takeaway here is that there is no one-size-fits-all approach. If you want to make something as low-level as a programming language, do it the way you want to, not how others tell you to.

Advantages of taking it slow

The upside of the slow approach is that after the building blocks are in place, the visible advancements come in rapid bursts before the next part of the laying of the foundation begins anew.

There is also something I find pleasing, that is that when I take the time to think about the features I can find connections between them that I absolutely would miss with faster-paced development. This makes the language much more self-consistent than it would otherwise be.

How my typesystem, metaprogramming features and module system play together are significantly more in-sync with each other than in any of my earlier plans. Sure, half of that isn't implemented yet (and there will be a blog post about each in isolation when they are), but all of those are something that would be very difficult to change if I had rushed them as separate components.

The interplay here, where types are compile-time functions, namespaces are types, and modules are namespaces is something I find elegant, and something I find quite unique in Flower. And discovering and trying out things like this is what makes me enthusiastic about continuing the development. And things like this would be difficult if I just kept building on top previous iterations instead of sometimes tearing down parts of the foundation I don't like and figuring what was wrong with them.

This is also why I don't really show too much details of the language yet, it is still so much open to change that I don't want to bring other people aboard before I've got a solid foundation to build on, and I do not want to present the foundation to other people before it is in a shape that isn't a bloody mess.

And then the disadvantages

I had worked with the language for about a year, and with the current codebase around half of that when I decided to start writing a blog about the language. It is just not very exciting to report "yeah, I did work on the internals this month too". I have no illusion that I could make an actually decent and pragmatic programming language all on my own. And at such a glacial pace as Flower is progressing, it is hard to inspire anyone to join me at the task. It is a problem I know I need to address in the future.

Another thing is that while I do find the journey quite thrilling, I cannot deny that seeing everything progress at such a glacial pace is sometimes disheartening. I know that there's been a ton of progress since three months ago, but as the output remains close to the exact same, it doesn't always feel like it. That in turn makes me pick up a side project every now and then, which again reduces the already low amount of time I can use to develop Flower.

Things to leave behind

I mentioned that I'm in the "think about everything at the cost of development time"-camp. I've been there a long time, and it certainly has brought me some self-awareness which has proven useful with Flower.

I believe avoiding the scope creep is imperative in software development, and doubly so in cases like Flower. The good part is that the scope has remained somewhat constant (even though I've realised the amount of work is way more than I initially thought).

Tooling is something that flew out of the window in my conciderations quite early. I try to make it not needlessly difficult to make tooling, but I am working alone with a language with extensive metaprogramming features. So tooling was the first of the big sacrifices I realised I had to make if I ever wanted to get the language to a point where I could use it.

It doesn't mean that I wouldn't want good tooling, or even that I didn't think about how that would work with the compiler every now and then, but it does mean that it is not in the list of things I focus on. And there are more of these sacrifices than I'd like. But I want to keep the core functionality I initially envisioned, and other things may need to be retrofitted.

There are other, smaller, things. I brought up the tooling part because it was one of the more painful ones, and should be somewhat alarming to others who make their own languages.

Current status and next to come

You might've noticed the lack of status report on February. That's because absolutely nothing changed in February. I barely had time to think about Flower at all. In fact, I had planned to make this one-year-codebase-anniversary post in February, but couldn't muster time until mid-March to do so. As my schedule is still pretty packed right now, I wouldn't expect one in March either. It seems inevitable that more steady development on Flower will continue only in April or May.

The next things to do when I do get back to it, is to take the virtual machine to a state where it can reliably handle all the calculator stuff with test suite and somewhat decent debugging tools (read: disassembler) in place.

After that, it's time to tear into types and modules and get rid of the current draft implementations in favour of more permanent ones.

See you again when I (hopefully) get to write more about that!