Writing a Compiler in 2024

Towards Higher Level Syntax for Programming Languages 2024-12-05

Dec 05, 2024

I believe that no-one should write a compiler in 2024.

If you want to learn how compilers historically work under-the-hood, then, of course, studying compiler construction is a reasonable approach.

If, though, you simply want to invent a new language and write a compiler for it, or, just write a compiler for an existing language, you are better off studying PEG parsing with technologies like OhmJS and Ohm-editor, or getting ChatGPT (or Claude) to generate Ohm grammars. Learning how to build compilers the “old way” is more work than necessary these days.

In 2024, we already have enough compilers. We can lean on existing languages to act like “assemblers” for new languages[1].

Overview - PHI

Steve Phillips created a new language that he calls PHI. He intended to write a compiler for it, but, instead, got an LLM (ChatGPT/Claude) to generate an Ohm grammar and corresponding code that converts PHI into legal Go programs.

Now, PHI programs execute successfully in the Go environment.

The PHI transpiler converts PHI programs into Go, then the Go compiler compiles and runs the programs.

Overview - RT

I took a manually-written Python program, consisting of about 1,400 lines of code and created a new syntax (RT[2,3]) that can be used to describe the same program details as the Python program.

I wrote an Ohm grammar for the new syntax and used a corresponding custom DSL (RWR) to write a transpiler for RT.

The RT transpiler successfully compiles RT programs into executing Python and Common Lisp programs.

[The manually-written 1,400 LOC Python program happens to be a mutual-multitasking kernel (called “0D”) plus stock libraries plus code for implementing the Larson Scanner regression test. This experiment is still WIP, and, is regression tested by compiling and running a DPL (diagrammatic programming language) program that I call “Larson Scanner”. The source code for the “Larson Scanner” program is written using the draw.io[4] editor].

The main goal of this experiment is to convert the kernel and the DPL program (Larson Scanner) into Javascript. That step is yet to come - I expect it to take a day or two (of effort, modulo interruptions) and to run on the command-line using node.js initially.

Details

I want to keep this article succinct - more details gladly supplied on request.

Bibliography

[1] In a 'real' Computer Science, the best languages of an era should serve as 'assembly code' for the next generation of expression (Alan Kay, 31:50) from

[2] RT Transpiler from https://programmingsimplicity.substack.com/p/rt-transpiler?r=1egdky

[3] RT repository from https://github.com/guitarvydas/rt

[4] Draw.io from https://app.diagrams.net

Paul’s Substack

Discussion about this post