Programming Language Workbench - Beginnings
Towards Higher Level Syntax for Programming Languages 2024-11-15
Inspired by my previous REPL experiments, I’ve begun playing around with a more complicated example - a programming language workbench.
I manually ported the 0D kernel to Python, some 1400 LOC.
I’m trying to “lift” the syntax into a VHLL, so that it can generate the same Python code, and, generate CL code. Later, I hope to generate Javascript.
I wrote a Larson scanner program in drawware. I’m using that code as a regression test for this stuff.
I believe that Python and Common Lisp are at two opposite ends of the code-generation spectrum. I think that generating Javascript will be easy, once I get both ends of the spectrum - Python and Common Lisp - working.
I plan to explain more about what’s going on here. Questions welcome.
The inhale and exhale ends of the pipeline, shown, convert the incoming text into an internal form that makes it easier to use Ohm space-skipping grammars. For example, I don’t want to skip spaces in strings nor comments, so I pre-process the code to encode such things, then unencode them at the last moment.
Likewise, I’m accustomed to using comma-less languages, like Lisp. This kind of thing makes spaces significant, but, only in some cases. I pre-process the incoming code to wrap Unicode brackets around identifiers to tokenize them. This simple pre-processing makes it much easier to use Ohm syntactic rules in downstream processes. I put the heavy-thinking code in downstream processes. The downstream processes receive a tokenized stream, which reduces cognitive load and allows me to concentrate on the interesting grammar transformation instead of dealing with niggly lower-order details.
Repository
If you want to follow along, the WIP code is in branch CL of https://github.com/guitarvydas/rt/tree/cl.
The Larson scanner for regression testing is in https://github.com/guitarvydas/rtlarson. The current version of the VHLL generates Python code that works with the Larson scanner, but, I don’t have the Common Lisp version working yet.
See Also
References: https://guitarvydas.github.io/2024/01/06/References.html
Blog: https://guitarvydas.github.io
Videos: https://www.youtube.com/@programmingsimplicity2980
Discord: https://discord.gg/qtTAdxxU
Leanpub: [WIP] https://leanpub.com/u/paul-tarvydas
Gumroad: https://tarvydas.gumroad.com
Twitter: @paul_tarvydas