Text-Based Portability

Towards Higher Level Syntax for Programming Languages 2024-12-10

Dec 10, 2024

My answer to the portability issue is to write a VHLL syntax that is kind of like a target-dependent-macro.

Much of RT[1, 2] is just syntax, and, "syntax is cheap"™️. Using today's tools (t2t[3], OhmJS[4], PEG[5], etc.) I can rewrite the syntax to do whatever is needed for the target.

In the extreme edge-cases, I resorted to using just a function-call syntax prefixed by "#". The rewriting code can twiddle the args as necessary to emit legal target-language code.

This is what Lispers have been doing for ages. Now, everyone else can do it, too, with tools like t2t and Ohm and PEG.

Each target language gets its own custom rewrite, but all rewrites are based on the same starting grammar. Creating custom `.rewrites` is trivial if you are already familiar with the target language. The common stuff is in the `.grammar`, while the custom stuff is in the `.rewrite`. This is IMO better than pragmas and conditional compilation and trying to force-fit portability into one-language-to-rule-them-all.

A lot of work was done on portability with the early compiler-oriented research (e.g. RTL[6], OCG[7]), but, hasn't made it into the mainstream because of the stigma associated with the word `compiler`. I’m just lifting these ideas into the realm of text-to-text rewriting for text-based and DPL-based languages. When you squint just the right way, you can see that this is exactly what `compilers` do anyway (i.e. text-to-text transpilation), but, it was a lot harder in the early days than it is now in 2024. The trick is "can you do this in a couple of hours, instead of a couple of weeks/months/years?" - and - how does that change your attitude towards software development?

To get more concrete, a dumb example is this RT snippet...

#print_stdout ("*** PALETTE ***")

Which gets rewritten in Python as:

print ( "*** PALETTE ***") #line 78

And gets rewritten in Common Lisp as:

(format *standard-output* "~a" "*** PALETTE ***") #|line 78|#

And gets rewritten in Javascript as:

console.log ( "*** PALETTE ***");/* line 78 */

Some more examples of simple rewrites are in the RT article[2] (section "Snippets").

My claim is that if you hang such tiny text-rewrites off of BNF grammars, it gets to be really easy to do, and, opens up new possibilities...

LLMs

Steve Phillips created a new language that he calls Voltair (formerly called Phi). He created a compiler for it in about 8 hours by getting Claude 3.5 Sonnet to write code that maps Voltair to legal Go code (approx. 2000 LOC). Voltair programs now build and run, while the Go compiler does the heavy lifting of doing the actual compilation work. As I understand it, Phillips started out by getting Claude to write OhmJS code to do the mapping.

Bibliography

[1] RT repository from https://github.com/guitarvydas/rt

[2] RT Transpiler from https://programmingsimplicity.substack.com/p/rt-transpiler?r=1egdky

[3] t2t from https://github.com/guitarvydas/t2t

[4] OhmJS from https://ohmjs.org

[5] Parsing Expression Grammar from https://en.wikipedia.org/wiki/Parsing_expression_grammar

[6] Jack W. Davidson, and Christopher W. Fraser. The Design and Application of a Retargetable Peephole Optimizer.

[7] J. R. Cordy. An Orthogonal Model for Code Generation.

Paul’s Substack

Discussion about this post