RT - Now What?

Towards Higher Level Syntax for Programming Languages 2024-12-07

Dec 09, 2024

Goal

The point of this note is to step back and to take a deep breath, trying to figure out what to do next with the RT project.

The RT project consists of a rough-in for a new VHLL - Very High Level Language - and a transpiler from RT to Javascript (and Python, and Common Lisp).

In essence, RT is a compiler for a language that uses existing languages - Python, Common Lisp, Javascript - as assembler[1].

I think that the next step is to hoist the compiler and to make it work in a browser. Yet, I wonder if I’m thinking inside-the-box. Suggestions (and help) appreciated.

RT makes it possible to run message-passing, mutual multitasking software components in Javascript.

RT makes it possible to use a DPL - Diagrammatic Programming Language - to create software components and to run them using Javascript.

Status 2024-12-07

In the above diagram, the green-coloured files contain code written by a programmer.

The Leaf components are text-based code written using a standard programming editor in the RT language.

The Container components are DPL-based code written as diagrams[2] using the draw.io[3] editor. Draw.io saves diagrams out in XML[4] format. Such output is culled and converted to JSON, then fed as input to the Larson Scanner application during execution. In essence, the JSON constitutes a graph / routing table between instances of Leaf components. Leaf components cannot communicate directly with one another, nor call can they one another. Leaf components can only generate output messages that are routed by their Containers to other components (Leaf and/or Container components).

The 0D kernel consists of about 1,200 lines of RT code, compiled to about 1,300 lines of Javascript code. The kernel code contains some 11 stock components along with the machinery to instantiate components and to send and route messages between software components.

Das2json

The conversion of .drawio files to .json files is performed by program called ‘das2json’. Currently, this program is written in the Odin programming language1. It consists of a straight-forward use of an XML parser library (part of the Odin off-the-shelf stock library). As it stands, the ‘das2json’ app is compiled to run on a Mac Mini M3 and on Mac Intel. Given that ‘das2json’ is written in Odin, it should be possible to recompile it for other operating systems, such as Linux. The original source code for ‘das2json’ is contained in the 0D repository[5]. It should be easily possible to write ‘das2json’ in t2t[6], making it available to many more architectures (while, also, demonstrating how to write an XML parser in t2t).

In any case, since ‘das2json’ does so little work, the absence of a working ‘das2json’ program does not preclude the use of the RT transpiler - one can simply use the existing .drawio.json files and comment out the parts of the build script (‘rebuild.bash’) that invoke ‘das2json’.

Rt2js

The ‘rt2js’ program is described in more detail in[7].

Rt2js is a simple pipeline that leans on t2t technology[6] to transpile (“compile”) RT source code into legal Javascript.

As shown in the above diagram, two kinds of RT code have been written:

The 0D kernel
Larson Scanner Leaf components.

The 0D kernel, once written, does not need to be rewritten, other than to clean it up.

Every project, though, like the Larson Scanner, needs to have customized Leaf components written for it. Leaf components simply consist of several lines of source code written in RT. The average size of Leaf components in the Larson Scanner is about 25 LOC. Furthermore, the Larson Scanner source code contains 2 simple diagrams containing 15 instances of 6 different kinds of components. Container and Leaf components are like classes in popular programming languages, and, can be instantiated multiple times in a project.

T2t

The text-to-text transpiler - t2t - inhales source text and exhales different text.

Thus far, I’ve mostly used t2t to rewrite programming source code from one language to another, like in this RT project, but, t2t can used to rewrite any text. This would be like using a code editor that had a powerful search-and-replace feature, e.g. emacs regular expression search and replace (but, better).

In the past, I have, also, used t2t ideas to rewrite text that is not program source. For example, in kinopio2md I rewrite Kinopio mind-maps, exported to JSON, as markdown (.md). The idea was to brainstorm ideas, leaving them in point form in Kinopio, then rearranging the ideas into point-form markdown format and feeding the result to ChatGPT for turning them into full-prose blog posts.

The inhalation step of t2t is performed using a grammar written in the syntax of OhmJS. I use file suffixes ‘.ohm’ and ‘.grammar’ interchangeably for such grammar text files.

The exhalation step of t2t is performed using a rewrite spec written in a custom DSL[8]. The DSL, itself is written in OhmJS[9], with semantic operations written in Javascript (maybe the last time that I got Javascript under my fingernails).

The rewrite DSL makes it unnecessary to write Javascript code to accompany OhmJS grammars. The rewrite DSL provides a very limited set of operations - just enough to create strings using constant text, string interpolation and calls to a support library.

Rewrite does the dirty work of walking the parse trees created by the OhmJS parser.

The capabilities of rewrite seem to be very restrictive, but, these restricted capabilities are provably enough to write new programming languages with. I’ve been using rewrite for several years and haven’t found it to be wanting. In fact, the restrictions of rewrite provide more freedom in design of projects, allowing one to design without needing to worry about niggly implementation details and premature optimization and premature typing.

PLWB

PLWB is an experimental programming-language-workbench. It currently consists of a jury-rigged combination of 3 windows:

Command line, the choreographer running in a terminal window
IDE - browser
Drawing editor - draw.io.

There are many “moving parts” in this workbench. It is in need of a general overhaul to make it more widely accessible as off-the-shelf software, but, it plants seeds for what kinds of things can be done with our current level of hardware (2024 developer laptops and operating systems).

PLWB is a poor-man’s REPL - Read Eval Print Loop. You can edit a DPL program in draw.io, then see results in the IDE instantaneously.

These ideas allow us to strip down our notion of programming languages to include only the necessary features, allowing us to drastically reduce cognitive loading during software development.

PLWB is, currently, included in the rt repository. I expect to break it out into its own repository over time and further use. The germination of ideas that led towards PLWB are described in other articles[10].

Larson Scanner

The Larson Scanner was first seen in television shows like ‘Knight Rider’. It consists of a row of lights that turn on and off in sequence. An electronic version can be seen in kit form[11].

In this variant, to keep things simple, the lights are either turned on or off, with no in-between brightness levels.

This software version runs on the command line, printing out a numeric digit of which lights have been fired. To make the result more palatable, spaces are used to pad the output.

The source code for the software Larson Scanner consists of 2 diagrams:

The top level, containing the main ‘Larson’ sub-component and a delay Leaf component.
The ‘Larson’ sub-component. A Container component that composes 3 Leaf components with 10 monitor Leaf components.

The diagram compiler distinguishes only between rectangles and arrows. Colour and size don’t matter. For example, I’ve drawn the monitor components as small red rectangles while drawing all other components as blue, larger components.

Rounded rectangles, possibly containing text, are used to represent ports on components. Input ports are white, whereas output ports are turquoise.

Input and output ports of Containers are drawn as rhombuses.

Some arrows are drawn using solid lines, others are drawn as dotted lines. To the compiler, there is no difference, but, I believe that the resulting diagrams are more meaningful to human readers. A similar readability effect can be seen in many textual programming languages that use indentation to visually signify scoping and colourization of text to differentiate various kinds of identifiers and keywords.

In this design, the main scanner is driven by a succession of ‘tick’ messages. At the top level, a Delay component is used to feedback ‘tick’ messages into the scanner, once started.

Main Body

Larson Component

The main body of the scanner consists of a loop containing a counter and a decoder. The counter reverses direction every time when the decoder fires the 10th output port.

Monitor components are hooked to the decoder outputs. The monitor components display a digit on stdout of the terminal window.

The ‘reverser’ component has two inputs ‘J’ and ‘K’ which cause a data-less message (a “bang”2) to be fired out of its output. When the ‘Count’ component receives a “bang” on its ‘rev’ port, it sets its own internal state to count in a direction opposite to that in which it was counting.

The ‘Count’ component has a simple counter in its internal state. When a “bang” message arrives on its ‘adv’ input port, the counter is incremented and the current value of the count is sent out of its output port (that port has the empty string “” as its name). When a “bang” message arrives on its ‘rev’ input port, the internal state is modified to increment/decrement the counter in the opposite direction.

As can be seen from the diagram, input ‘tick’ events are routed to the ‘adv’ port of the ‘Count’ component. The diagram does not show where such ‘tick’ events come from, except that they arrive on the input gate (input port rhombus). We know, from examining the upper level diagram, that in this design, ‘tick’ events come from the startup code and from the feedback loop that involves a Delay component.

One could re-design the circuit by moving arrows and components around. The underlying goal of 0D is to make components that are completely isolated from one another, to allow layering and flexibility during the design stages of a project.

What Next?

Given the existence of the above, what should the next goal(s) be?

Suggestions would be most welcome.

It appears that it would be trivial to move the ‘main.js’ component into RT, making the whole project use RT and generated code.

Maybe ‘das2json’ should be completely rewritten using the PLWB?

Probably, the project should be “lifted” to work in browsers. I don’t know enough about HTML, CSS, Javascript to imagine how to do this well, nor how to implement it well.

Maybe an RT to WASM emitter should be written. Again, I don’t know enough about WASM to do a reasonable job.

Maybe the Common Lisp version should be ported to emit Scheme code, allowing RT to be used in itch.io ‘Lisp Game Jams’?

Maybe more projects should be built in RT and made to work? For example, maybe E(C)S coupled with Urlang intended for use in itch.io Lisp Game Jams?

Maybe the peephole optimizer, included only in the Common Lisp version, needs to be expanded to show how text-to-text peep-holing can be implemented?

Maybe Sector Lisp[12] or BLC[13] should be implemented in RT, making them available for inclusion in any Javascript project?

A few years ago, I ported Nils Holm’s Scheme code (implementing a Prolog) to Javascript using OhmJS. Should this be re-done using t2t? Q: Why would one even want Prolog? A: For exhaustive search using a declarative syntax3, without the need to write many nested, buggy loops.

Repositories

rt[14]
rtlarson[15]
t2t[6]
0D[5]
Article about building software using black boxes[2]
PLWB beginnings[16]

Bibliography

[1] In a 'real' Computer Science, the best languages of an era should serve as 'assembly code' for the next generation of expression (Alan Kay, 31:50) from

[2] Building Software Using Black Boxes from https://programmingsimplicity.substack.com/p/building-software-using-black-boxes?r=1egdky

[3] Draw.io from https://app.diagrams.net

[4] GraphML from http://graphml.graphdrawing.org

[5] 0D from https://github.com/guitarvydas/0D

[6] t2t from https://github.com/guitarvydas/t2t

[7] RT Transpiler from https://programmingsimplicity.substack.com/p/rt-transpiler?r=1egdky

[8] Experiments With Text to Text Transpilation from https://programmingsimplicity.substack.com/p/experiments-with-text-to-text-transpilation?r=1egdky

[9] OhmJS from https://ohmjs.org

[10] Building A REPL in 2024 from https://programmingsimplicity.substack.com/p/building-a-repl-in-2024?r=1egdky

[11] Larson Scanner Kit from https://shop.evilmadscientist.com/productsmenu/152

[12] LISP with GC in 436 bytes from https://justine.lol/sectorlisp2/

[13] BLC Binary Lambda Calculus from https://justine.lol/lambda/

[14] RT repository from https://github.com/guitarvydas/rt

[15] Larson Scanner repository from https://github.com/guitarvydas/rtlarson

[16] Programming Language Workbench - Beginnings from https://programmingsimplicity.substack.com/p/programming-language-workbench-beginnings?r=1egdky

Paul’s Substack

Discussion about this post