Functions and Pipes

2025-01-26

Jan 27, 2025

Regarding the video “UNIX before Linux (1982)” (included below), the most import part is at 13:40.

The transfer of data from A to B is not a function call, it is something-else.

A function call is a 2-state state machine ([1]call, [2]wait-for-return), which is not what this video’s animation shows.

A pipe is not a function call.

In fact, software parts A and B are completely independent from one another, which means that the pipe is a queue (FIFO queue, not LIFO stack) and it implies no time-synchronization of A and B. Functions, on the other hand, use a LIFO stack and do imply time-synchronization - the callee is expected to execute immediately and to return, while the caller waits (“blocks” in operating system terminology).

The Bourne Shell has syntax for this kind of something-else (“|”). Shell syntax enables composing new software parts using this something-else instead of using function calls.

The drawback of the Bourne Shell is that it uses a Gutenberg-style textual syntax to represent pipes, which discourages the use of multi-branching and explains why goofy-less syntax applies to only 2 of the 16 (more now) pipes available to UNIX processes, i.e. only stdin, stdout, “<“, “>”, respectively. One has to resort to 2-letter, Gutenberg-linearized syntax for even something as ubiquitous as stderr, i.e. “2>”.

Programming Language Design

Programming languages could make pipeline-sends as syntactically-fundamental as function calls. Maybe something like f«x» in addition to f(x).

Programming languages are simply caveman IDEs for programming, heavily influenced by the meagre abilities of 1950s hardware. Back then, computers could only manipulate characters as grids of non-overlapping, non-resizable cells. Today, computers can manipulate vector graphics as resizable, overlapping windows and widgets. Representations, like SVG, allow us to manipulate graphical entities with existing text-based tools (like parsers such as Ohm[1]) in 3D and 4D (x/y/z/t).

Unfortunately, in the early days of UNIX, complete isolation of software units was implemented as heavy-weight operating system processes. The idea of pipes was tangled up with the notion of heavy-weight processes. The idea of pipes is essentially ignored in programming language design. Today, though, we have much lighter-weight closures. Queues take only a few lines of code to create.

For historical reasons, programming languages and operating systems - both being components of modern programming IDEs - have been kept separated. The few operating system ideas - like threads - that have leaked into programming, have been kept syntactically inferior to concepts like function calling. Programming languages have first-order syntax for function calling, e.g. f(x), but, only second-order syntax for thread libraries, e.g. explicit function calls to thread libraries.

Some constructs for asynchronousity have appeared in programming languages, like ‘async’ and ‘await’ and ‘.then’, but, are too heavily influenced by the desire to stick to Gutenberg type-setting.

In 1986, Harel proposed a graphical syntax - Statecharts[2] - for concepts like concurrency and multiple inputs and multiple outputs.

Prior to that, we already had a non-textual syntax for asynchronous parts with multiple I/O ports - electronic’s schematics.

1-in, 2-out

It's too hard to meaningfully say “1-in, 2-outs” in Gutenberg type-setting syntax, instead of using a little diagram.

Many other combinations of multiple inputs and multiple outputs are possible, but, discouraged by the use of Gutenberg type-setting syntax.

Feedback vs. Recursion

It should be possible to express feedback loops, but, textual syntax discourages this idea.

The incompatible notion of recursion is conflated with feedback.

Recursion is based on LIFO stacks, where the recursive call “jumps to the head of the line” and executes immediately, whereas feedback is based on FIFO queues where a message is placed “at the end of the line” and has to wait for its turn. The effects of each - recursion vs. feedback - are quite different.

Flow Based Programming

The idea of passing flows of data between independent programming units is not new.

One variant, called FBP[3], was put into production in the late 1960s.

Part Based Programming

I’m exploring part based programming which allows composition of software units similar in manner to the Bourne Shell, but at a lower, programming language level.

I call this 0D[4].

One of the aspects of 0D is the idea of creating 2 kinds of parts:

Workers (code, aka “Leaf components”)
Choreographers (recursive compositions of parts, aka “Container components”).

FP (Functional Programming) make use of only 1 kind of part: the function. Functions are workers and, at the same time, choreographers. In 0D, these concepts are pulled apart into 2 kinds of things.

The concepts of 0D are in production use at kagi.com.

VSH - Visual SHell

A nearly trivial side-product of 0D is the invention of a prototype Visual SHell called “VSH”.

Even in this trivial example, VSH allows fan-out (from the ps part to, both, the grep and wc parts). This isn’t as straight-forward to express using the Bourne Shell, nor with FP, nor with FBP.

The WIP code for VSH is at https://github.com/guitarvydas/vsh.