Ink & Switch malleable software essay

If you wish to make a malleable system from scratch you must first invent the universe.

When Carl Sagan talked about making an apple pie from scratch, I think he meant that it’s impossible: everything depends on everything else.

In any creation project, you have to get something from someone else.

A pie needs flour. If you want to make your own flour, you’ll need wheat, and milling equipment. To get that, you’ll need farms and metal, which needs soil and minerals. To truly say you made it, you’d have to make the dirt and minerals. To make iron you’ll need a star to form iron out of lighter elements.

And to make a star.. All the way back to the Big Bang.

In the physical world, we’re not so evolved yet to snap our fingers and let there be light. But in the software world - and other fields like mathematics and literature - it is possible to make an apple pie from scratch by inventing a universe. Not the universe, but a standalone microcosmos in the sense of MicroWorlds and Knowledge Machines in Seymour Papert’s research.

An example is Drifblim, the Uxntal assembler in the Uxn ecosystem. Uxn programs are written in the Uxntal language, whose assembler is self-hosted.

..written entirely in Uxntal, doing away with any dependencies beyond the thin emulation layer of the virtual machine. The assembler is written in the language it assembles and is bootstrapped from a minuscule binary seed.

I find it conceptually beautiful. Like John McCarthy’s original Lisp evaluator in Lisp - as Alan Kay called it, the Maxwell’s equations of software.

The code for eval is a beautiful expression of the core ideas of Lisp, written in Lisp.

..Just as with Maxwell’s equations and electromagnetism, there is a sense in which if you can look at this compact little program and understand all its consequences, then you understand all that Lisp can do.

And because Lisp is universal, that means that inside these few lines of code is everything that a computer can do.


Another example of inventing a universe is Mu.

(mu) means “nothing” or “emptiness” in Japanese (wu in Chinese).

File:無-cursive-order.gif

It’s funny how a word for nothing requires so many lines to represent it. Originally it was used for the word “dancer” (wu), but the character came to be used for “not, without” (wu). A more intuitive symbol is the empty circle, called Enso.

Compilers are often written in the language they are compiling. This creates a chicken-and-egg problem that leads users and distributors to rely on opaque, pre-built binaries of those compilers that they use to build newer versions of the compiler.

To gain trust in our computing platforms, we need to be able to tell how each part was produced from source.

Bootstrappable Builds

Stage zero is a way to bootstrap an entire operating system from scratch - from hex and macro assemblers, to C compilers and POSIX-compatible tools (stage0-posix). It looks insanely difficult though.


The other day I read an article, Writing Toy Software Is A Joy.

Some will tell you to avoid reinventing the wheel, but they’re wrong: you should build your own wheel, because it’ll teach you more about how they work than reading a thousand books on them ever will.

..The beauty and craft of writing software is being eroded. Software development needs more simple joy, and I’ve found that creating toy programs is a great way to remember why I started working with computers

From the list of toy programs the author has attempted:

  • Regex engine
  • x86 OS kernel
  • Gameboy emulator and/or game
  • Physics engine
  • Dynamic interpreter
  • Compiler for a C-like language
  • Text editor
  • Async runtime
  • Hash map
  • 3D graphics render engine
  • Threaded virtual machine
  • GUI toolkit
  • CHIP-8 emulator
  • Chess engine
  • POSIX shell

That’s plenty material for many microcosmoi (plural). But the creation of such will require a deeply technically fluent person - unless they create a universe in which it’s easier for a less technical person to create their own universe within it. Like certain cosmologies where lesser gods rule over the lower realms.

My point, if there was one, is that even a tiny toy universe can be self-complete and free in a malleable sense, within its space and set of rules.

8 Likes

Damn Eliot, once again, you’ve articulated and curated precisely how I feel about a topic, in better words than I could. I just just would like to say thanks for all these amazing post you’ve made across many of these threads recently.

This post of yours makes me think of the appeal of finite field arithmetic.

3 Likes

I would say that creating a microcosmos is not that interesting, a language that is Turing complete.
The interesting part is the type of language, its properties. Domain specific languages are very useful.

It would be good to construct a language / formal system for both computation and physical systems , for cyberphysical systems. One such system is dynamicland, where computation is affected by the physicality of the medium. A language that describes only the internals of the computer cannot really help here.

For dynamicland, i dont know if it is useful, but as @khinsen pointed, the effectiveness of cyberinfrustructure is dependent to socioeconomic factors / dynamics, there it would be really useful to have such a language.

Regarding the joy of creation, i think that Joy and utility are two unrelated things. It can be really fun to create a language like Malbolge but it is not very useful.

it would be good to have both joy and utility.

3 Likes

Worth mentioning in the context of bootstrapping: GNU Mes, where “Mes” stands for “Maxwell’s Equations of Software”. It’s a small-kernel bootstrapping stack for the actual messy software universe that most of us use everyday, it’s top level being GCC. It’s actually used by at least one Linux distribution, GNU Guix.

3 Likes

It is though it should be noted that they’re retrofitting bootstrappability on to a massive system that was never built with it in mind.

2 Likes

Exactly. I’d love to see someone try building a lean bootstrapped system that can run existing software, possibly with some adaption of the OS interface, but doesn’t aim at having all the features and cruft of a modern Linux system. A reasonable migration target for a leaner and simpler world.

1 Like

that can run existing software

Some or all?

Enough to make migration a realistic option for enough people to make the system viable. That would probably be a somewhat homogeneous audience. A specific application domain, for example. I see the point in demonstrating that such moves are possible.

1 Like

Amazing, the project did achieve bootstrapping of the entire GNU/Linux stack “from scratch”. I didn’t know this - so pleased and impressed to learn about it.

GNU Mes is a Scheme interpreter and C compiler for bootstrapping the GNU System.

It was created to address the security concerns that arise from bootstrapping an operating system using binary blobs, which is common practice for all software distributions. Mes was inspired by The Maxwell Equations of Software: LISP-1.5 – John McCarthy page 13..

The Scheme interpreter is written in a simple subset of C. The C compiler is written in Scheme and comes with a small, bootstrappable C library. These are mutually self-hosting.

..Starting from this 357-byte hex0-seed binary provided by bootstrap-seeds, the stage0-posix package first builds hex0 and then all the way up: hex1, catm, hex2, M0, cc_x86, M1, M2, get_machine (that’s all of MesCC-Tools), and finally M2-Planet. Mes can now be bootstrapped from M2-Planet and Mescc-Tools.

Mes can build TinyCC that is self-hosting. Using it and the Mes C library we can build an ancient version of the GNU tools triplet: glibc, binutils, gcc. This is enough to bootstrap Guix for i686-linux, x8664-linux, armhf-linux and aarch64-linux.

The final goal is to help create a full source bootstrap as part of the bootstrappable builds effort for any UNIX-like operating system.

As I write this comment I’m running the build process for Mes in a side terminal. It’s currently compiling lib/stdio/vfprintf.c. Watching the C and Scheme files (mostly C) scrolling by, it gives me a renewed sense of respect for the decades of collective thinking that went into the system.

How cool is it that I can see all the code that compose the entirety of it, down to the tiny “seed”. For x86 architecture it’s this file, POSIX/x86/hex0_x86.hex0. I’ll leave that as reading material for further study.

3 Likes

I believe Doug Engelbart did move us in that direction, till we abandoned it for short-sighted convenience and commerce. See my post in the catalog on nLS/Augment.

3 Likes

Exactly! Beyond its practical utility, Mes is a good example for knowledge refinement in computing. I wish we had more software that could take this role. Better yet, a collection explicitly aimed at showcasing such stuff.

2 Likes

Reminds me of Olia Lialina’s call to respect the role of “General Purpose Users” in Turing Complete User.
She paints a very unflattering view of U* Design as a discipline, which I don’t feel is fair, but she also raises interesting criticism on a general trend.

1 Like