This is a very unformed idea that I’m trying to crystallise into a better shape, or any shape at all.
It’s been haunting me for decades, though. Probably since the era of OS/2 2.0, and my disappointment when I first saw its “Workplace Shell” and found that it wasn’t like a shell at all. That is, you could use graphical objects, even send messages to them… but you couldn’t make your own, at least not interactively.
What I’ve wanted since the 1980s is a GUI where the components are “text-like”, by which I mean:
-
A window, or any other graphical component, is composed entirely from a set of smaller graphical components. Those components can all be readily seen, and they can be also be individually selected, copied and pasted. There is nothing in fact to a window other than all its individual visible components. There are, specifically, no hidden elements: no code, no configuration, no binary files. Everything you see, you get, and only that.
-
Obviously this wouldn’t get us to parity with a full GUI toolkit, because you wouldn’t (at least at first) be able to program your GUI objects. This would just be a view layer. So if there were programmed, reactive components, with configuration or code or hidden state, we’d probably want some kind of “control view” (like right-click, only more powerful) which would let us view this state. And however we viewed it (not necessarily as text - perhaps as some kind of circuit diagram), that view would also be text-like, composed of components each individually selectable.
Perhaps this idea could be called a “symbolic GUI”, because it would be composed of symbols rather than text. But “symbolic” is such an already overloaded term. “Symbol-oriented” maybe?
The important part of this idea is the selectability of all the subcomponents. Probably with some kind of grouping construct (eg a box, as the equivalent of brackets). Components would be not necessarily of a fixed size, but they wouldn’t be arbitrarily sized either. They’d probably expand to fit the size of the components inside them.
To achieve this, we’d need at least three things:
- A set of symbols, with a basic sort of “visual grammar” on how they can connect (eg: left/right, up/down, inside/outside). The equivalent of Unicode, but for GUI elements.
- canvases which can display these symbols, and allow selection, grouping, insert, delete, copy and paste, analogous to text areas. Those canvases must be completely unrelated to the software which hosts them. You must be able to copy symbols to and from a canvas, ship them between systems, and have them render recognisably and with the same semantics.
- probably a strict, one-to-one mapping to and from Unicode, though that wouldn’t be necessary if we had decent interoperability between systems that wasn’t just Unicode, but we don’t.
- possibly multiple “fonts” which would change how the symbols display, but crucially, NOT their basic identity or behaviour. CSS almost does this - and yet also does too much.
This is the fork in the path of GUIs that I’ve been expecting to happen multiple times, and it didn’t. I first thought Workplace Shell might be it, and then OpenDoc, and then HTML, and then Java Abstract Window Toolkit, and then I thought surely KDE or GNOME would, and then maybe OpenOffice Draw… but that path still hasn’t forked, not in the simple, clean, standard, inter-system, composable way that seemed obvious to me as the next step from the 1980s.
Anyway, that’s the idea. Putting it down as a flag because it feels like something that surely has to happen at some point. Until it does, we won’t have an actual “graphical language” because we won’t have the linguistic equivalents of glyphs from which to build larger components.
HTML gets us halfway there. There is a textlike markup language that defines the things on the screen, but there’s also a lot of invisible stuff, and most annoyingly, you can’t select, copy or paste any HTML element from its visible representation and move it elsewhere. And HTML elements all have to exist inside a “page”, rather than on their own.
Postscript /PDF and TeX and OpenOffice XML also, infuriatingly, almost get there, and yet stop well before the point where they would close the loop.
It might be interesting to think about how many of the HTML elements could be repurposed to be completely free of the context of a “page”, and/or how many Unicode codepoints might be able to be used as graphical drawing elements, if things could be put inside them, or they could be used as links between other elements.