From the Web to a Metaverse OS (with no apps!)

General
1

Hi, this is my first post here, but I’ve been reading along for some weeks since I discovered you all. Some of you here will know me from the FoC Slack (hi!)

So, as the title says: “From the Web to a Metaverse OS (with no apps!)” - I think this is a topic of interest here (there’s A World Without Apps for example). I’d like to present my ideas on all that, briefly at first but hopefully you’ll prompt me to tease out more detail…

Tim Berners-Lee’s original Web introduced an innovation that I feel is still little understood or appreciated: links to data of standardised format, creating a single global graph as a 2D hypermedia space. It’s more than an innovation, it’s what I call an Inversion. This architectural model (REST) is the dual of RPC and its related tech (SOAP, web APIs, etc.).

I think we should embrace this radically Inverted model all the way down to the operating system!

It would be a 2D hyperdata OS, with links between standard data object types forming a single global graph or “data fabric”, hosted across all our devices and machines. Just as the base Web (without Javascript) didn’t have apps, this OS would also be app-free.

The Web’s URLs will need to be upgraded for this: to break down aggregate formats into little elemental standard types (paragraphs, calendar events, media objects, tasks in a to-do list, contact cards, and sequences of links to all these); to add liveness so that data object changes are propagated as they happen; to add true decentralisation through local-first or peer-to-peer interaction between OS instances.

These data objects interact by mutual observation - allowing spreadsheet-like interdependencies.

This architecture will contain all our little, live 2D digital property chunks linked up within and across local OSes, creating a single global space. A space that can then extend from 2D to 3D, effectively forming a single global scenegraph. A Metaverse or Spatial OS without apps!

What do you all think? I’ve got mountains of articles on this topic that I can provide links to for anyone interested in following the idea through in more detail, starting at About - The Reality Computer.

4 Likes
2

You’re definitely singing my song here! I’ve been dreaming of something like this for about 20 years now. Was expecting that surely some of the professionals (up to and including Tim Berners-Lee and his Semantic Web team) would see the obvious need for this and build it, so my bumbling amateur efforts would only get in the way. And then spent 20 years being shocked as the professionals charged off in the exact opposite direction, building the Cloud (ie a centralised 1970s Bell/IBM mainframe Net with slightly fancier dumb terminals) instead.

So I guess it’s up to us amateurs now.

A “data fabric”, yes absolutely. Lets build that.

My first thought was “so what if something like: HTTP but we fetch JSON instead of HTML, everything’s a worldwide soup of JSON objects”.

(Actually my first thought was “worldwide soup of RDF triples”, because that was Sir Tim’s vision, but I became convinced that RDF is one of the nastiest and least user-friendly data formats around, and the rise of JSON instead suggests that’s correct.)

My next thought was “but even JSON is pretty terrible, try writing a programming language over it, that would suck pretty hard. And arrays aren’t great for doing recursive parsing over. So what about maybe going full 1957 and doing a worldwide soup of Lisp S-expressions?”

My next thought was “well, we obviously don’t want a string URL for every cons cell, that’d be insane. And we need something better than servers and DNS and PKI. Probably a content-addressed store using sha512 or somesuch, which means we need to chunk all that data fabric up into blocks”.

And I haven’t managed to progress my thinking much further. Been watching to see if the IPFS or Nostr people might solve some important groundwork there, but both of those worlds have had their credibility damaged by being sucked into cryptocurrencies.

But I still think content-hashed read-only block storage as a publishing layer, then something S-explike on top. And some way of dealing with mutable block storage with RAM as cache, don’t quite know how to do that.

Most of the storage on my hard drive though right now would be immutable blocks, I reckon.

Creating roots of trust for a good alternative to DNS/PKI (please don’t tell me that Let’s Encrypt is any kind of answer to any kind of question) is very hard, yet must be done at some point. But maybe it doesn’t have to be done first.

A crazy part of me wonders: what if we leant really hard into the 1957 retro cons-cell model - or the Xanadu “tumbler” model, or the 1980s FidoNet mail routing model - and just used sequences of numbers instead of text URLs? If we want to run our desktops on this (and I do), then I don’t really want every mouse click or widget paint operation to have to construct a 100-character string URL and send that plus an XML document to a HTTP server implemented in Python or Node. I want a core that’s small, light and fast, and then I’d like to run that same protocol over a wire (with only as much encryption as necessary).

And further wondering: what if every cons cell was immutable, so the numbers just kept going up? Possibly not a great future-proofed solution, the node numbers would have to be pretty big to not run out, which is why a block-based system for at least the disk or net layer seems practical. But… well, testing ideas to destruction is fun. If we did have a block world at the disk/wire level and inside it at the RAM level a byte/node/cell world… at some point we have to deal with how to link the two. How to canonically handle byte or word-level addressing inside a very large and very sparse single address space (eg with “ROM pages” rooted in SHA hashes, and “RAM pages” rooted in Curve25519 pubkeys or disk volume GUIDs), but without wasting space.

2 Likes
3

Hi NateCull! Thanks for the reply. Good to know you’re thinking along similar lines! I basically agree with all you say there. You mention stuff around the efficiency of any tech implementation, which to be honest I’m largely punting off to the future in my own work, to free myself up to focus on researching more conceptual stuff! My own technical design is scattered across some documents, old ones published and local notes not, and, really, the code itself. I do need to write an up-to-date architecture, protocol, language design and object types doc!

1 Like
4

I think we need to figure out a way to move past find a concrete atomic structure for elements. Instead building a new set of primitives we should see if there’s some way to build decomposable pieces so that you may ship someone a pragraph as an atomic unit but they can build a URL that operates as a lens into a position in that paragaraph.

I’m afraid that if we allow for rich atomic components we’ll just see a new phase of “apps” like what’s happened to the web of today.

2 Likes
5

That’s an important point. It’s not enough to enable an app-less world, it must also be more attractive to most stakeholders than a silo-app world, otherwise apps will come back in a different technical incarnation.

Conway’s law seems the most important foundation of information technology. We have apps because our information technology is created by institutions that are app owners.

2 Likes
6

OK! Good points and challenges to address. But I think there’s a lot of mileage in the app-free computing journey and much to be achieved. We can deal with silo-heads with increasingly clever counter-moves as we build experience. I don’t underestimate the challenge of “The Inversion” - people have always lived in apps, even before smartphones with application programs on PCs. But if you believe apps are bad, you simply have to keep ploughing on building the alternative future.

3 Likes
7

The risk of sneaky re-app-ification of an upgraded web-like architecture shouldn’t stop us trying to do it better this time!

3 Likes
8

I’m thinking of starting a thread on the more general topic of “app-free operating systems”. Thus including Obenauer’s Items, Victor’s Dynamicland, TBL’s Web and SOLID (bit of a stretch to call that an OS?), Beaudouin-Lafon’s Substrates and Webstrates, Gebhardt’s Infocentral.

Are there any other entries for that list? Suggestions would be much appreciated (before I post, missing something obvious!!)

2 Likes
9

It really depends on what you require for an “operating system”. SOLID is a bit of a stretch indeed, but listing both the Web and technologies that depend on the Web (SOLID, Webstrates) looks weird in general.

I’d definitely add the two grand-daddies: Smalltalk and Lisp Machines. Emacs is a more difficult question: is it an OS by your definition, given that it sits on top of something like Unix?

1 Like
10

I’m thinking anything that is a means for general purpose access to everything a computer offers, systems that could be implemented as operating systems even if they’re not yet. So a browser has often been talked about in these terms, plus witness Chrome OS. Things that build on the Web (SOLID, Webstrates) may do this better and are distinct projects; so having all of that doesn’t actually look “weird” to me at least!

Maybe I should start by describing what I feel are common reasons people have for wanting to dump apps in an OS. I have written an article with some thoughts around this here: Apps considered harmful - by Duncan Cragg

Yes, I was going to include Smalltalk, and Brad A. Myers’ Open Data Model, and all that OLE stuff. But I see these as systems for smashing up apps into smaller apps, rather than eliminating them entirely! An object or co-operating cluster of objects still feels very much like an app, or a micro-app, to me. OO and these other component and interop models still suffer from all the problems that are the reasons I personally would want to have an app-free OS: hiding the data and allowing it to be arbitrarily defined and re-defined, inscrutability or arcanity of the imperative code, custom interfaces or APIs that can change too easily and which don’t interoperate universally or only in limited ways - friction beyond what most normies would tolerate!

I don’t know enough about Lisp machines (or Emacs) to know how they handle processes with respect to data, perhaps you could elaborate?

1 Like
11

Jack Rusher and I recently had a conversation on what constitutes an Operating System @jackrusher.com on Bluesky

He uses it more for the hardware abstraction layer and I use it more as the top-level user interface into the computer. I may switch to start saying “platform” to try to avoid confusion.

4 Likes
12

Yes! In my own personal notes I say “OS+” and “OS-” for the shell and kernel respectively. And “OS*” for the whole stack. We do need to be explicit.

In this app-free OS context, it’s definitely manifest to the user in “OS+” or shell level that there’s no apps, but in fact this app-freeness or process-freeness can drop all the way down into the kernel, which is responsible for the processes and threads that appear as applications. One day I hope to build a purely reactive, asynchronous kernel where threads are used to animate data, not user programs or processes. I did start building that long ago, but it’s hard work hacking through the Linux Kernel!! For now I have my “OnexKernel” repo that holds the code that one day I’d like to merge down under the SysCall interface. It handles persistence, networking, threading, cacheing, memory, low-level i/o and so-on.

2 Likes
13

Oh, and of course, there’s no window manager, display manager, display server, desktop environment, etc., in my “OS+” layer: just Vulkan piping into direct render!

14

Ahh… I am with Jack on this: an OS is a software layer that provides reasonably portable affordances for application software. Which can include UI affordances, and from there it isn’t far to the other point of view.

15

Both Smalltalk systems and Lisp machines have a single unified UI for which you write extensions that add functionality. These extensions are not siloed and can freely access each other’s data. That’s why I wouldn’t call them apps.

Lisp encourages the use of basic data structures, with code written “on top” of them. Smalltalk encourages a view of data as accessible through a specific object’s API. But this is a difference in degree rather than in kind. Much modern Lisp code uses CLOS (the Common Lisp Object System) in a very Smalltalk-like way, and Smalltalk has basic data structures (in particular the Collection classes) implemented as objects but used exactly like Lisp’s fundamentals.

2 Likes
16

In my opinion, it is impossible to program without limiting the scope of variables, of data.

It is extremely dangerous and actually eventually impossible, even for a single human being, because of all the unintended consequences.

What we need is to be able to view at all the levels of abstractions, and to change code at all the levels of abstractions.

2 Likes
17

With regards to type theory, univalent mathematics, it has only being recently that equivalent types are equal. (In standard Agda, we have types by name,(nominal))

Let us say that we have two different code bases with two different types. If the types are equivalent , then they can work together.

18

This sounds closer to my interpretation than Jack’s to me :sob:. My interpretation of his defn is that it’s just about managing hardware resources and scheduling. So just the kernel.

19

Well, then I am somewhere in between you two. For me it’s kernel + drivers + file system + network stack, for something similar to Unix.

20

Interesting, that’s something I have always wanted from type systems. Libraries being interoperable only if their authors chose the same names for common abstractions is not a good feature for building large systems.

What would be a language that is based on type equivalence?