Thanks for your replies! And yes, it’s great to see something like “Steam Engine Time” happening, where lots of people seem to be getting similar ideas.
To your points:
What is “safe enough to cope with the Internet environment of today”? Well security and privacy, etc., are “solved” separately by using the latest crypto goodness - you simply add read and write permissions to these data Items/Objects and enforce that on the wire. Or did you mean something else?
Yes, I meant a little bit more than that. I’m assuming that security “on the wire” is solved with crypto goodness. What I mean is more security between and among running objects, because “the wire” is oldschool; your own computer’s RAM is the new frontline for cybersecurity breaches. Especially if we’re going to share fine-grained objects around. Some fraction of those objects, maybe a large fraction, are going to be written by our adversaries. And they’re gonna run, because that’s what code does.
I’m talking things like Excel spreadsheet templates with macros in them: Microsoft thought they’d be perfectly innocent office automation scripts when they created that engine. They weren’t, however. And now everyone in every corporation gets strict lectures about “never open any email because there could be an Excel spreadsheet template inside”. We don’t want to have to be in that situation of telling people not to ever do the one thing that the computer was designed to do.
Evil objects will get into our machine, and they will run. What happens next will be up to the architecture of our VM and whether we thought ahead and prepared for this.
Our machines’ internal RAM being the frontline of world cyberwar means that we need to at least have a notion of “unforgeable pointers”. You can’t just iterate through RAM reading it all, you have to have been given a pointer/link, you can’t fake one up yourself. C and C++ don’t have this quality (big yikes), and neither do todays indie darling Forths (Retro, Dusk, Uxn, etc). That’s not good enough anymore and will continue to not be good enough - essentially, it’s like building a house entirely out of plastic with no fire retardant. Lisp and Smalltalk and Javascript and most other VM-based systems have this quality. I want to be sure that your system has this - that it’s not doing C or Forth like naked RAM access through integers.
However: if you have unforgeable pointers to objects in RAM, there will be a little bit of fiddliness there. You will not necessarily be able to export them as pointers and reimport them, the way the Forth people can. Or like oldschool 1990s Microsoft word did, just dump the whole RAM struct to disk and read it in again. Or not as integers anyway. You probably shouldn’t want to do that because that’s super dangerous. I think Python people are still doing this today, and it’s still super dangerous. You’ll definitely wantto convert all object pointers to quite large cryptographically secure integers (32 bytes might be enough) when writing on the wire, but you may need to do this even when you write objects to disk. Maybe. A virtual RAM system that keeps the integer representation of the pointers well away from ALL executing code might be safe enough. (But no machine code escapes allowed, ever! Not even for speed! Not even if you’re writing an operating system! That’s what killed Java in the web browser.)
Data sets: just create sequences, and sequences of sequences.
Yep, that’s basically where I’m at. I think sequences (although typed sequences - and your system has a couple of other things than types, it has an object ID and a “rule” ID which I guess is a bit like a class or function ID - which is why I suggest “sequences plus metadata, which is also a sequence”).
One nontrivial knob to turn here is whether by “sequence” we mean “array” or “Lisp-style linked list”. It matters quite a bit for some problems which one of those two kinds of sequences we pick. I’m not sure there’s a good decision rule as to which is best, though. Lisp lists are easier to write a memory allocator for (and prove that it’s correct), they’re better at sharing fine-grained structure (good for dense linking), they provide better security because absolutely every cell is an unforgeable capability. Downsides: they waste half your RAM, but RAM is cheap; they can only be iterated, so access may be slow; they maybe mess up cache, although garbage collection probably fixes cache; they do need garbage collection, but so does any object system.
You wouldn’t “take that data and copy/paste it anywhere else”. The OS implements link management, so you don’t copy paste whole Items/Objects (paras, docs, feeds) you just manage links to them.
Yep, that’s what I mean. No need for the OS/language/kernel to copy more data than it needs to. But there’d be some kind of user interface operation for selecting an object and “inserting” it as a link. The user might think of performing this reference operation as “copy/paste”.
There’s not really any “recursive cleverness” - storing the last state is just the object’s state. An object’s next state is simply a (not-necessarily-Turing Complete!) function of its current state.
Yes, but this part is still where the recursiveness cleverness comes in - because if your “animation rules” are functional/declarative, then they’re very likely defining that object’s state as a function of itself. Which is a function of itself, which is a function of itself… That’s a recursive self-reference. An ordinary functional language can’t handle that very well. Even a spreadsheet has issues with this - the dreaded “circular reference error”.
Obviously we know that we sometimes mean the previous state not the current state! So what I mean is that your underlying VM - and the language/calculus its based on, because I think ordinary lambda calculus won’t quite cut it - needs a built-in notion of “previous state” which it automatically maintains for all objects. And it will also need an initial state when it’s first created, because it won’t have a previous state. You’ll need an agreement about what that initial state should be. Probably a “Nil” or “Null” value of some kind might be good enough. There might also be some nontrivial complications around sequencing of recomputation events, to make sure that they don’t get out of order, and disconnecting/reconnecting from objects as they go in and out of scope… although the way you’ve described it to me, it does feel like that shouldn’t be too much of a hassle. But I know that the Functional Reactive world - stuff like React in web browsers - is full of weird mystifying things to do with timing, and baroque steampunk complexity around this, which it seems like it really shouldn’t be.
The state is always out in the open and visible. The behaviour or animation is potentially more hidden, and internal to any object.
Right, so my next question is: does the “behaviour or animation” inside the rules include any internal state of it own, or is it a pure function? I’d like it to be a pure function, I think. But it might potentially need some hidden state there. Because:
Having state out in the open and visible is modulo read and write permissions, of course. So again no more security holes than any other system.
The question is: how are you going to implement “read and write permissions”?
Because the simplest and most secure way is the “capabilities” way: “read permissions” at least would be handled by whether you have a reference to an object or not. And that reference would be a piece of state.
(Potentially, you might not need “write permissions” at the VM level if data ever only flows one way; but you might need some kind of convention for how an object discovers a new object that might like to suggest changes, and also how changes themselves are described. I think this would be something like a “transaction” or a “delta”… ie “add this field, delete that field, change that other field”… and that’s a whole another rabbithole I fell into for many years, because it seems surprisingly ill-defined exactly how to represent arbitrary changes to key/value kind of objects, let alone sequences.)
But if all state is always fully exposed to the world… well then you may be automatically leaking read permissions to all the world. You might not want to do that. You might in fact find that you can’t implement any model of “permissions” without some kind of hidden state, somewhere.
If your idea of “animation rules” already includes hidden state, then we’re probably on the same page. That objects - just like functions, in fact they would be functions that just have that magic “previous state” thing which is done for them by the VM - need public state (their computed value) as well as private state (their environment).
It would be very nice to be proven wrong on this suspicion and if everything could be done just with fully public state on all functions/objects. But not sure that it can.
Yeah, you don’t need files or file hierarchies: just sequences and links in a global graph of data that you can explore - like the Web, but of fine data not fat docs
Yep, very much this. I want my actual documents to be digested into chunks: chapters, pages, paragraphs, etc.
routing is actually proxy-cache routing: you put out a request for an object’s current state
Yes, this is I think how it would work at the top layer, running over today’s IP network. But I’m also thinking that a functional-reactive model, if it can be made simple enough (and “just cache the previous computed value of all functions and make it available as a magic variable” is possibly simple enough), could also describe the lower levels of a network. Such as the non-IP networks we find inside today’s computers. USB, north/south bridges, etc. Down to the transistors. Maybe. That’s the hope.
Edit: On timing issues, here’s an example. Suppose your animation rule is a function. That means, in order for an object to update its value during “one clock tick”, it has to make a function call and wait for the return value. But one function call could require an arbitrary number of expansions of subfunctions. Processing that function then may take more than one clock tick; the object then has to somehow freeze its value until the completion of its function evaluation. In the meantime, while it’s frozen and recalculating, it might receive multiple update events for its inputs, triggering further recalculations, none of which can be ignored because they all may depend on the state caused by the previous update. Also, any of the subfunction calls could potentially want to observe an object elsewhere, and we need to be sure that they don’t observe a version of an object that’s from a later clock-tick than when the recomputation began. This is still probably fine, we can’t avoid the time cost of computation, but there is the potential for becoming desynchronised if the order of events isn’t managed carefully.
But I’ll continue to noodle on what implication of that isn’t covered by my description. Where am I missing steps that are obvious in my head but not in the prose. You’re right that it’s very challenging to talk about stuff like this where you know almost everything you can say is obvious to others. Like the Turing machine point.
We’re discussing a different ill-posed term, not the ill-posed term I mentioned off the cuff merely for contrast. The PoMo-like point I’m trying to make is that this isn’t the ideal joint to cleave reality at.