When trying to split a program into multiple parts, you need to serialize the data. Maybe this project can help in avoiding this.
I do need to sit down and understand this work more deeply, so Iāve added it to my reading queue. 
That reminds me, I recently saw a thread from one of the Gibbon authors discussing their 2019 LoCal paper:
1 Like
The link needs updating.
My recent attempt at a substrate led to the following discoveries regarding serialization and persistence.
In order to understand the following please note that I am writing it in Java/Kotlin and targeting JVM. The language inside the substrate is a simple functional language, currently Clojure. (Long live embeddable languages!)
The input events have no more complexity than a tree. Input cannot be a lattice (non-trivially), because I/O or perception is āflatā. Just think about your mode of perception. Your perception is a space, which can be divided into regions. Two disjoint regions of your perception space are independent. There is no let ... in in perception. There is also no pointer.
The input is also immutable and stored on disk. So, I would like to serialize the data in a compact way.
But then, according to my plan, the user also needs to maintain some state folded from the input. I initially used Protobuf Message.Builder (which is a mutable, typed message before serialization) for all the state. For a while I thought this style of mutability was necessary. Then,
- State requires more complexity than a tree. It can contain a loopy graph. Every off-the-shelf solution, including Protobuf, serializes this kind of data as a tree with references inlined, which makes the data structure wrong.
- A similar limitation was noted on LoCal page 58. LoCal can only serialize lattices, not loopy graphs. It must have used some equivalent of
let ... inconstruct. - If you want to store a loopy graph in a tree, you have to store the adjacency map. The authors of LoCal did not take note of this. We in fact have a capable design!
- State does not necessarily use the same data schema as input. This is the whole point of using
foldoverreduce, and is also the reason why I decided to leave the state serialization to the user.
This approach adheres to the āparse, donāt validateā principle pretty much everywhere, giving freedom to higher-level code.
In summary, loopy graph > lattice > tree in terms of complexity and serialization cost. LoCal can do a lattice. My substrate allows trees in input, and loopy graphs in state.
2 Likes
Gibbonās goal is about the automated compilation of functional programs, to use serialized data.
My interest is in the āautomatedā part, and in the probable ability to use it to split programs into smaller ones. I am not sure if the second case is something that they want to support.
And I havenāt looked into the technical details, so I cannot really provide any comment on that.