Superego: a Malleable Personal Computing Platform

[pscanf]

Hello! I’m the creator of Superego, an open-source personal database that I think qualifies as malleable software.

I started Superego because I wanted a place where I could gather all my data so that I could then slice it and dice it however I wanted. For example, making charts with custom stats, identifying correlations, easily searching for old records, etc. (As you can tell, I’m a data geek).

The software I had in mind was first of all a general-purpose database where I could store every possible piece of information about myself in an organized manner (i.e., enforcing a schema for each type of data). But I also wanted it to be a hackable platform for small apps that use my data. That would allow me to build a personal dashboard, a specific “input view” for a particular type of data, a hook to make a custom automation, etc. Back in March, I wrote a bit more about my vision in this blog post–Own Data, Own Software.

Fast forward a few months of intense work, and now I have an actual implementation of that vision. It’s still not everything I want it to be, of course, but it’s a start.

Along the way I also succumbed to the AI hype. Initially I wasn’t really giving much thought to AI, but then–as an experiment–I tried integrating an LLM to answer questions about the data, and it blew my mind that I could actually get interesting and meaningful insights, so I made it a permanent feature.

From there, the AI integration started spreading. Now I’m using it for many different things, but the most interesting uses from the “malleable perspective” are:

• It simplifies the creation of collections in the database. As I wrote above, I wanted the database to enforce a schema for each type of data, so creating a collection in the database involves defining the schema, which is a JSON object similar to JSON Schema. Creating it “manually” is basically impossible for non-developers, but with the AI you can do it by chatting, which is of course much easier.
• It simplifies the creation of apps. Apps in Superego are snippets of React/TypeScript code. Again, basically impossible to create manually for non-devs, but it’s ~easy with AI.

The Ink & Switch essay on malleable software talks about the role AI can have in smoothing “the slope to tailorability”, and I think Superego demonstrates this quite well. It’s still not at the point where my parents could make their own apps, but it’s definitely accessible to users at the “create Excel formulas” level of expertise.

This is a long blurb already, so I’ll stop here, but of course I’m happy to answer questions.

Btw, you can try Superego online at https://demo.superego.dev/. (It’s a desktop app, but–for the demo–I also made it work in the browser using IndexedDB as the data store.)

P.S. I’m brand new to the community, which I actually only discovered a couple of weeks ago! I hope this is the right place to post this. Short self-intro: I’m Paolo, dev/consultant from Italy, living in Vilnius, Lithuania. I’ve always been an OSS enthusiast and contributor, and over the years I’ve worked on many unpopular open-source projects.

[Offray]

Hi Paolo and welcome!

I have just started to see at Superego, trying to see the overlap between it and wiki engines, as my last malleable project, Cardumem, is a wiki engine.

I like the idea of a conversational interface (but I don’t know if I will put it as a default) and with it, I was able to easily create a new collection called “Wiki snippets”, but adding further content was difficult as the sourceUrl is required and left it in blank (that was the AI response) seems non-intuitive or required, particularly if our collections are of thoughts, text snippets and so on (Superego seems to imply the idea of everything has an external URL).

I have wondered how to add progressively a conversational interface to Cardumem, trying to keep its minimal computing requirements and maybe something like SQLite-Vec or Vector SQLite is good enough. Given that you’re using SQLite, have you thought in using this tools? And related with malleability, Could the storage and AI engine of Superego be used in other projects that doesn’t work with Typescript as its programming language?

Cheers,

[pscanf]

Hello Offrey, thanks!

I have just started to see at Superego, trying to see the overlap between it and wiki engines, as my last malleable project, Cardumem, is a wiki engine.

I can see some overlap with personal wikis, though it might be incidental. I mean, Superego aims to be a general purpose database, so it can be also used as a personal wiki. However, it doesn’t have many of the defining features of a wiki (I’m thinking things like cross-page links, TOCs, different page layouts… but I’m not a wiki expert, so my idea of “what makes a wiki” might be off).

Thanks to its malleability, you can build an app inside Superego that implements those features, but it’s not “native” of Superego, if that makes sense.

But I’m interested to hear your point of view on this.

I was able to easily create a new collection called “Wiki snippets”, but adding further content was difficult as the sourceUrl is required and left it in blank (that was the AI response) seems non-intuitive or required, particularly if our collections are of thoughts, text snippets and so on (Superego seems to imply the idea of everything has an external URL).

Collecting thoughts, text snippets, etc seems like a very fitting use case for Superego. There is no baked-in concept of an external URL, so it might just be the specific collection that the AI suggested for you to create that required it, but you can make it not required or skip it entirely.

I agree that collection creation is a bit unintuitive. Maybe I should tell the AI to explain better what it entails. But if you want we can have a call and we can review the collections you created and figure out how to make them more fitting for your use case.

I have wondered how to add progressively a conversational interface to Cardumem, trying to keep its minimal computing requirements and maybe something like SQLite-Vec or Vector SQLite is good enough. Given that you’re using SQLite, have you thought in using this tools?

I’m actually right now adding search to Superego. I have considered both, but sqlite-vec seems to be abandoned, while sqlite-vector is not OSS, so I discarded both. I am also considering Turso, which is SQLite-compatible and has native vector support. But it’s still missing some SQLite features I use, so I can’t make the switch just yet.

Though actually I’ve also re-considered vector (semantic) search as a whole, and for now I’m just implementing good ol’ lexical search. For what I’ve seen it’s much simpler, faster, requires less storage, and the results most of the times are just as good.

Could the storage and AI engine of Superego be used in other projects that doesn’t work with Typescript as its programming language?

Not really, or at least not at the moment. I was thinking of exposing the core functionalities through a localhost HTTP API, so you could have Superego running in the background and another app using it as its database. But for now it’s just a possible plan for the future. Rust-ifying the core and making it a language-agnostic embeddable component would also be cool, but for now I’m focused on developing the user-facing functionalities to see if the whole concept is sound / interesting / useful.

[Offray]

@pscanf

Collecting thoughts, text snippets, etc seems like a very fitting use case for Superego. There is no baked-in concept of an external URL, so it might just be the specific collection that the AI suggested for you to create that required it, but you can make it not required or skip it entirely.

I agree that collection creation is a bit unintuitive. Maybe I should tell the AI to explain better what it entails. But if you want we can have a call and we can review the collections you created and figure out how to make them more fitting for your use case.

I have learnt to non trust in AIs (or Apparent Intelligence as I call it) by default. But maybe Superego users will think not that the default data template it gave them is wrong, but that the tool is enforcing too much structure by default. Maybe some templates of loose structure data, as most GUIs offer could be a better starting point.

Regarding my use case, as I’m building my own Lua based wiki engine, maybe is too specific for Superego, and so I was looking if some of the SQLite backend technologies could be an intersection point in the future as I’m using Fossil, which uses SQLite in the backend too. But Cardumem is pretty pre-alpha for a meeting ATM. I’ll share advances here to see it there are some intersections when it is more mature.

@pscanf

sqlite-vec seems to be abandoned

Despite last commit being 11 months ago, I remember listening a podcast (not sure how old) from Alex Garcia, its creator and it seemed like a pretty active project, at least, his website says so.

@pscanf

Rust-ifying the core and making it a language-agnostic embeddable component would also be cool, but for now I’m focused on developing the user-facing functionalities to see if the whole concept is sound / interesting / useful.

Cool. The project seems interesting despite of being too far apart of what I’m looking in UI/UX and tech stack for my minimalist and extensible wiki engine.

Now that the is this rustify everything fashion, I wonder if Nimifying things could be and alternative for high performance, self contained apps, without all the Rust borrow checker overhead. But again it can be that we are called by different language/stacks aesthetics (Lua, Nim, Pharo in my case and JS/TypeScript, Rust in yours).