Semantic Runtimes

General
1

The team here at Yorba is building a transparent Clojure runtime that enables a common data language for any entity that interacts with our system.

The principles are patterned after existing Clojure conventions that often use keywords for things such as polymorphic dispatch and architectural descriptions (using Integrant).

These keywords are human-readable with machine-introspectable values. As you’ll see, they already allow us to communicate across multiple substrates (Redis, Postgres, Logging) and interpret data across multiple runtimes (Clojure servers, JavaScript servers, JavaScript web browsers)

A few highlights.:

  • Runtime data is specified and verified using a namespaced semantic convention (e.g. :yorba.member/name).
  • Clojure logs contain named components with dereferencable definitions (e.g. :yorba.member/name).
  • System cache keys follow the same semantic conventions, thus the data contained within is already defined (e.g. the value of :yorba.member.<member.id>/name is of the type :yorba.member/name).
  • Serialization/Deserialization specifications for how Clojure keywords turn into meaningful URLs to the outside world (e.g. :yorba.member.<member.id>/name ↔ yorba.member.1234/name)
  • Endpoint names that also follow the same semantic conventions (e.g. GET <base-url>/yorba.member.<member.id>/name is a :yorba.member/name).
  • Uniform data definitions (i.e. portable Clojure specifications) in logical locations (e.g. GET <base-url>/yorba.member.<member.id>/name/json-schema).

I think the benefits we’ve experienced of process decoupling and meaningful system-wide monitoring should be pretty obvious. Furthermore, we can plug in more runtimes and get meaningful system-wide dataflow with minimal effort.

A few other notes:

Metadata can be also be associated with the system components, e.g. the TTL for the :yorba.member.<member.id>/name cache values.

Much of our system ends up being defined in data next to existing specifications (such as a name is a string) in existing namespaces.

IMHO, these namespace semantics provide more expressive semantics than Niklaus Wirth-inspired modular semantics. And it’s absolutely critical that the namespaces are not tied to file path.

There are bigger plans for this, but this is what we have running in production today. Does it remind the folks here of other existing architectures? We generally take inspiration from Linked Data, but I’m also curious about more specific parallels.