The Willow Protocol

The Willow Protocol specification has just been released: https://willowprotocol.org/

From the “Comparison to other Protocols” page ( Willow Specifications - Comparison to Other Protocols ) Willow appears to be a rewrite / generalization of Earthstar, which was essentially a rewrite of Secure Scuttlebutt with somewhat wider goals.

Like Scuttlebutt, Willow users post updates of some kind and sign them cryptographically. Unlike Scuttlebutt, updates are not assembled into a linear “log” but into text string “paths” forming a namespace. And updates can be deleted, an important feature which Scuttlebutt lacks. (Although a “delete” mechanism for shared content can also very easily be abused, and I’m not sure whether Willow counters this.) Namespaces only exist under a cryptographic identity, so there is no global DNS-like system but presumably the intent is for various forum-like communities to emerge which manage their own namespaces.

Willow is also described as a “higher-order protocol”, in other words it is not a concrete protocol but rather a general framework for designing multiple special-purpose protocols.

Willow’s approach to data storage is… unusual. The authors state "Unfortunately, Willow requires some features not found in off-the-shelf solutions. " ( Willow Specifications - Three-Dimensional Data Storage )

My personal feeling is that Earthstar started out too complex and that Willow has become wider in its scope rather than simpler. I would like to see more iteration on this kind of protocol so eventually something will gel, but I’m not currently understanding what Willow brings that will help me create the kind of system I want to see.

There’s a Hacker News discussion (Willow Protocol | Hacker News ) which is more positive about the protocol than I am, so maybe there’s something there.

h/t Profile - andrew - Malleable Systems Forum

3 Likes

Yes, saw this on HN too, and kept some notes. Willow is also implemented in Iroh project.
(“Iroh is a protocol for syncing bytes. Bytes of any size, across any number of devices. Use iroh to create apps that scale different.”)

This is the Willow family of specifications:

  • Data Model: The fundamental data model, a system for giving structured names to bytestrings.
  • Meadowcap: A capability system for controlling access to Willow data.
  • Synchronisation: A network protocol for efficiently and privately synchronising Willow data.

I found it quite interesting that they have Meadowcap and cc’ed Christine Webber of Spritely on that (but got no response). I don’t know if both projects already have a collab going, but seems a good opportunity to cross-ref Willow on Spritely Community forum.

Willow is already mentioned on the Spritely forum, but not in reference to Meadowcap.

Update: Created Meadowcap: Capability system for controlling access to Willow data - Spritely Networked Communities Institute

2 Likes

Are there other implementations than iroh? I find it hard to believe that a protocol of that complexity could be developed without at least one reference implementation being developed in parallel. But the Willow Web site only says that some implementations are “under development”.

One of the developers is on the Fediverse, announcing Willow and in one of the replies mentions working on their own implementations.

1 Like