This post (along with a recent Hank Green video about the Navier-Stokes equation) is a major inspiration for both the Math is Dead post, and an upcoming sister post around (what ended up being defined, here, as) Boundary Objects.
Here are some of my notes around the post, as well as my experience using Chopin as a multiplayer harness.
From Planning with Agents: Divided Worlds, Boundary Objects, and Thicker Interfaces on maggieappleton.com
Annotations
When you or I start planning with an agent today, we’re in our CLI or desktop app of choice, alone, and we usually begin in some kind of plan mode or with the /grill-me skill enabled, and then proceed to answer 40 multiple-choice questions.
Eventually, the agent stops asking questions and gives me a huge wall of Markdown to read. Then it asks: do you approve this plan, yes or no?There are some very obvious problems here. Especially in a CLI interface, I can’t edit the plan directly. The plan almost always has open questions in it, despite the grilling process, and I don’t have an interface for answering them.
As the person who introduced skills in general to my team/company, it is a very difficult thing to try and balance two truths:
/grillis a better way to make sure agents have the information they need to successfully draw in the lines (feedforward); and- It’s very, very imperfect.
It’s the subtlety that a solution can be better than nothing, but far from perfect.
A lot of why this is a difficult problem to solve boils down to humans and agents being very different kinds of beings, with very different worlds, that are only partially legible to one another.
The recent Navier-Stokes kerfuffle is a huge highlight of this: the fact that the proof is there, but that we don’t understand it, is exactly what Hank Green talks about in his recent video with Grant Sanderson (3blue1brown) about understanding and elegance: we (through agents) have proven something that we (as people) do not yet understand.
These kinds of rich, interactive plans are what I’m currently exploring in a project called Chopin . This is a multiplayer, real-time planning prototype that my coworker Krzysztof and I are making.It’s currently a very basic proof of concept and exploration space.
Pulling and using Chopin with my team was, genuinely, the most mind-blowing moment that lit up “multiplayer” in all of our minds. I talk about this a bit in Season of Stamps, but a lot of my time in the last two years has been spent talking to robots alone. I was having trouble deciding if the problem in that state of affairs was the “robots” part, or the “alone” part.
Getting to properly use a multiplayer harness pretty definitively answers: it’s the alone part.
While there’s little ticky-tack things about Chopin that I had trouble with (notably, having to cobble together a local-first model, without the need for API tokens), it was greatly illuminating.
In my mind, to be multiplayer, the architecture of these types of harnesses should truly leverage actual game development prior art. The one that rattles around my brain is the idea of a Listen Server architecture. The best example I can think of is Minecraft’s join-by-LAN feature. Whenever you play single-player Minecraft, you are secretly playing a multiplayer server by yourself.
I think that the future of harnesses is listen server architecture: multiplayer by default, even if you are alone.
It’s built with MDX.
You may’ve noticed this on my own blog, just in the general uptick of diagrams and graphs in the last ~12 months, but it is easier than ever to add interactivity to static pages. At this point, I think any multiplayer harness that doesn’t leverage MDX (over pure Markdown like an agent outputs in a TUI) is seriously, fundamentally lacking.
It is so important to understand, conceptually, that
grillis both a step forward in minimizing “hallucination” (which I’d define as just imprecise definition of a problem that folks truly misdiagnose at this point), but is far from a perfect solution.