All organizations - tech companies, social clubs, governments - have both a legible and an illegible side. The legible side is important, past a certain size. It lets the organization do things that would otherwise be impossible: long-term planning, coordination with other very large organizations, and so on. But the illegible side is just as important. It allows for high-efficiency work, offers a release valve for processes that don’t fit the current circumstances, and fills the natural human desire for gossip and soft consensus.
Subscribe for new posts
Blog posts only. No commonplace entries. Never sold or shared.
A related idea that I have been mulling over: illegible AI and our insatiable need to use it for legible things. AI researchers often talk about “growing” LLMs, in the sense that it’s difficult to provide very specific instructions to train an LLM and get predictable, expected results. Instead, you set the conditions and feed it data and see where it leads.
In this sense, the models that are powering our current AI boom are highly illegible in some sense. And yet, our very first instincts are to try and make them legible: arguing about whether or not they will increase GDP or productivity, what percent of AI-driven experiments succeed, and what percent of jobs they can or cannot replace. There are more and more pushes to point AI at “verifiable” tasks (e.g., “legible”) to advance progress. No surprise that coding and math are making huge gains. Labs are accused of “benchmaxxing”, i.e, training models against established benchmarks rather than on more realistic tests of utility.
One of the most frequently drawn arrows in the anti-AI quiver is pointing to these numbers and the (lack of) impact on them. If the progress is not legible, it must be illusory.
And yet, whenever a new model is released, one of the biggest threads of conversation is around “vibe” evaluation. How does the model feel? What is its aesthetic ability? Is the writing good? We’re bad at measuring these things, but they’re important nonetheless.