On Pessimization

Richard Ngo, an AI safety advocate formerly with Google DeepMind and OpenAI, writes a really lovely piece about "pessimization", the phenomenon where working towards some goal counterintuitively helps to achieve the opposite of that goal (i.e., an anti-goal).

I love this kind of concept; they appear all over the social sciences with various names. They are a vital part of examining how systems and institutions and the actors within them operate, but are woefully underdiscussed outside of the academy. Even when they are discussed, in my experience, they are treated as interesting thought experiments or narrow applications. In reality, I think they’re far more common that acknowledged.

Ngo breaks down pessimization into a three-part typology:

• Direct pessimization: when opponents actively try to bring about the anti-goal.
• Indirect pessimization: when work towards a goal encourages others to work on the anti-goal either through raising awareness of the anti-goal or simply by having bad or uncompelling ideas about the goal itself.
• Perverse pessimization: when ostensibly aligned actors sabotage progress towards the goal. This is common when organizational and institutional structures emerge in support of a goal whose existence would be threatened if the goal was realized. Ngo cites the prime example here of environmentalists opposing nuclear power.

One of the feedback effects that Ngo doesn’t discuss, and one I think a lot about, is something akin to but not quite equivalent to the No True Scotsman fallacy. The idea being here that, in the face of failure to achieve a goal, the proposed solution is to double down on existing methods and tactics rather than admit that they are not succeeding. “Real <approach> has never been tried, that’s why we’re failing!”