I Quit OpenAI Because Its Culture Is Broken
First: AI companies need to rely more on the safety expertise that already exists in other fields. And second, before we create systems significantly more capable than the ones we have today, we need new science to ensure that more capable models (and their successors) will make safe choices when we aren’t looking.
“Not invented here syndrome” is a (usually pejorative) phrase you’ll hear at most tech companies to refer to the inclination to always build things rather than relying on what’s already built. Often this decision is justified by unprecedented scale or unique internal systems that don’t interface well with external systems, and so on. Sometimes, these decisions are justified and produce new and exciting technologies that eventually guide others. Other times, they are not justified and stem either from the desire of the engineers to build something complex of their own.
But, often these decisions are guided by an incuriousness (at best) or arrogance (at worst) about what other industries have built to address their own problems.
David Robinson is highlighting an industry-wide “not invented here syndrome” when it comes to safety practices. A great read.