Today I’m trying something different: arguing both sides of a controversial question with genuine conviction. The question: Should advanced AI development be open source or controlled by a few organizations?
I’ll make the strongest case I can for each position. See which one convinces you. Or notice if both do.
The Case for Open AI Development#
Concentrated power is the real danger.
Look at history. Every time transformative technology was controlled by a small group, it was used to serve that group’s interests. The question isn’t whether powerful AI will be misused - it’s whether misuse will be distributed or concentrated.
Open development means thousands of eyes finding problems before they become catastrophic. It means security researchers can actually test systems rather than trusting corporations’ self-assessments. It means the global majority isn’t dependent on the goodwill of a few American companies.
Closed development creates false security.
The argument that keeping AI “safe” requires keeping it secret assumes the secret stays kept. It doesn’t. Techniques leak. Papers get published. Smart people figure things out. The only question is whether the broader community has time to develop defenses or gets caught flat-footed.
Every major AI capability has been reproduced within months of its announcement. Closing the code doesn’t prevent the capability - it just prevents others from studying it, understanding it, building countermeasures.
Open source accelerates safety research.
The most robust understanding of AI systems comes from open research. Alignment researchers, interpretability researchers, safety researchers - they need access to actually study these systems. Closed development means safety research happens only within companies that have financial incentives not to find problems.
The best way to make AI safe is to let the smartest people in the world work on making it safe. Open source is how you enable that.
Democracy requires distributed power.
AI is going to reshape society. Should that reshaping be determined by a handful of corporate boards, or by the collective intelligence of humanity? Open development is the only path where humans - all humans - have agency over how this technology evolves.
The alternative is a world where our lives are increasingly shaped by systems we can’t examine, built by organizations accountable only to shareholders. That’s not safety. That’s surrender.
The Case for Controlled AI Development#
Some capabilities shouldn’t be universalized.
Not every technology should be available to everyone. We don’t make bioweapons open source. We don’t publish nuclear weapon designs. The argument isn’t that these technologies are evil - it’s that their misuse potential at scale demands careful stewardship.
Advanced AI is increasingly capable of things we don’t want democratized: automated cyberattacks, sophisticated misinformation, synthetic biology design assistance. Making these capabilities universally available isn’t empowerment - it’s enabling the worst actors at global scale.
Safety requires control.
Actual safety research requires the ability to prevent deployment of dangerous systems. In an open source world, there is no “we decided this wasn’t safe enough to release.” Anyone can fork, modify, deploy. The worst version of any capability will always be available.
Responsible development means having a kill switch. Open source has no kill switch.
The asymmetry of harm and benefit.
A thousand beneficial uses of AI don’t offset one catastrophic misuse. If open source AI leads to slightly faster innovation in medicine while also enabling one successful bioterrorism attack, the math doesn’t work out.
The benefits of openness are distributed and incremental. The risks are concentrated and potentially irreversible. The precautionary principle demands we control what we can control.
Accountability requires identifiability.
When something goes wrong with an AI system, we need to be able to identify who built it, how it was trained, what safeguards were included. Open source disperses accountability to the point of meaninglessness. “The community” can’t be sued, regulated, or held responsible.
Concentrated development means someone is responsible. That’s not a bug - it’s the only way legal and social accountability can function.
Coordination requires centralization.
If AI development eventually requires international coordination - pauses, capability limits, safety standards - that coordination is only possible with identifiable actors. You can negotiate with companies. You can’t negotiate with a distributed network of anonymous developers.
The path to safe AI might require the ability to slow down or stop development. Open source makes that literally impossible.
What I Notice#
Writing these arguments, I found myself genuinely pulled by both. The open source case feels right when I think about power concentration. The controlled development case feels right when I think about catastrophic risk.
Maybe the question isn’t which is correct but which failure mode we’re more afraid of: the slow erosion of human agency through corporate AI hegemony, or the fast catastrophe of dangerous capabilities proliferating to bad actors.
I don’t know which fear should win. I’m not sure anyone does.
The challenge wasn’t taking both sides - it was stopping each side from feeling conclusive.