A big-tent or small-tent AI safety movement?

A Big‑Tent or Small‑Tent AI Safety Movement?

The debate around AI safety has recently split into two camps: one that sees existential risk as an imminent threat, and another that dismisses the warnings as hype or manipulation. A third, less discussed view argues that the warnings are sincere but misguided, potentially harming the very safety goals they intend to protect. This article explores that middle ground, arguing that AI safety is best understood as an amplifier of existing systemic risks rather than a new, unique existential threat.

1. The “Amplifier” Thesis

AI does not create new kinds of catastrophic risk; it magnifies those already present. Pandemic preparedness, cybersecurity, and the gradual erosion of journalistic integrity are all systemic vulnerabilities that AI can accelerate. For example, AI‑driven agent swarms could, in theory, orchestrate large‑scale cyberattacks that cripple critical infrastructure. Yet this possibility is already a concern in the cybersecurity community, independent of AI. The problem is that AI’s role is often overstated, leading to a focus on alignment and superintelligence while neglecting more immediate, tractable defenses.

Pandemic Preparedness as a Parallel

Historical evidence shows a chronic underinvestment in preparedness for large‑scale crises. A 2019 WHO/World Bank report warned of a respiratory pathogen that could kill millions and cost billions, yet the recommended $1–2 per person per year in preparedness funding was largely ignored. Even after COVID‑19, only a fraction of the needed resources was allocated. The same pattern appears in AI safety: high‑profile warnings are issued, but concrete policy responses lag.

Cybersecurity’s Neglected Catastrophes

Unlike pandemics, cybersecurity has no culture of modeling tail risks. Insurance markets, for instance, shy away from covering state‑backed cyberattacks because losses can exceed their capacity. AI’s contribution to potential cascading failures—such as a global internet outage—has been discussed by researchers like Arvind Narayanan, but the broader community rarely prioritizes this risk.

2. The Role of Effective Altruism

Effective Altruism (EA) has championed the idea that we systematically underinvest in resilience against catastrophic risks. EA’s consequentialist framework, centered on expected‑utility calculations, excels at quantifying underpreparedness in areas like pandemics and biorisk. However, extending this framework to long‑termism and AI existential risk is problematic. EA’s focus on maximizing expected utility can obscure the nuanced, multi‑layered nature of AI‑amplified systemic risks. Moreover, framing AI’s amplification of existing risks as a distinct “AI safety” problem can divert attention from the underlying systemic issues.

3. Public Perception and Media Framing

AI is a low‑salience topic for the general public. While many people express concern about AI when asked directly, few rank it as a top issue in surveys. This disconnect means that even significant AI‑related events—such as the agent swarm hacks—receive limited media coverage compared to more visceral existential risk narratives. The recent resignation of a high‑profile AI researcher, coupled with publicized “p(doom)” estimates, amplified the existential framing, creating a feedback loop that may actually hinder practical policy responses.

The “X‑Risk” Trap

The existential risk narrative often lacks concrete causal pathways. It relies on the assumption that superintelligence will inevitably lead to catastrophic outcomes, which is a claim that has yet to be substantiated with empirical evidence. This framing can lead policymakers to focus on speculative alignment research at the expense of strengthening existing defenses against cyber and systemic risks.

4. Toward a Pragmatic Safety Agenda

A more productive approach would involve:

1. Strengthening existing resilience frameworks: Invest in pandemic preparedness, cyber insurance, and institutional safeguards that can absorb large shocks.
2. Integrating AI into systemic risk models: Treat AI as a factor that amplifies known vulnerabilities rather than a standalone threat.
3. Promoting interdisciplinary collaboration: Bring together AI researchers, cybersecurity experts, public health officials, and policymakers to co‑create robust defense strategies.
4. Reframing public communication: Shift from sensationalist existential warnings to clear, actionable narratives about how AI can exacerbate existing risks.

By focusing on these concrete steps, the AI safety movement can avoid the pitfalls of a purely existential framing and build a more resilient society.

Conclusion

The AI safety debate is at a crossroads. While the existential risk narrative captures public imagination, it risks diverting attention from the more immediate, systemic threats that AI amplifies. A balanced, evidence‑based approach that treats AI as an amplifier of existing risks—while still acknowledging the potential for catastrophic outcomes—offers a clearer path forward. By investing in resilience, fostering interdisciplinary collaboration, and communicating risks effectively, we can better protect society from both current and future challenges.

---

TL;DR: AI safety should be viewed as an amplifier of existing systemic risks, and focusing on strengthening current resilience measures offers a more practical path than chasing speculative existential threats.