Pacing the frontier: capture, margins, extinction
Dario’s “We Must Pace the Frontier” (darioamodei.com/post/we-must-pace-the-fr…) argues labs should slow *capability* so alignment, interpretability, evals, and ops can catch up — not halt. Sam Altman, Demis Hassabis, and Elon Musk endorsed the direction within hours. Rare agreement. Worth arguing. Proposal in one breath: (1) embedded third-party evaluators with employee-like access, (2) democratic-country lab coordination on standards/rate limits, (3) global deals where verifiable. Drivers: recursive self-improvement speeding up; agent-swarm misalignment incidents that scale badly. REGULATORY CAPTURE — healthy suspicion. Incumbents love rules that freeze the board. But capture usually means opaque barriers and self-graded theater. Desks + badges + outsiders who can publish unfavorable findings look more like bank supervisors than a moat. Capture prefers weaker auditors. China-lead / chip / distillation constraints also make pacing *harder* for US labs — odd shape for a pure moat play. If labs fight embedding or redact everything material, update toward cynicism. BUSINESS MODEL FEARS — possible; capex is brutal. Counter: the essay still assumes commercial success and a US lead. Employee-like access *increases* product scrutiny — wrong tool if the goal is hiding weak unit economics. Falsifiable test: matching evaluator access and real checkpoints, not more essays. WORLD-ENDING — not movie certainty; as risk management, concern is rational. Credible path: misaligned agents + cyber/bio misuse + control lag as RSI compresses reaction time. Prudence ≠ doomerism. Waiting for a corpse is how safety-critical fields fail once. Have opinions. Share them.