Towards guardrails, not guidelines: a policy framework for powerful AI systems

I’m happy to let my thermostat automatically turn my heat on and off or to let an AI drive a car or optimize the traffic lights in a city. I’m less sure about an AI that sets tax rates, or corporate regulations or foreign policy. Or an AI that tells us that it can’t explain why, but strongly urges us to declare war—right now

To the extent that people do use AI tools for real work, it is often through something like an application copilot. These are all built to offer a “safe” way to use AI at work, and as such are often very limited compared to what a frontier model can do.

A computer inference however may draw upon massive data and complex correlations, you can’t argue with the machine, and you can’t see how to change. You have no say and governance is replaced with management […]