Lessons from the Impossibility of Safety

What kind of results is impossible for safety research, and what pathways forward can we hope to achieve? First, we will discuss theoretical results on rule-following that demonstrate token-level jailbreaks as an architectural inevitability of attention (LogicBreaks). While initially pessimistic, these theoretical insights can also be leveraged to steer models to state of the art performance in five lines of code (InstABoost). Lastly, we will argue for a shift in safety strategy away from aligning model weights to stateful monitoring, as the only level at which one can hope to stop misuse (https://modelmisuse.com/ <https://modelmisuse.com/> ).

More Information