Quantifying the Bitter Lesson: How Safety Benchmarks Measure Capabilities Instead of Safety

Authors: Richard Ren, Steven Basart, Adam Khoja, Alexander Pan, Alice Gatti, Long Phan, Xuwang Yin, Mantas Mazeika, Gabriel Mukobi, Ryan Kim, Stephen Fitz, Dan Hendrycks