Invited Talk (AdvML): Nicole Nichols — How Do Secret Squirrels Share Benchmarks? Coping with the Dual-Use Risks of AI Security Evaluations

Palo Alto Networks

Abstract: In machine learning, scientific progress is driven by open, reproducible benchmarks. But in cybersecurity, evaluating autonomous AI agents introduces a severe dual-use paradox. To accurately measure an agent's capabilities, researchers must build high-fidelity, realistic enterprise environments. However, transparently sharing these environments doesn't just distribute a benchmark—it distributes a literal attacker roadmap. This talk explores the friction between operational security and transparency caused by "architecturally entangled" evaluation scripts. By transitioning to declarative frameworks that separate test intent from execution, we can rigorously verify agent capabilities and build cumulative science.

Bio: Nicole Nichols is a Distinguished Engineer in Machine Learning Security at Palo Alto Networks. She previously held senior roles at Apple and Microsoft, and has contributed to both academic and industrial advancements in adversarial machine learning and security. She has published at ACM, IEEE, and CVPR workshops, was co-chair of ICML ML4Cyber workshop, and is an advisor to the AI Security Forum. She is a Polymath Fellow in Artificial Intelligence for the Global Fellowship Initiative at the Geneva Centre for Security Policy. She has a PhD in Electrical Engineering from the University of Washington.