Invited Talk (AdvML): Nathalie Baracaldo — Can we really make AI Safe?

IBM Research

Abstract: Generative AI applications have taken the world by storm at an incredible speed, offering and promising tremendous benefits in a variety of domains. Unfortunately, the pace at which they have been safeguarded seems to be slower, as demonstrated by the number of incidents that have been discussed in the public media. The key question is: can we make AI safe? To answer this question, in this talk, I will go through a tour of the good, the bad and the ugly when it comes to deployment practices found in the wild. To systematically analyze and highlight what can go wrong and how we can improve it, I will present a methodology that I recently created. Using this methodology, I will provide recommendations to overcome some of the common pitfalls that include overreliance and lack of defense in depth. Finally, I will discuss open challenges and research questions for the community to address.

Bio: Nathalie Baracaldo is a Senior Research Scientist and Master Inventor at IBM Research in San Jose, California, where her work focuses on building trustworthy AI systems. She has extensive experience delivering impactful machine learning solutions that are highly accurate, withstand adversarial attacks, and protect data privacy. Her current research focuses on safeguarding generative AI. She served as the principal investigator for the DARPA GARD program, leading efforts to extend and maintain the Adversarial Robustness Toolbox (ART) for red teaming evaluations. She also led IBM's federated learning initiative and co-edited two books: "Federated Learning: A Comprehensive Overview of Methods and Applications" (Springer, 2022) and "Machine Unlearning for Governance of Foundation Models" (2026). Her research has been published in top AI and security conferences, earning multiple best paper awards and thousands of citations. She received the IBM Master Inventor distinction in 2020 and the Corporate Technical Recognition in 2021. She holds a Ph.D. from the University of Pittsburgh.