When Should We Trust AI? From Adversarial Robustness and Explainability to Evidence-Based Decision Support
Date:
Modern AI systems achieve remarkable performance across a wide range of tasks, yet their deployment in real-world settings remains hindered by issues of reliability, robustness, and trust. In this talk, I will present a research journey aimed at understanding why modern machine learning systems fail and how these failures can be mitigated.
I will first discuss my work on adversarial machine learning, spanning efficient adversarial attacks and robustness analysis across image classification and 3D representation. These studies reveal fundamental vulnerabilities in learned representations and highlight the challenges of deploying AI systems in safety-critical environments.
I will then turn to explainability and interpretability, presenting recent work on optimizing saliency maps, evaluating explanation methods, and developing concept-based approaches for more transparent machine learning. These results suggest that explanations themselves must be rigorously assessed, as commonly used explanation techniques may provide a misleading sense of understanding.
Finally, I will argue for a broader perspective on trustworthy AI. While robustness and explainability are often studied separately, they address a common question: when should humans trust AI systems? Building on this observation, I will outline a future research direction toward evidence-based and justifiable AI systems that can not only make predictions, but also provide verifiable support for their decisions. I will conclude by discussing opportunities at the intersection of robustness, interpretability, foundation models, and AI-assisted decision-making.