Towards Building Safe and Secure AI: Lessons and Open Challenges

Recent advancements in AI and LLM agents have unlocked powerful new capabilities across a wide range of applications. However, these advancements also bring significant risks that must be addressed. In this talk, I will explore the various risks associated with building and deploying AI and LLM agents and discuss approaches to mitigate them. I will also examine how frontier AI and LLM agents could be misused, particularly in cyber security attacks, and how they may reshape the cyber security landscape. Ensuring a safe AI future demands a sociotechnical approach. I will outline our recent proposal for a science- and evidence-based AI policy, highlighting key priorities to deepen our understanding of AI risks, develop effective mitigation approaches, and guide the development of robust AI policies.