Overflow: Building Safe and Robust AI Systems
As AI systems become more powerful, it is increasingly important that developers be able to strictly enforce desired policies for the systems. Unfortunately, via techniques such as adversarial attacks, it has traditionally been possible to circumvent model policies, allowing bad actors to manipulate LLMs for unintended and potentially harmful purposes. In this talk, I will highlight several recent directions of work that are making progress in addressing these challenges, including methods for robustness to jailbreaks, safety pre-training, and methods for preventing undesirable model distillation. I will additionally highlight some of the areas I believe to be most crucial for future work in the field.
Building Safe and Robust AI Systems
As AI systems become more powerful, it is increasingly important that developers be able to strictly enforce desired policies for the systems. Unfortunately, via techniques such as adversarial attacks, it has traditionally been possible to circumvent model policies, allowing bad actors to manipulate LLMs for unintended and potentially harmful purposes. In this talk, I will highlight several recent directions of work that are making progress in addressing these challenges, including methods for robustness to jailbreaks, safety pre-training, and methods for preventing undesirable model distillation. I will additionally highlight some of the areas I believe to be most crucial for future work in the field.
TU-SOFIA
Qualified with a Master's degree in Marketing Management (Sofia/Technical University), I conduct management training seminars (discussions) and offer individual coaching for modern companies to help them make informed decisions about their activities.
Birds of a Feather Session: "Towards tokenizer-free, end-to-end architectures"
Birds of a Feather is a gathering to bring together researchers, practitioners, and anyone interested in tokenizer-free, end-to-end architectures topics.
We'll have a short panel discussion with researchers from industry and academia, followed by Q&A and free time for people to discuss in smaller groups.
We want to foster innovative ideas and connections among peers, provide a platform for emerging research that might not get the spotlight in the main conference tracks and bridge theoretical discussions with practical implementations. We will also invite young researchers working on similar underrepresented topics who want to share their poster in a smaller setup.
Our target audience is anyone interested in the topic, but we believe it will be interesting for researchers and practitioners working on tokenizer-free approaches, students interested in pursuing work in this area, and industry professionals looking to understand cutting-edge developments.
LLMs in the Public Sector
The proposed ICLR social, "LLMs in the Public Sector," organized by GovTech Singapore, will commence with an insightful presentation highlighting key advancements and applications of Large Language Models (LLMs) across essential public sector domains, including Education, Healthcare, and Labour Markets. Following the presentation, participants will engage in a networking session designed to facilitate structured discussions on innovative use-cases, practical implementation experiences, and prevailing challenges. Emphasis will be placed on public sector use-cases, challenges in development and deployment, Responsible AI.
Mentorship Hour
MENTORS: Nouha Dziri, Rene Vidal
Part of the ICLR experience is meeting people and talking with them about their research interests and experiences. To facilitate these conversations, we are thrilled to announce the third iteration of Mentoring Chats at ICLR (previously called Office Hours).Mentoring Chats will be 45-minute round-table sessions, held during lunch (12:30-1:15 pm and 1:15-2:00 pm) in the Topaz Concourse every day of the main conference (April 24-26). There will be a bell ring approximately 22 minutes in, urging participants to switch tables, or switch topics while staying at the same table.Following ICLR 2024, we have a list of topics and questions that you may wish to ask mentors. We hope to see you there!
Research agenda
- Where should I start if I want to do research in ML? What kind of mathematical/programming skills are required for ML research?
- What are good courses to take? How should I use different modes of learning, such as classroom courses, video lectures, and reading a book?
- How to keep track of all the research literature? How to balance breadth vs depth?
- What are some broader goals of academic machine learning research in the era of LLMs?
- How can one set themselves apart in this crowded research space?
- What is ethical research?
- How to decide on a research area? How to decide on a research project?
- How to adapt my research according to the current trends/community interests?
- How to cope with the pressure of publishing while working on riskier/harder projects? Should I be worried about other groups scooping my research and how to deal with such situations?
- Should I establish myself as an expert in one area/technique or explore a breadth of topics? Should I master a technique and apply it to different problems, or should I master a subfield by finding all useful techniques (hammer vs nails)?
ML+X: Multidisciplinary research
- What are good strategies for starting an interdisciplinary project?
- When working across disciplines, should I have one of them as my “home” community or try to be equally visible in both?
- What are the most efficient ways to help establish my ML+X area as a more active area? Should I organize workshops, teach tutorials, ...?
- How to deal with different incentive structures in interdisciplinary collaborations (e.g., journals vs conferences)?
Advisor and collaborators
- Should I follow my advisor’s agenda or define my own?
- What are the pros and cons of being co-advised?
- When is it appropriate to change advisors and how to go about it?
- How to navigate conflicts with an advisor?
- How to get a good balance between collaborating with other researchers while also distinguishing my own research? Will too much collaboration hurt my job prospects?
- What to look for in a collaborator?
- How do I convey the level of commitment I am willing to have in a project without it being awkward? How to say no to collaborations?
- How to navigate different conventions wrt author ordering? Alphabetical vs contributional ordering? Should my advisor always be a coauthor because they are funding me?
- What do I do if my collaborator is not responsive?
Communicating research and networking
- How to find mentors and allies beyond my advisor?
- What is the best way to communicate my research? Blogs, videos, presentations?
- How to write a good research statement? How to apply for fellowships?
- Should I present my work in poster sessions and workshops? Should I be scared of getting scooped? What are the pros of presenting my work early?
Beyond your institution: Internships and research visits
- Should I do a research internship on a topic different from my dissertation?
- Does it make sense to do a software engineering/development internship if it is not research-related?
- When is a good time to look for internships? Should I apply online or email people?
- Should I do research visits to other universities? Does it make sense to go to semester-long programs as a junior student?
- How to get the most out of my internship? What should be the main goal of doing an internship?
Planning after grad school: academia vs industry
- What should I consider when planning for the next step? How should I decide whether to go to academia or industry?
- How to select a postdoc advisor?
- Should I apply to different departments than my core department? How can I prepare for that, and how early?
- Is it ok to quit your PhD? How can I plan my next steps if so?
Work ethics, open research discussion, personal challenges
- How to balance work-life? How much work is too much work?
- How to take care of mental and physical health?
- How to learn about the ethical implications around the topics of my research?
- How to foster inclusion in research and teaching?
AI for Mathematics and Theorem Proving
AI for Mathematics and Theorem Proving is an emerging area as LLMs are becoming strong reasoners, and its promise in advancing the future of mathematics is evident. We hope that through this social, we are able to bring together both ML researchers and mathematicians in order to come together and discuss emerging ideas in the field. We will provide a welcoming environment for everyone to share their unique expertise and wish to foster exciting collaboration opportunities in both fields. We will potentially have discussion groups to facilitate the social process and potentially have mentorship for more junior attendees who are interested in the field.
Agent research in the real world
RAG is fundamental to real-world agents, but accuracy limitations persist. This talk presents Google Cloud's research on improving RAG for practical applications. We'll discuss findings from real-world deployments and share our approaches to intelligent knowledge integration, end-to-end tuning, and other techniques achieving state-of-the-art performance, demonstrating how we're making RAG more reliable for real-world scenarios.
InclusionAI is a project at Ant Group aiming to develop fully open-sourced AI ecosystem, from algorithm and models to training infra and data. This talk will highlight two particular projects in InclusionAI, AReaL and AWorld. AReaL is an open-sourced RL training systems for training large reasoning models. We will discuss the design details and our training experiences using AReaL. AWorld (Agent World) is a comprehensive framework that simplifies the building, evaluation, and deployment of general multi-agent assistance systems. We'll demonstrate its capabilities and explore how AI agents collaborate to solve real-world tasks.
Beyond Chain-of-Thought: Towards Autonomous Knowledge Management in Alibaba Cloud Tongyi Agentic Systems
This talk focuses on Alibaba Cloud‘s latest research progress in retrieval-augmented system in large language model agents, exploring core technical pathways for knowledge storage, comprehension, reasoning, and planning, while proposing external information enhancement strategies to expand cognitive boundaries. For complex multi-modal search scenarios, it systematically elaborates an agent-based task planning framework and its autonomous decision-making capabilities. Furthermore, by analyzing practical implementations of knowledge enhancement technologies in AI Search applications such as Q&A and cross-modal retrieval, the study provides technical frameworks and actionable insights for building self-evolving agent systems that transcend traditional paradigms.
Mentorship Hour
MENTORS: Bo Han, Taylor W. Killian, Claire Vernade
Part of the ICLR experience is meeting people and talking with them about their research interests and experiences. To facilitate these conversations, we are thrilled to announce the third iteration of Mentoring Chats at ICLR (previously called Office Hours).Mentoring Chats will be 45-minute round-table sessions, held during lunch (12:30-1:15 pm and 1:15-2:00 pm) in the Topaz Concourse every day of the main conference (April 24-26). There will be a bell ring approximately 22 minutes in, urging participants to switch tables, or switch topics while staying at the same table.Following ICLR 2024, we have a list of topics and questions that you may wish to ask mentors. We hope to see you there!
Research agenda
- Where should I start if I want to do research in ML? What kind of mathematical/programming skills are required for ML research?
- What are good courses to take? How should I use different modes of learning, such as classroom courses, video lectures, and reading a book?
- How to keep track of all the research literature? How to balance breadth vs depth?
- What are some broader goals of academic machine learning research in the era of LLMs?
- How can one set themselves apart in this crowded research space?
- What is ethical research?
- How to decide on a research area? How to decide on a research project?
- How to adapt my research according to the current trends/community interests?
- How to cope with the pressure of publishing while working on riskier/harder projects? Should I be worried about other groups scooping my research and how to deal with such situations?
- Should I establish myself as an expert in one area/technique or explore a breadth of topics? Should I master a technique and apply it to different problems, or should I master a subfield by finding all useful techniques (hammer vs nails)?
ML+X: Multidisciplinary research
- What are good strategies for starting an interdisciplinary project?
- When working across disciplines, should I have one of them as my “home” community or try to be equally visible in both?
- What are the most efficient ways to help establish my ML+X area as a more active area? Should I organize workshops, teach tutorials, ...?
- How to deal with different incentive structures in interdisciplinary collaborations (e.g., journals vs conferences)?
Advisor and collaborators
- Should I follow my advisor’s agenda or define my own?
- What are the pros and cons of being co-advised?
- When is it appropriate to change advisors and how to go about it?
- How to navigate conflicts with an advisor?
- How to get a good balance between collaborating with other researchers while also distinguishing my own research? Will too much collaboration hurt my job prospects?
- What to look for in a collaborator?
- How do I convey the level of commitment I am willing to have in a project without it being awkward? How to say no to collaborations?
- How to navigate different conventions wrt author ordering? Alphabetical vs contributional ordering? Should my advisor always be a coauthor because they are funding me?
- What do I do if my collaborator is not responsive?
Communicating research and networking
- How to find mentors and allies beyond my advisor?
- What is the best way to communicate my research? Blogs, videos, presentations?
- How to write a good research statement? How to apply for fellowships?
- Should I present my work in poster sessions and workshops? Should I be scared of getting scooped? What are the pros of presenting my work early?
Beyond your institution: Internships and research visits
- Should I do a research internship on a topic different from my dissertation?
- Does it make sense to do a software engineering/development internship if it is not research-related?
- When is a good time to look for internships? Should I apply online or email people?
- Should I do research visits to other universities? Does it make sense to go to semester-long programs as a junior student?
- How to get the most out of my internship? What should be the main goal of doing an internship?
Planning after grad school: academia vs industry
- What should I consider when planning for the next step? How should I decide whether to go to academia or industry?
- How to select a postdoc advisor?
- Should I apply to different departments than my core department? How can I prepare for that, and how early?
- Is it ok to quit your PhD? How can I plan my next steps if so?
Work ethics, open research discussion, personal challenges
- How to balance work-life? How much work is too much work?
- How to take care of mental and physical health?
- How to learn about the ethical implications around the topics of my research?
- How to foster inclusion in research and teaching?
Framework, Prototype, Definition and Benchmark
In this talk, I will present a set of work done at the Beijing Institute of General Artificial Intelligence (BIGAI) and Peking University on AGI, which is also called TongAI as the Chinese character ‘Tong’ means ‘general’ and contains the letters ‘A’, ‘G’ and ‘I’. I will start with introducing a digital agent --- a little girl who lives and learns continuously in simulated diverse physics-realistic environments with multi-physics and social interactions. The little girl with a nickname ‘TongTong’ is driven by her own value system with desires and goals which generates plans and actions. Then I will reveal the framework underneath this self-conscious agent with three interconnected components: Cognitive architecture (C), the potential functions (U) representing skills, and the value functions (V). Then we define various AGI systems as points in this joint (C,U,V) –space. This framework represents a paradigm shift from the popular “data-driven” "large data for small tasks" statistical paradigm which we pioneered at Harvard, Brown and UCLA since the early 1990s, to the “value-driven” "small data for large tasks" paradigm which I have been advocating since 2010. Then I will introduce TongTest as new criteria, benchmarks and test platform for measuring the general intelligence of various AI agents on performing multi-modal embodied tasks in complex environments. The TongTest has gone way beyond the Turing test in complexity and integrates results from developmental psychology and anthropology. It assesses the intelligence of TongTong to match a 3-4 years old child. In the talk, I will also show some recent work on humanoid robotics and applications, and discuss the Eastern philosophical thinking what makes humans and intelligence, and how morality and social norm emerge from the CUV framework as a solution to AGI safety.
Overflow: Framework, Prototype, Definition and Benchmark
In this talk, I will present a set of work done at the Beijing Institute of General Artificial Intelligence (BIGAI) and Peking University on AGI, which is also called TongAI as the Chinese character ‘Tong’ means ‘general’ and contains the letters ‘A’, ‘G’ and ‘I’. I will start with introducing a digital agent --- a little girl who lives and learns continuously in simulated diverse physics-realistic environments with multi-physics and social interactions. The little girl with a nickname ‘TongTong’ is driven by her own value system with desires and goals which generates plans and actions. Then I will reveal the framework underneath this self-conscious agent with three interconnected components: Cognitive architecture (C), the potential functions (U) representing skills, and the value functions (V). Then we define various AGI systems as points in this joint (C,U,V) –space. This framework represents a paradigm shift from the popular “data-driven” "large data for small tasks" statistical paradigm which we pioneered at Harvard, Brown and UCLA since the early 1990s, to the “value-driven” "small data for large tasks" paradigm which I have been advocating since 2010. Then I will introduce TongTest as new criteria, benchmarks and test platform for measuring the general intelligence of various AI agents on performing multi-modal embodied tasks in complex environments. The TongTest has gone way beyond the Turing test in complexity and integrates results from developmental psychology and anthropology. It assesses the intelligence of TongTong to match a 3-4 years old child. In the talk, I will also show some recent work on humanoid robotics and applications, and discuss the Eastern philosophical thinking what makes humans and intelligence, and how morality and social norm emerge from the CUV framework as a solution to AGI safety.
Oracle AI Science Social
Have a chance to met top AI Science professionals from Oracle, Dan Roth Chief AI Scientist and Sujith Ravi, VP of AI Science.
Muslim in ML Social
The Muslims in ML Social is a community gathering aimed at fostering connections among Muslim researchers, engineers, and professionals in machine learning and artificial intelligence. This informal event offers a welcoming space to meet peers, share experiences, and build lasting professional relationships. Attendees will have the opportunity to network, engage in conversations about career paths and research, and connect across regions and backgrounds. While centered on the Muslim community, the event is open to all ICLR participants who are interested in inclusive dialogue and community-building in ML.
Webpage: https://www.musiml.org/ Next events: ICML 2025, AISTATS 2025
AI and Human Agency
Artificial Intelligence (AI) influences every facet of human life, from professional environments to daily routines and interpersonal relationships. While technology aims to fulfill desires and enable new capabilities, questions are raised about whether human agency is truly enhanced in a world shaped by these technologies across social, economic, and political dimensions In this social event, we will engage in a discourse on the impact of autonomous AI on human agency and explore solutions to potential challenges. Our discussion will focus on how to socio-technically prioritize human agency as a central criterion in AI development, aiming for positive outcomes in AI alignment and safety.
The future of AI is global—but how do we ensure it serves everyone? This social session invites ICRL participants from different cultural and linguistic backgrounds to share insights on how AI development and deployment vary across diverse geographical regions and cultural contexts. The session will showcase multilingual and localization case studies across different AI use cases. Explore strategies around data representation, linguistic challenges, and value alignment across cultures, and broaden your AI perspective.