AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitions

Data science competitions on Kaggle, which represent real-world programming challenges, require sophisticated problem-solving approaches. While LLM-based agents demonstrate potential in various fields, their application to data science tasks often falls short due to difficulties in adapting to data changes in multi-stage reasoning and the need for precise reasoning. To address this, we propose AutoKaggle, a robust and user-centric framework that solves Kaggle problems through a collaborative multi-agent cooperative system. AutoKaggle implements an iterative development process that combines code interpretation, debugging, and comprehensive unit testing covering over 30 tests, ensuring code correctness and quality through LLM-based evaluation. It prioritizes user experience by generating detailed reports that elucidate feature engineering processes, data transformations, model selection criteria, and the reasoning behind each decision. It offers customizable workflows, allowing users to intervene and modify each stage of the process, thus combining the advantages of automated intelligence with human expertise. Additionally, we build a universal data science tool library, including carefully verified functions for data cleaning, feature engineering, and modeling, which form the foundation of this solution. We evaluate the framework on 8 carefully selected Kaggle competitions, achieve 83.8% in average completion rate and 42.8% average rank in Kaggle.