Towards Green, Accurate, and Efficient AI Models Through Multi-Objective Optimization

Machine learning is one of the fastest growing services in modern hyperscale data centers. While AI’s exponential scaling has enabled unprecedented modeling capabilities across computer vision, natural language processing, protein modeling, personalized recommendation, it comes at the expense of significant energy and environmental footprints. This work aims to co-optimize machine learning models in terms of their accuracy, compute efficiency, and environmental sustainability by using multi-objective bayesian optimization. We aim to extend current multi-objective optimization frameworks, such as the openly available Ax (adaptive experimentation) platform to balance accuracy, efficiency, and environmental sustainability of deep neural networks. In order to optimize for environmental sustainability we will consider the impact across AI model life cycles (e.g., training, inference) and hardware life cycles (e.g., manufacturing, operational use). Given this is a research proposal, we expect to demonstrate that designing for sustainable AI models yields distinct optimal neural network architectures than ones designed for accuracy and efficiency given the external impacts of varying renewable energy and tradeoffs between compute and storage for embodied carbon overheads.