Invited Talk
in
Workshop: Deep Learning for Code (DL4C)

BigCode: open and responsible development of LLMs for code

Harm de Vries · Leandro von Werra

2023 Invited Talk
in
Workshop: Deep Learning for Code (DL4C)

Abstract

In this presentation, we will share several accomplishments of the BigCode project, a community effort working on the responsible development of LLMs for code generation through open-science and open-governance. These include:

A new 15B parameter LLM for code
The Stack, 6.4 TB of permissively licensed source code with opt-out mechanism
Novel insights on the LLM scaling laws, suggesting we haven't reached the limit of training smaller LLMs for longer

Video

Chat is not available.