How Programmers Interact with AI Assistants

Powered by recent advances in code-generating models, AI assistants like Github Copilot promise to change the face of programming forever. But what is this new face of programming? And how can we help programmers use these assistants more effectively?

In the first part of the talk, I will present the first grounded theory study of how programmers interact with Copilot, based on observing 20 participants with varying levels of experience. Our main finding is that interactions with programming assistants are bimodal, with programmers using Copilot either in acceleration mode or exploration mode.

Based on the observations of this first study, we designed a new interaction model, dubbed Live Exploration of AI-generated Programs (LEAP), with the goal to better support programmers in exploration mode. The main idea of LEAP is to use Live Programming, a continuous display of a program’s runtime values, to help the user understand and validate AI code suggestions. In the second part of the talk, I will discuss LEAP and our user study, which shows that Live Programming lowers the cost of validating AI suggestions, thereby reducing both under- and over-reliance on the AI assistant.