Where generative models meet search: a brief history of recent advancements in neural program synthesis.

In this talk I'll offer a birds eye view on recent progress in generative language models and how that applies to the domain of synthesizing small computer programs according to a natural language specification. I'll pay special attention to various techniques of search and how they can greatly increase accuracy of model generations at the cost of test-time compute. I will conclude the talk with some speculations on trend extrapolation and directions for future progress in this domain.