Invited Talk

A Neural Network Model That Can Reason

Christopher Manning

2018 Invited Talk

Abstract

Deep learning has had enormous success on perceptual tasks but still struggles in providing a model for inference. To address this gap, we have been developing Memory-Attention-Composition networks (MACnets). The MACnet design provides a strong prior for explicitly iterative reasoning, enabling it to learn explainable, structured reasoning, as well as achieve good generalization from a modest amount of data. The model builds from the great success of existing recurrent cells such as LSTMs: A MacNet is a sequence of a single recurrent Memory, Attention, and Composition (MAC) cell. However, its design imposes structural constraints on the operation of each cell and the interactions between them, incorporating explicit control and soft attention mechanisms. We demonstrate the model’s strength and robustness on the challenging CLEVR dataset for visual reasoning (Johnson et al. 2016), achieving a new state-of-the-art 98.9% accuracy, halving the error rate of the previous best model. More importantly, we show that the new model is more data-efficient, achieving good results from even a modest amount of training data. Joint work with Drew Hudson.

Speaker

Christopher Manning

Christopher Manning is the inaugural Thomas M. Siebel Professor of Machine Learning in the Departments of Linguistics and Computer Science at Stanford University, Director of the Stanford Artificial Intelligence Laboratory (SAIL), and an Associate Director of the Stanford Institute for Human-Centered Artificial Intelligence (HAI). His research goal is computers that can intelligently process, understand, and generate human languages. Manning is a leader in applying Deep Learning to Natural Language Processing (NLP), with pioneering research on the GloVe model of word vectors, attention, self-supervised model pre-training, tree-recursive neural networks, machine reasoning, and direct preference optimization. Earlier on, he worked on probabilistic NLP models for parsing, sequence tagging, and grammar induction and he also focuses on computational linguistic approaches to natural language inference and multilingual language processing, including being a principal developer of Stanford Dependencies and Universal Dependencies. Manning manages development of the open-source Stanford CoreNLP and Stanza software, and his software and algorithms are used in the systems of many companies for tasks such as sentiment analysis, machine translation, dependency parsing, summarization, and question answering. Manning has coauthored leading textbooks on statistical approaches to NLP (Manning and Schütze 1999) and information retrieval (Manning, Raghavan, and Schütze, 2008), and teaches the popular NLP class CS224N, which is available online. He is an ACM Fellow, a AAAI Fellow, and an ACL Fellow, and a Past President of the ACL (2015). His research has won the 2024 IEEE John von Neumann Medal; ACL, Coling, EMNLP, and CHI Best Paper Awards; and a 2023 ACL Test of Time award. He has a B.A. (Hons) from The Australian National University, a Ph.D. from Stanford in 1994, and an Honorary Doctorate from U. Amsterdam in 2023, and he held faculty positions at Carnegie Mellon University and the University of Sydney before returning to Stanford, where he founded the Stanford NLP group (@stanfordnlp).

Chat is not available.