Neural models of code are successfully tackling various prediction tasks, complementing and sometimes even outperforming traditional program analyses. While most work focuses on end-to-end evaluations of such models, it often remains unclear what the models actually learn, and to what extent their reasoning about code matches that of skilled humans. A poor understanding of the model reasoning risks deploying models that are right for the wrong reason, and taking decisions based on spurious correlations in the training dataset. This work investigates to what extent the attention weights of effective neural models match the reasoning of skilled humans. To this end, we present a methodology for recording human attention and use it to gather thousands of human attention maps for two tasks: code summarization and automated program repair. Comparing the human attention to four attention-based neural models shows that these models correlate with the way humans reason about code. At the same time, there are important differences: For example, some models ignore kinds of tokens, such as strings, that are deemed important by the humans, and other models miss most of the code around a line of interest. The results also show that human-model agreement positively correlates with accurate predictions by a model, which calls for neural models that even more closely mimic human reasoning. Beyond the insights from our study, we envision the release of our dataset of human attention maps to help understand future neural models of code and to foster work on human-inspired models.

Michael Pradel is a full professor at University of Stuttgart, which he joined after a PhD at ETH Zurich, a post-doc at UC Berkeley, an assistant professorship at TU Darmstadt, and a sabbatical at Facebook. His research interests span software engineering, programming languages, security, and machine learning, with a focus on tools and techniques for building reliable, efficient, and secure software. In particular, he is interested in dynamic program analysis, test generation, concurrency, performance profiling, JavaScript-based web applications, and machine learning-based program analysis.