Mon 13 Jun 2022 14:15 - 14:30 at Boardroom - Afternoon Chair(s): Charles Sutton
Tue 14 Jun 2022 02:15 - 02:30 at Boardroom - Afternoon

Neural code intelligence (CI) models are opaque black-boxes and offer little insight on the features they use in making predictions. This opacity may lead to distrust in their prediction and hamper their wider adoption in safety-critical applications. Recently, input program reduction techniques have been proposed to identify key features in the input programs to improve the transparency of CI models. However, this approach is syntax-unaware and does not consider the grammar of the programming language.

In this paper, we apply a syntax-guided program reduction technique that considers the grammar of the input programs during reduction. Our experiments on multiple models across different types of input programs show that the syntax-guided program reduction technique is faster and provides smaller sets of key tokens in reduced programs. We also show that the key tokens could be used in generating adversarial examples for up to 65% of the input programs.

Mon 13 Jun

Displayed time zone: Pacific Time (US & Canada) change

13:30 - 15:00
AfternoonMAPS at Boardroom +12h
Chair(s): Charles Sutton Google Research
13:30
45m
Keynote
Can Transformers Code?virtual
MAPS
14:15
15m
Talk
Syntax-Guided Program Reduction for Understanding Neural Code Intelligence Modelsvirtual
MAPS
Md Rafiqul Islam Rabin University of Houston, Aftab Hussain University of Houston, Amin Alipour University of Houston
DOI Pre-print
14:30
15m
Talk
A Systematic Evaluation of Large Language Models of Codevirtual
MAPS
Frank F. Xu Carnegie Mellon University, Uri Alon Carnegie Mellon University, Graham Neubig Carnegie Mellon University, Vincent J. Hellendoorn Carnegie Mellon University
14:45
15m
Poster
Poster Session
MAPS

Tue 14 Jun

Displayed time zone: Pacific Time (US & Canada) change

01:30 - 03:00
AfternoonMAPS at Boardroom
01:30
45m
Keynote
Can Transformers Code?virtual
MAPS
02:15
15m
Talk
Syntax-Guided Program Reduction for Understanding Neural Code Intelligence Modelsvirtual
MAPS
Md Rafiqul Islam Rabin University of Houston, Aftab Hussain University of Houston, Amin Alipour University of Houston
DOI Pre-print
02:30
15m
Talk
A Systematic Evaluation of Large Language Models of Codevirtual
MAPS
Frank F. Xu Carnegie Mellon University, Uri Alon Carnegie Mellon University, Graham Neubig Carnegie Mellon University, Vincent J. Hellendoorn Carnegie Mellon University
02:45
15m
Poster
Poster Session
MAPS