ELLIS header
University of Stuttgart Logo
Max Planck Institute for Intelligent Systems Logo

Nalin: Learning from Runtime Behavior to Find Name-Value Inconsistencies in Jupyter Notebooks

Jibesh Patra, Michael Pradel

Proceedings of the 44th IEEE/ACM International Conference on Software Engineering (ICSE), pp. 1–13, 2022.


Variable names are important to understand and maintain code. If a variable name and the value stored in the variable do not match, then the program suffers from a name-value inconsistency, which is due to one of two situations that developers may want to fix: Either a correct value is referred to through a misleading name, which negatively affects code understandability and maintainability, or the correct name is bound to a wrong value, which may cause unexpected runtime behavior. Finding name-value inconsistencies is hard because it requires an understanding of the meaning of names and knowledge about the values assigned to a variable at runtime. This paper presents Nalin, a technique to automatically detect name-value inconsistencies. The approach combines a dynamic analysis that tracks assignments of values to names with a neural machine learning model that predicts whether a name and a value fit together. To the best of our knowledge, this is the first work to formulate the problem of finding coding issues as a classification problem over names and runtime values. We apply Nalin to 106,652 real-world Python programs, where meaningful names are particularly important due to the absence of statically declared types. Our results show that the classifier detects name-value inconsistencies with high accuracy, that the warnings reported by Nalin have a precision of 80% and a recall of 76% w.r.t. a ground truth created in a user study, and that our approach complements existing techniques for finding coding issues.



@inproceedings{patra22_icse, title = {Nalin: Learning from Runtime Behavior to Find Name-Value Inconsistencies in Jupyter Notebooks}, author = {Patra, Jibesh and Pradel, Michael}, year = {2022}, pages = {1--13}, booktitle = {Proceedings of the 44th {IEEE/ACM} International Conference on Software Engineering ({ICSE})}, preprint = {https://software-lab.org/publications/icse2022_Nalin.pdf} }