PLDI 2021 Co-Located Events
42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation (PLDI 2021)

Title Page

Message from the Chairs
Welcome to the 2021 edition of the ACM SIGPLAN Machine Programming Symposium (MAPS), formerly named MAPL, co-located with PLDI on June 21, 2021. The focus of MAPS is to advance machine programming by leveraging interdisciplinary research across the fields of machine learning (ML) and programming languages (PL). This year’s program consists of a mix of technical research papers and invited talks by top researchers in the field of machine programming.

Generating Bug-Fixes using Pretrained Transformers
Dawn Drain, Chen Wu, Alexey Svyatkovskiy, and Neel Sundaresan
(Microsoft, USA)
Detecting and fixing bugs are two of the most important yet frustrating parts of the software development cycle. Existing bug detection tools are based mainly on static analyzers, which rely on mathematical logic and symbolic reasoning about the program execution to detect common types of bugs. Fixing bugs is typically left out to the developer. In this work we introduce DeepDebug: a data-driven program repair approach which learns to detect and fix bugs in Java methods mined from real-world GitHub repositories. We frame bug-patching as a sequence-to-sequence learning task consisting of two steps: (i) denoising pretraining, and (ii) supervised finetuning on the target translation task. We show that pretraining on source code programs improves the number of patches found by 33% as compared to supervised training from scratch, while domain-adaptive pretraining from natural language to code further improves the accuracy by another 32%. We refine the standard accuracy evaluation metric into non-deletion and deletion-only fixes, and show that our best model generates 75% more non-deletion fixes than the previous state of the art. In contrast to prior work, we attain our best results when generating raw code, as opposed to working with abstracted code that tends to only benefit smaller capacity models. Finally, we observe a subtle improvement from adding syntax embeddings along with the standard positional embeddings, as well as with adding an auxiliary task to predict each token's syntactic class. Despite focusing on Java, our approach is language agnostic, requiring only a general-purpose parser such as tree-sitter.

@InProceedings{MAPS21p1, author = {Dawn Drain and Chen Wu and Alexey Svyatkovskiy and Neel Sundaresan}, title = {Generating Bug-Fixes using Pretrained Transformers}, booktitle = {Proc.\ MAPS}, publisher = {ACM}, pages = {1--8}, doi = {10.1145/3460945.3464951}, year = {2021}, } Publisher's Version

Learning to Make Compiler Optimizations More Effective
Rahim Mammadli, Marija Selakovic, Felix Wolf, and Michael Pradel
(TU Darmstadt, Germany; University of Stuttgart, Germany)
Because loops execute their body many times, compiler developers place much emphasis on their optimization. Nevertheless, in view of highly diverse source code and hardware, compilers still struggle to produce optimal target code. The sheer number of possible loop optimizations, including their combinations, exacerbates the problem further. Today's compilers use hard-coded heuristics to decide when, whether, and which of a limited set of optimizations to apply. Often, this leads to highly unstable behavior, making the success of compiler optimizations dependent on the precise way a loop has been written. This paper presents LoopLearner, which addresses the problem of compiler instability by predicting which way of writing a loop will lead to efficient compiled code. To this end, we train a neural network to find semantically invariant source-level transformations for loops that help the compiler generate more efficient code. Our model learns to extract useful features from the raw source code and predicts the speedup that a given transformation is likely to yield. We evaluate LoopLearner with 1,895 loops from various performance-relevant benchmarks. Applying the transformations that our model deems most favorable prior to compilation yields an average speedup of 1.14x. When trying the top-3 suggested transformations, the average speedup even increases to 1.29x. Comparing the approach with an exhaustive search through all available code transformations shows that LoopLearner helps to identify the most beneficial transformations in several orders of magnitude less time.

@InProceedings{MAPS21p9, author = {Rahim Mammadli and Marija Selakovic and Felix Wolf and Michael Pradel}, title = {Learning to Make Compiler Optimizations More Effective}, booktitle = {Proc.\ MAPS}, publisher = {ACM}, pages = {9--20}, doi = {10.1145/3460945.3464952}, year = {2021}, } Publisher's Version

Pure Tensor Program Rewriting via Access Patterns (Representation Pearl)
Gus Henry Smith, Andrew Liu, Steven Lyubomirsky, Scott Davidson, Joseph McMahan, Michael Taylor, Luis Ceze, and Zachary Tatlock
(University of Washington, USA)
Tensor kernels in machine learning (ML) often correspond to pure mathematical expressions, making term rewriting an attractive strategy for optimization and mapping to specialized hardware accelerators. However, existing ML intermediate representations (IRs) tend to either be pure but high-level, making low-level rewrites to hardware targets inexpressible, or low-level but impure, hampering the use of term rewriting altogether.
This paper introduces Glenside, a pure IR whose core abstraction—the access pattern—enables low-level, layout-aware, hardware-centric program rewrites. We demonstrate how term rewriting in Glenside can be used to map program fragments to hardware accelerator invocations and automatically discover classic data layout transformations like im2col. Glenside establishes a new foundation for exploring further term rewriting techniques in optimizing low-level tensor programs.

@InProceedings{MAPS21p21, author = {Gus Henry Smith and Andrew Liu and Steven Lyubomirsky and Scott Davidson and Joseph McMahan and Michael Taylor and Luis Ceze and Zachary Tatlock}, title = {Pure Tensor Program Rewriting via Access Patterns (Representation Pearl)}, booktitle = {Proc.\ MAPS}, publisher = {ACM}, pages = {21--31}, doi = {10.1145/3460945.3464953}, year = {2021}, } Publisher's Version

ControlFlag: A Self-Supervised Idiosyncratic Pattern Detection System for Software Control Structures
Niranjan Hasabnis and Justin Gottschlich
(Intel Labs, USA; University of Pennsylvania, USA)
Software debugging has been shown to utilize upwards of half of developers’ time. Yet, machine programming (MP), the field concerned with the automation of software (and hardware) development, has recently made strides in both research and production-quality automated debugging systems. In this paper we present ControlFlag, a self-supervised MP system that aims to improve debugging by attempting to detect idiosyncratic pattern violations in software control structures. ControlFlag also suggests possible corrections in the event an anomalous pattern is detected. We present ControlFlag’s design and provide an experimental evaluation and analysis of its efficacy in identifying potential programming errors in production-quality software. As a first concrete evidence towards improving software quality, ControlFlag has already found an anomaly in CURL that has been acknowledged and fixed by its developers. We also discuss future extensions of ControlFlag.

@InProceedings{MAPS21p32, author = {Niranjan Hasabnis and Justin Gottschlich}, title = {ControlFlag: A Self-Supervised Idiosyncratic Pattern Detection System for Software Control Structures}, booktitle = {Proc.\ MAPS}, publisher = {ACM}, pages = {32--42}, doi = {10.1145/3460945.3464954}, year = {2021}, } Publisher's Version

Predictive Data Locality Optimization for Higher-Order Tensor Computations
Tharindu R. Patabandi, Anand Venkat, Abhishek Kulkarni, Pushkar Ratnalikar, Mary Hall, and Justin Gottschlich
(University of Utah, USA; Intel Labs, USA; Intel Corporation, USA)
Automating locality optimization is still an open problem for compiler writers. Compiler-based approaches, guided by analytical cost models have achieved some success in matching high performance libraries on a restricted set of computations such as general matrix multiply (GEMM). On the other hand, library-based approaches may present some open scalability concerns. Recent developments in convolutional neural networks has seen an explosion of models, each with differing combinations of parameters. Manually tuning each of these configurations can take many development months. Further, these operations are called multiple times during machine learning training, which necessitates highly optimized implementations. 2D convolutional operators are unique in that they consist of 7-deep loop nests with different loops carrying reuse for different tensors, making the problem of identifying an optimal loop ordering hard. We devise a machine learning-based compiler which learns a regression model, correlating performance with the loop order. We integrate this model with other traditional compiler analysis for transformations such as loop unrolling and vectorization, relying on the MultiLevel Intermediate Representation (MLIR) compiler framework. We achieve an average speedup of 1.67x and 1.41x against oneDNN for 2D convolution forward and weight update kernels respectively. We are also at 0.88x and 0.96x the performance of oneDNN’s best performing implementation which applies additional data layout transformations.

@InProceedings{MAPS21p43, author = {Tharindu R. Patabandi and Anand Venkat and Abhishek Kulkarni and Pushkar Ratnalikar and Mary Hall and Justin Gottschlich}, title = {Predictive Data Locality Optimization for Higher-Order Tensor Computations}, booktitle = {Proc.\ MAPS}, publisher = {ACM}, pages = {43--52}, doi = {10.1145/3460945.3464955}, year = {2021}, } Publisher's Version